Detect Face Poses

This post will look into a technique that might help to detect face orientation or pose. And we will focus on only detecting three main poses which are:

1. Frontal Face.
2. Right Profile.
3. Left Profile.

Detecting tilted faces either front or back is out of the scope of this post. The technique detects only out-of-plane orientation estimation which is face rotation with respect to the y-axis (pan rotation). Since the technique relies heavily on the face detection model, we used MTCNN because it produces facial landmarks that we will use to detect face poses. Simply explained when a face is detected for the first time, we extract face landmarks (i.e., right eye, left eye, nose, left mouth, and right mouth) and from these points, we calculate the angels between the right eye, left eye, and the nose using a Cartesian coordinate system for Euclidean space 2D. Setting threshold ranges -which we already experimented with (look at the end of the post for full documentation about the experiment we did)- for the right eye angle and left eye angle can estimate if the face is left, or right profile, or frontal face.

Facial Landmarks and Illustration for eyes angles

As shown in the figure we are going to calculate the angles θ1 and θ2. After the model produces the landmarks we draw an “imaginary” line between three points i.e., the right eye, left eye, and the nose. Forming a triangle and through that triangle, we calculate the angles θ1 and θ2. In this step, a geometric object possesses both a magnitude and a direction. A vector can be pictured as an arrow or a line. Its magnitude is its length, and its direction is the direction that the line points. For more details, I would recommend you to see Manivannan's post about how to calculate angles with python, also thanks to him since his post helped us a lot.

How to calculate angles

And here is an example from Head Pose Image Database:

TL;DR:

Starting to import libraries:

from facenet_pytorch import MTCNN
from PIL import Image
from matplotlib import pyplot  as plt
import numpy as np
import math
import torch
import cv2

Now we have to declare the MTCNN model (i.e., detection model):

Define MTCNN model

Now let us define a function to calculate the angles:

Calculate the angles

For the next step it is just a visualization function:

Visualization

Now for the main “prediction” function:

Prediction step

We only accept faces with a probability of more than 90%, just to avoid false-positive detections.

The last step is just to run the code and pass the image path to the predFacePose function:

Prediction of face pose function

If you are interested to read more about this work please look into my full “paper” here. Or check out the repo:

GitHub - nawafalageel/Side-Profile-Detection: Detecting face profiles (i.e., frontal, left, and…