Face detection is a computing technology term, that is used when software is used to determine the existence, location and size of a human face in a particular photo. The software is clever enough to detect the facial features, while at the same time ignoring other objects like trees, buildings and bodies.

A person’s face is actually a rich source of information. It is possible to tell, simply from a human face, whether a person is male or female, approximately how old they are, and from their expression, how they are feeling.

Human beings can process faces very quickly. It only takes you a split second to determine the key attributes of anyone you look at. In the case of computers, however, it is a somewhat more complex process.

In simple terms, a computer attempting face detection will begin by examining either a still photo or a video image. It then has the task of determining if there are any faces present in that picture, distinguishing these faces from whatever else is in the background. It has to do this regardless of illumination, orientation or camera distance.

There are a number of methods which a computer can use to achieve this.

Finding Faces in Images with Controlled Backgrounds

There is a relatively straight-forward method that can be used when you simply have a frontal face image against a plain background. In this case the software can easily remove the background, leaving face boundaries. If software uses this approach it tends to have a number of different classifiers for detecting different types of front-on faces, along with some for profile faces. It will attempt to detect eyes, a nose, a mouth, and in some cases even a whole body for pictures that show more than just a face.

Finding Faces by Color

This is a relatively simplistic method that a computer can use to look for faces. It obviously requires that the photos or video images used be color. The software scans the picture looking for areas that are of a typical skin color, then looking for face segments. A problem with this technique is that skin color varies from race to race, and this method does not work as well for all skin colors. Varying lighting conditions changes the exact hue of a person’s skin in the picture, and that can have a major effect on facial detection, too.

Finding Faces by Motion

When you are using video images you can use movement as a guide. Faces are usually moving in real-time videos, so one option is for the software to capture the moving area. Of course, other parts of videos also move, so the software needs to look for particular reference points to indicate that it is actually a face that is moving.

One specific face movement is blinking. If the software can determine a regular blinking pattern (two eyes blinking together, symmetrically positioned) then this is a good indication that there is a face. From this regular blinking pattern the computer can determine the area of the video image that is actually the face, using one of a number of face models.

There will be a number of face models in the software, containing the appearance, shape and motion of faces. There are actually a variety of different face shapes, roughly categorised as oval, rectangle, round, square, heart and triangle.

As well as blinking, there are various other motions that signpost to the computer that the image may contain a face. These include raised eyebrows, flared nostrils, wrinkled foreheads and opened mouths.

Once one of these actions is detected, the computer will pass their face models over the video image and try and determine a facial match.

Once a face is detected, and a particular face model matched with a particular movement, the model is laid over the face, enabling face tracking to pick up further face movements.

Differences between Face Detection and Face Recognition

The terms face detection and face recognition have tended to be misused, particularly by media outlets, who often do not distinguish between the two actions. The reality is that they are different, although for face recognition to occur, there must first be face detection.

As referred to above, face detection is the process computer software needs to go through to determine whether there actually is one or more faces in a photo or video image. It does not determine whose faces are in the picture, only whether there actually faces there. Therefore facial detection by itself does not remember or store details of faces. If the software detects a face of a particular person in one picture, and then later detects that same face in another picture, it does not recognize that they are the same person - it only recognizes that there is a face of someone in each picture.

It may be able to pick up some demographic information, for instance the age or gender of the person in each picture, but that is all. Facial detection software by itself is not capable of recognizing individuals.

On the other hand, face recognition is all about establishing identity. The aim of face recognition software is to make a positive identification of a face in a photo or video image against a pre-existing database of faces. For face recognition to be successful faces need to be enrolled into the system, to create the database. At this point, the software will determine unique facial identifying features, which is what it will actually store in the database. Later, when other pictures are used to establish identity, the software will break the new picture down into its key features and compare these features against the information stored in the database. If it finds a match, at a high enough level of confidence, it will have “recognized” that particular face in the picture.

Real-Time vs Server Side Face Detection/Recognition

There have been a number of developments in face detection and recognition technology in recent years. Different technologies have been developed to solve different problems.

The Kairos Face Recognition API was developed to allow third party developers to submit still photos for analysis and tagging. The apps that these developers have built use the Kairos API to send an image file containing faces to the Kairos servers. If they use it for recognition purposes, they first need to enroll and identify facial images. Once they have created a database of identified faces, they can then send through further photos to compare against that database for face recognition purposes. Obviously, in both the enrollment and the recognition stages, face detection has to occur first.

This server side technique provides a high level of recognition accuracy, but it does not allow for real-time face detection and tracking due to the latency of the network connection.

Real-time face detection, like that used in the Kairos Human Analytics SDK runs locally on the client side and this makes real-world detection of multiple faces simultaneously possible. The processing time needed for analysis is less than 100 milliseconds.

Benefits of Face Detection

Face detection is used for a number of purposes.

As stated above, it is the first essential step needed in any application that uses face recognition. With no detection, there can be no recognition. Face recognition itself has been found very useful in areas like automated access control and computer vision communication.

Face detection has been used to assist with energy conservation. For instance there are now smart televisions sets that can sense when they are being watched, and adjust their brightness accordingly.

Marketers have found real benefit from much of the real-time analytics in areas like gender and age recognition. This has enabled them to modify their displays to attract the people actually in the vicinity. They are also able to gauge consumers’ interest in displays and other marketing materials thanks to detection of facial expressions (whether they are happy, surprised, angry or sad, for example).

Have a look at how the Kairos Face Detection and Face Recognition products can help you - Get a free API key.