Face tracking & computer vision

26 October 2009 | Maurice Chu

This video clip demonstrates the processing of video to track six parts of the face — left eye, right eye, left eyebrow, right eyebrow, nose, and mouth.

The output of the face tracker is a cloud of points representing the location of the six face parts in image coordinates. These coordinates can be used to determine the 3D orientation of the face relative to the video camera.

The face tracker algorithm:

• runs in real-time (currently at about 10 fps);
• is people-generic;
• requires no initial calibration; and
• overcomes the challenge of handling “deformable movements” such as putting on glasses, blinking eyes, and moving lips.

We developed this algorithm to use in the Countertop Responsive Mirror, as seen in this video post at BoingBoing Gadgets “PARC’s Responsive Mirror = Every Girl’s Shopping Fantasy Come True“.  The technology can also be used to detect a player’s movement when interacting with a video game.

Comments (1)