Instead of a straightforward application of deep network, SeetaFace Alignment implements a Coarse-to-Fine Auto-encoder Networks (CFAN) approach, which cascades a few Stacked Auto-encoder Networks (SANs) to progressively approach the accurate locations of the facial landmarks. The algorithm details can be found in our ECCV-2014 paper CFAN. The released SeetaFace Alignment is trained with more than 23,000 images and can accurately detect five facial landmarks, i.e., two eye centers, nose tip and two mouth corners. Please note that this implementation is slightly different from that described in the corresponding paper: only two stages are cascaded for the purpose of higher speed (more than 200 fps on I7 desktop CPU).
SeetaFace Alignment is implemented for running on CPU with no dependence on any third-party libraries. Currently it is only tested on Windows, but it does not include any Windows-specific headers. Versions for more platforms, e.g., Linux, will be released in the future. The open source is released under BSD-2 license (see LICENSE), which means the codes can be used freely for both acedemic purpose and industrial products.
To evaluate the performance of SeetaFace Alignment, experiments are conducted on AFLW, following the protocol published in [3]. The mean alignment errors normalized by the inter-ocular distance are shown in the following figure. As you can see, our SeetaFace Alignment achieves better accuracy than comparative methods.
Where LE, RE, N, LM, RM denote the left eye center, the right eye center, the nose tip, left mouth corner and right mouth corner respectively.
[1] Xuehan Xiong, Fernando De la Torre. Supervised descent method and its applications to face alignment. CVPR 2013
[2] Yi Sun, Xiaogang Wang, Xiaoou Tang. Deep Convolutional Network Cascade for Facial Point Detection. CVPR 2013
[3] Zhanpeng Zhang, Ping Luo, Chen Change Loy, Xiaoou Tang. Facial Landmark Detection by Deep Multi-task Learning. ECCV 2014
As for speed, it takes about 5 milliseconds per face to predict the 5 facial points, given a face bounding box reported by SeetaFace Detector, running on a single Intel 3.4GHz i7-3770 CPU with no parallel computing.
- Create a dll project: New Project -> Visual C++ -> Win32 Console Application -> DLL.
- (Optional) Create and switch to x64 platform.
- Add header files: all
*.hfiles ininclude. - Add source files: all
*.cppfiles insrcexcept for those insrc/test. - Define
SEETA_EXPORTSmacro: (Project) Properities -> Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions. - Build.
This version is developed to detect five facial landmarks, i.e., two eyes' centers, nose tip and two mouth corners.
To detect these facial landmarks, one should first instantiate an object of seeta::FaceAlignment with path of the model file.
seeta::FaceAlignment landmark_detector("seeta_fa_v1.0.bin");Then one can call PointDetectLandmarks(ImageData gray_im, FaceInfo face_info, FacialLandmark *points) to detect landmarks.
seeta::ImageData image_data(width, height);
image_data.data = image_data_buf;
image_data.num_channels = 1;
seeta::FaceInfo face_bbox;
seeta::FacialLandmark points[5];
landmark_detector.PointDetectLandmarks(image_data, face_bbox, points);Where image_data denotes an input gray image, face_bbox is the face bouding box detected by [Seeta - Face Detection] (https://github.com/seetaface/SeetaFaceEngine/tree/master/FaceDetection), The landmarks detection results are returned in points. An example can be found in file face_alignment_test.cpp.
If you use the code in your work, please consider citing our work as follows:
@inproceedings{zhang2014coarse,
title={Coarse-to-fine auto-encoder networks (cfan) for real-time face alignment},
author={Zhang, Jie and Shan, Shiguang and Kan, Meina and Chen, Xilin},
booktitle={European Conference on Computer Vision},
year={2014},
organization={Springer}}
SeetaFace Alignment is released under the BSD 2-Clause license.