Neurokai’s services offer different solutions for industrial and academic researchers. Our Experience Lab service focuses on providing a custom solution for emotion recognition based on a multimodal system (EEG, ECG, GSR and facial). In order to develop our vision-based emotion recognition system, integrated within the multimodal one, we analysed several libraries for face detection. The main objective of our research at that point was to find a robust computer vision library to be integrated in our multimodal emotion recognition system, which includes vision as well as electrophysiological modalities. In this post we will share with you what computer vision libraries we found in the community and how the methods performed, when it was possible, after running them on our own video recordings. It is worth mentioning that in our videos people were recorded wearing the Enobio cap and the videos were performed with low lighting conditions, which made the face detection task slightly more difficult.
Betaface (shareware). The Betaface company is mostly focused on biometric services. The company offers a complete face library SDK (software development kit). This SDK allows face detection, facial points detection, face analysis and classification, face comparison and person identification in images. They also offer customized software development. While the results shown in their website seem to work, the results we obtained using their online demo tended to fail in our videos. The detector was confused sometimes with the Enobio Cap and one of the electrodes located in the face. In some cases, the library was not even able to detect the face.
Beyond-Reality-face Nxt (shareware). The SDK developed by Tastenkunst permits point tracking, face detection, face shape tracking and placing 3D objects on the face. After examining the online demo, it seems they rely on the FaceTracker library used by [1]. The approach tends to lose the eyebrows, and sometimes misses the face. We were not able to test the software in our videos. Nevertheless, we reckon the approach may perform poorly.
Cognitec Systems (shareware). This company, Cognitec Systems, has a SDK that enables clients to develop new face recognition and face detection applications. So far, it seems they do not have any demo or video available. It would be great if the company could provide a demo.
Keylemon (shareware). This is a software application developed by KeyLemon for authentication. We tested their trial version in our computer, it seems to work fine for authentication, but no SDK is mentioned in their website.
VeriLook (shareware). We downloaded and used the demo available in the Neurotechnology website, which has an enrolment/authentication application. It did not detect the face in most of the cases. Their SDK includes facial points extraction.
Luxand FaceSDK (shareware). Their SDK permits to detect and recognise faces. Moreover, the library also allows to extract facial points. We have evaluated their SDK through the demo available in their website. This demo allows both to evaluate single images and connect any camera. The results in both cases were quite promising. In practically all the videos we have the library was able to perform perfectly.
Shore (shareware). The Fraunhofer Institute ISS has an object recognition library which includes detection and tracking of face parts, recognition of facial expressions (“Happy”, “Surprised”, “Angry” and “Sad”), age estimation, gender classification, and other features. We have been testing its demo. The demo, similar to the FaceSDK, allows to evaluate videos, single images and even real-time image sequences coming from a camera. The positive thing about this library is that it includes facial-expression recognition. We have tried both, the web camera, and the single image evaluation. In some cases the face detector fails. On the other hand, the face-expression recognition system does not always work perfectly, which underscores how challenging the problem of facial-expression recognition is. The emotion recognition system has recently been integrated in the GoogleGlasses.
Discrete Area Filters (DAF, freeware). This is a face detection library developed by Semantic Vision Technologies which provides the fiducial points corresponding to the eyes, mouth and nose. We ran the available binary on different still images, and it did not detect most of the faces, and when it did, the results were not as satisfactory as one would expect. Basically, some features points were not correctly located.
Fdlib (Face Detection library). This library developed by Wolf Kenzle only provides face detection. There is nothing regarding facial points detection and tracking, which depending on the use can be an important drawback (e.g. emotion recognition). When performing the face detection using the web cam, the detector was performing fine.
Flandmark (freeware). The Center of Machine Perception in the Czech Technical University has an open source library implementing a facial landmark detector in static images. The method detected the face in many cases, but in others, when detected, the facial points were not in place.
So far, after conducting this study we could state that face detection remains a challenging problem to be solved in spite of it being considered in the computer vision community as an already solved issue for such a long time. Nonetheless, thanks to the efforts made by the scientific research community and private companies it seems we are heading in the right direction. After this review we concluded that Luxand FaceSDK was the best library in terms of face detection and tracking.
If you want to share some suggestions or comments (e.g. new available libraries), please leave them below.
[1] el Kaliouby, R. A. Mind-reading machines: automated inference of complex mental states.
Technical Report. Num. 636, 2005.