3D Facial Recognition using Deep Learning

With the continuous development of Deep Learning, facial recognition technologies based on CNNs (Convolutional Neural Networks) became the main method adopted in the field of facial recognition. However, despite its promising potential, it still presents many concerns regarding privacy and safety when applied. Moreover, the first facial recognition algorithms, the 2D ones, despite having good performance, were revealed to suffer from several factors like the environment’s lighting conditions, pose, and facial expression of the subject in which the recognition was applied, hence compromising the model’s accuracy and, consequently, its effectiveness.
This technology became increasingly popular, leading to its use in several environments like in airports and even at a large scale in the smartphone consumer industry, thus providing a more natural and spontaneous interaction between human and machine.
Throughout this dissertation, a computer vision system was developed using Artificial Intelligence techniques, mainly Deep Learning that is able to recognize faces in 3 dimensions (3D) and in real-time. To do so, both RGB and depth images were captured in several subjects, thus implementing extra security measures, that consequently filled in the gaps inherent to 2D methods mentioned above.
An image capture algorithm was developed, which required to use of specific hardware, namely a camera that incorporates a 3D sensor. Following this is the image processing of each captured image, and, after this, two CNNs were developed and trained independently, being one of them specifically developed for RGB images, while the other one was developed with emphasis on depth map recognition. After concluding the training process, both networks become suitable to recognize new images from the subjects with which the capture process was taken with, in a real-time scenario. The final goal is to apply the resulting trained models into Laboratory of Automation and Robotics’ anthropomorphic robot, CHARMIE, with the purpose of making it to perform different tasks based on the person it has been able to perform the correct recognition.