Robust algorithms for 2D and 3D Face Morphing Attacks: Generation and Detection
Abstract
Biometric Authentication (Biometrics) is a powerful tool that authenticates individuals using digital means, which includes biological or behavioral characteristics. Biometrics harnesses biological features such as fingerprints, face, hand geometry, speech, iris, and fingerphoto. Face and finger modalities have generated the interest of biometric researchers thanks to their ease of use and high accuracy. Face biometric modality, in particular, is easy to use as it can be acquired passively. Furthermore, Face Recognition Systems (FRS) excel in real-world environments, thanks to the advancements in deep learning. However, it’s important to know that FRS are not immune to attacks. They are vulnerable to various types of attacks, including presentation attacks and morphing attacks to a large extent and deepfakes to a smaller extent.
This thesis focuses on Face Morphing Attacks (FMA), an active area of research in Biometrics. An FMA can be generated by linearly blending facial images in the color domain from two contributory data subjects. FMA has shown vulnerabilities in FRS when evaluated automatically by software or manually by human observers. Thus, FMA is a strong attack on FRS. Hence, detecting FMA is an actual problem from a security standpoint. Most FMA systems currently use full facial images from the two contributory data subjects. However, the part-based face morphing/compositing problem has received little attention, i.e., using facial parts from the two contributory data subjects to generate an FMA. Further, due to Generative Adversarial Networks (GANs), generating full photo-real synthetic faces or completing partial facial images is possible due to deep learning-based image synthesis advances. Thus, part-based facial morphing using the advances of deep learning could be a fruitful area of research.
Motivated by the challenges arising from attacks toward FRS, the thesis focus is two-fold. The first is to increase the attack strength by generating higher quality attacks and the second is to advance the mitigation measures, a.k.a countermeasures for the generated attacks. We focus on Morphing Attacks, which include generation and detection, known as Morphing Attack Detection (MAD). Further, evaluating vulnerabilities imposed by part-based facial morphing could be a novel area of research and we have performed an extensive assessment of this nascent area. Currently, the critical problem is performing robust MAD in real-world environments, which have the challenges of facial pose, expression, illumination, image quality, print-scan variations and image capture distance. This brings us to building robust classifiers for the facial morphing problem. Morphing has been evaluated on face images, i.e., 2D image data. We generalize Morphing to 3D by performing first-of-its-kind 3D Morph operations on point clouds and present the results on both generation and detection. We generate a GAN-based facial composite of face images from face images of two contributory data subjects, with an extensive evaluation of different facial regions.
Has parts
Paper 1: Singh, Jag Mohan; Ramachandra, Raghavendra; Bylappa Raja, Kiran; Busch, Christoph. Robust Morph-Detection at Automated Border Control Gate using Deep Decomposed 3D Shape and Diffuse Reflectance. The 15th International Conference on SIGNAL IMAGE TECHNOLOGY & INTERNET BASED SYSTEMS (SITIS); 2019-11-26 - 2019-11-29. Copyright © 2019 IEEE.Paper 2: Singh, Jag Mohan; Ramachandra, Raghavendra. Reliable Face Morphing Attack Detection in On-The-Fly Border Control Scenario with Variation in Image Resolution and Capture Distance, In IEEE International Joint Conference on Biometrics (IJCB)) 2022), Abu Dhabi, UAE, pp. 1-10, IEEE. Copyright © 2022 IEEE.
Paper 3: Singh, Jag Mohan; Venkatesh, Sushma Krupa; Ramachandra, Raghavendra. Robust Face Morphing Attack Detection Using Fusion of Multiple Features and Classification Techniques. I: 2023 26th International Conference on Information Fusion (FUSION). IEEE (Institute of Electrical and Electronics Engineers) 2023 ISBN 979-8-89034-485-4. Copyright © 2023 IEEE. Available at: http://dx.doi.org/10.23919/FUSION52260.2023.10224168
Paper 4: Singh, Jag Mohan; Venkatesh, Sushma Krupa; Ramachandra, Raghavendra. Robust Face Morphing Attack Detection Using Fusion of Multiple Features and Classification Techniques. I: 2023 26th International Conference on Information Fusion (FUSION). IEEE (Institute of Electrical and Electronics Engineers) 2023 ISBN 979-8-89034-485-4. Copyright © 2023 IEEE. Available at: http://dx.doi.org/10.23919/FUSION52260.2023.10224168
Paper 5: Singh, Jag Mohan; Ramachandra, Raghavendra. Deep Composite Face Image Attacks: Generation, Vulnerability and Detection. IEEE Access 2023 ;Volum 11. s. 76468-76485. Published by IEEE. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License CC BY NC-ND. Available at: http://dx.doi.org/10.1109/ACCESS.2023.3261247
Paper 6: Singh, Jag Mohan; Ramachandra, Raghavendra. 3-D Face Morphing Attacks: Generation, Vulnerability and Detection. IEEE Transactions on Biometrics, Behavior, and Identity Science 2024 ;Volum 6.(1) s. 103-117. This article is licensed under a Creative Commons Attribution 4.0 International License CC BY NC-ND. Available at: https://doi.org/10.1109/TBIOM.2023.3324684
Paper 7: Jag Mohan Singh and Raghavendra Ramachandra. 3D Face Morphing Attack Generation using Non-Rigid Registration. 18th IEEE International Conference on Automatic Face and Gesture Recognition, 2024. Copyright © 2024 IEEE.