Automated Authentication of Audiovisual Contents: A Biometric Approach

Abstract

Following the introduction of image manipulation tools such as Adobe Photoshop in the early 2000s, the public trust in image authenticity dropped and the need for the development and deployment of image authentication techniques became apparent. Recently, we face a similar situation for video content as photo-realistic video manipulation tools like Deepfake are becoming available and within the reach of the general public as well as bad actors. In human to human communication, face and voice modalities play a crucial role, and not surprisingly, the same modalities are most under attack by forgers. Historically, the task of audiovisual content authentication was the focus of the field of multimedia forensics, with more than 15 years of accumulated literature. Following the increase in the popularity of biometric systems in practice, these systems have also faced similar challenges and felt the need for content authentication. Consequently, the field of presentation attack detection is born to protect biometric systems against fake biometric presentations. Due to the parallel nature of the presentation attack detection problem, defined as protecting a biometric system from presentation attacks, to the audiovisual content authentication problem, defined as protecting the viewer from fake content, the field of biometric presentation attack detection can provide a solid basis for approaching the multimedia authentication problem. The primary objective of this thesis is to address the audiovisual content authentication problem on the face modality by vulnerability assessment and mitigation of detected vulnerabilities with reliance on biometric and presentation attack detection knowledge. To this end, after producing a taxonomy of existing generation techniques, subjective tests are done to assess the vulnerability of viewers to the most prevalent generation techniques with reliance on data collected from the wild. Following this process, the generation techniques the viewers are most susceptible to were identified. The discovered vulnerabilities are then mitigated individually by the introduction of effective detection techniques that outperform existing solutions. Furthermore, the vulnerability of existing general-purpose detection methods was analyzed and it was discovered that these methods show limited generalization capacity when faced with new generation methods. To mitigate this vulnerability, with reliance on an anomaly extraction approach, a generalizable detection method is introduced and empirically evaluated against the state-of-theart methods. Additionally, all the datasets that are collected during the course of this thesis work are made publicly available to stimulate further research on this topic.