Top Research Papers on Deepfake Detection

This article explores the creation and detection of deepfakes and provides an in-depth view as to how these architectures work and the current trends and advancements in this domain.

Although Deep fake detection is extremely difficult and still an unsolved problem, a Deepfake detection model trained only on the DFDC can generalize to real "in-the-wild" Deepfake videos, and such a model can be a valuable analysis tool when analyzing potentially Deepfaked videos.

A survey on state-ofthe-art deepfake generation methods, detection methods, and existing datasets is made and future trends on deepfake detection can be efficient, robust and systematical detection methods and high quality datasets.

A new multi-attentional deepfake detection network that consists of three key components: multiple spatial attention heads to make the network attend to different local parts, a new regional independence loss and an attention guided data augmentation strategy, and state-of-the-art performance.

A systematic literature review (SLR) is conducted, summarizing 112 relevant articles from 2018 to 2020 that presented a variety of methodologies in Deepfake detection and concluding that the deep learning-based methods outperform other methods in Deep fake detection.

The first dataset of real deepfake tweets, TweepFake, is collected and 13 deepfake text detection methods are evaluated to both demonstrate the challenges that Tweepfake poses and create a solid baseline of detection techniques.

This paper considers the deepfake detection technologies Xception and MobileNet as two approaches for classification tasks to automatically detect deepfake videos and developed a voting mechanism that can detect fake videos using the aggregation of all four methods instead of only one.

Inspired by the success of masked modeling, Masked Relation Learning is proposed which decreases the redundancy to learn informative relational features and outperforms the state of the art by 2% AUC on the cross-dataset DeepFake video detection.

The research emphasises the dominance of deep learning-based methods in detecting deepfakes despite their computational efficiency and generalisation limitations, however, it also acknowledges the drawbacks of these approaches, such as their limited computing efficiency and generalisation.

Novel synthetic training data called self-blended images (SBIs) to detect deepfakes are presented and extensive experiments show that the method improves the model generalization to unknown manipulations and scenes.

A comprehensive review of deepfake generation and detection and the different ML/DL approaches to synthesize deepfake contents and a unique case study, IBMM, is discussed, which presents a multi-modal overview of deepfake detection.

A novel temporal modeling paradigm in TIM is presented by exploiting the temporal difference over adjacent frames along with both horizontal and vertical directions and the ISM simultaneously utilizes the spatial information from SIM and temporal information from TIM to establish a more comprehensive spatial-temporal representation.

This study provides a comprehensive overview of deepfake techniques and facilitates the development of new and more robust methods to deal with the increasingly challenging deepfakes.

Deep learning has been successfully applied to solve various complex problems ranging from big data analytics to computer vision and human-level control. Deep learning advances however have also been employed to create software that can cause threats to privacy, democracy and national security. One of those deep learning-powered applications recently emerged is deepfake. Deepfake algorithms can create fake images and videos that humans cannot distinguish them from authentic ones. The proposal of technologies that can automatically detect and assess the integrity of digital visual media is ther...

This work presents a model for audio deepfake classification which has an accuracy comparable to the traditional CNN models like VGG16, XceptionNet, etc.

Deep learning has been successfully applied to solve various complex problems ranging from big data analytics to computer vision and human-level control. Deep learning advances however have also been employed to create software that can cause threats to privacy, democracy and national security. One of those deep learning-powered applications recently emerged is "deepfake". Deepfake algorithms can create fake images and videos that humans cannot distinguish them from authentic ones. The proposal of technologies that can automatically detect and assess the integrity of digital visual media is th...

This survey will summarize the DeepFake detection methods in face images and videos on the basis of their results, performance, methodology used and detection type, and review the existing types of DeepFake creation techniques and sort them into five major categories.

The free access to large-scale public databases, together with the fast progress of deep learning techniques, in particular Generative Adversarial Networks, have led to the generation of very realistic fake content with its corresponding implications towards society in this era of fake news. This survey provides a thorough review of techniques for manipulating face images including DeepFake methods, and methods to detect such manipulations. In particular, four types of facial manipulation are reviewed: i) entire face synthesis, ii) identity swap (DeepFakes), iii) attribute manipulation, and iv...

When used maliciously, deepfake can pose detrimental implications to political and social forces including reducing public trust in institutions, damaging the reputation of prominent individuals, and influencing public opinions. As there is currently no specific law to address deepfakes, thus deepfake detection, which is an action to discriminate pristine media from deepfake media, plays a vital role in identifying and thwarting deepfake. This paper provides readers with a comprehensive and easy-to-understand state-of-the-art related to deepfake generation and detection. Specifically, we provi...

This paper designs an explicit identity contrast (EIC) loss and an implicit identity exploration (IIE) loss, which supervises a CNN backbone to embed face images into the implicit identity space and proposes a novel implicit identity driven framework for face swapping detection.

The rapid advancement of deep learning models that can generate and synthesis hyper-realistic videos known as Deepfakes and their ease of access to the general public have raised concern from all concerned bodies to their possible malicious intent use. Deep learning techniques can now generate faces, swap faces between two subjects in a video, alter facial expressions, change gender, and alter facial features, to list a few. These powerful video manipulation methods have potential use in many fields. However, they also pose a looming threat to everyone if used for harmful purposes such as iden...

This work proposes a method which uses the recognition signals from two networks to detect discrepancies between the two regions of the face, providing a complementary detection signal that improves conventional real versus fake classifiers commonly used for detecting fake images.

This study will be beneficial for researchers in this field as it will cover the recent state-of-art methods that discover deepfakes videos or images in social contents and help comparison with the existing works because of the detailed description of the latest methods and dataset used in this domain.

This paper introduces a new problem of partial face attack in DeepFake video, where only video-level labels are provided but not all the faces in the fake videos are manipulated, and proposes a sharp MIL (S-MIL), which builds direct mapping from instance embeddings to bag prediction, rather than from instanceEmbedded to instance prediction and then to bag Prediction in traditional MIL.

A disentanglement framework that decomposes image information into three distinct components: forgery-irrelevant, method-specific forgery, and common forgery features is proposed and can perform superior generalization than current state-of-the-art methods.

A novel and improved deep-CNN (D-CNN) architecture for deepfake detection with reasonable accuracy and high generalizability is presented.

This work introduces simple, yet surprisingly efficient digital forensic methods for audio spoof and visual deepfake detection that combine convolutional latent representations with bidirectional recurrent structures and entropy-based cost functions.

Hyper-realistic manipulations of audio-visual content, i.e., deepfakes, present new challenges for establishing the veracity of online content and suggest that people adopt a “seeing-is-believing” heuristic for deepfake detection while being overconfident in their detection abilities.

An Interpretable Spatial-Temporal Video Transformer (ISTVT) is proposed, which consists of a novel decomposed spatial-temporal self-attention and a self-subtract mechanism to capture spatial artifacts and temporal inconsistency for robust Deepfake detection.

An Audio-Visual Joint Learning for Detecting Deepfake (AVoiD-DF), which exploits audio-visual inconsistency for multi-modal forgery detection and yields superior generalization on various forgery techniques.

A new approach to detect Deepfakes generated through the generative adversarial network (GANs) model via an algorithm called DeepVision to analyze a significant change in the pattern of blinking, which is a voluntary and spontaneous action that does not require conscious effort.

A Multi-modal Multi-scale TRansformer (M2TR), which operates on patches of different sizes to detect local inconsistencies in images at different spatial levels, and learns to detect forgery artifacts in the frequency domain to complement RGB information through a carefully designed cross modality fusion block.

This paper summarizes what has been in the critical discussion about the problems, opportunities, and prospects of Deepfake technology and presents how a better and more robust Deepfake detection method can be designed to deal with fake content.

The proposed DeepfakeStack technique combines a series of DL based state-of-art classification models and creates an improved composite classifier that outperforms other classifiers by achieving an accuracy of 99.65% and AUROC of 1.0 in detecting Deepfake.

Despite the introduction of channel and compression variability which compound the difficulty, results for the logical access and deepfake tasks are close to those from previous ASVspoof editions.

It is found that manipulations designed to disrupt visual processing of faces hinder human participants’ performance while mostly not affecting the model’s performance, suggesting a role for specialized cognitive capacities in explaining human deepfake detection performance.

A summary of the top-performing systems for each task, new analyses of influential data factors and results for hidden data subsets, and a road-map for the future of ASVspoof 2021 are provided.

A new forensic technique able to detect fake and original video sequences is proposed; it is based on the use of CNNs trained to distinguish possible motion dissimilarities in the temporal structure of a video sequence by exploiting optical flow fields.

This work proposes an efficient and robust framework named LRNet for detecting Deepfakes videos through temporal modeling on precise geometric features, which is lighter-weighted and easier to train and has shown robustness in detecting highly compressed or noise corrupted videos.

This study gives a complete assessment of the literature on deepfake detection strategies using DL‐based algorithms, and suggests that the Conventional Neural Networks (CNN) methodology is the most often employed DL method in publications.