Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos

Abstract: Recent advances in media generation techniques have made it easier for attackers to create forged images and videos. State-of-the-art methods enable the real-time creation of a forged version of a single video obtained from a social network. Although numerous methods have been developed for detecting forged images and videos, they are generally targeted at certain domains and quickly become obsolete as new kinds of attacks appear. The method introduced in this paper uses a capsule network to detect various kinds of spoofs, from replay attacks using printed images or recorded videos to computer-generated videos using deep convolutional neural networks. It extends the application of capsule networks beyond their original intention to the solving of inverse graphics problems.

• The paper presents a novel approach by adapting capsule networks for comprehensive forgery detection in digital media.
• It integrates latent features from VGG-19 and incorporates Gaussian noise during training to reduce overfitting and enhance robustness.
• The method achieved remarkable results, including zero HTER on the Idiap REPLAY-ATTACK dataset and perfect accuracy in distinguishing CGI from PI images.

CAPSULE-FORENSICS: Detecting Forged Media Using Capsule Networks

The paper "CAPSULE-FORENSICS: Using Capsule Networks to Detect Forged Images and Videos" presents a novel approach to forensics tasks utilizing capsule networks to detect manipulated media. The research addresses a critical issue as advancements in media generation techniques have enabled the production of high-quality forged images and videos, posing risks in authentication systems and the proliferation of fake news.

Core Contributions

The primary contribution of this work is the adaptation of capsule networks, initially intended for computer vision tasks, to the domain of digital forensics. Capsule networks, introduced by Hinton et al., offer a sophisticated approach to modeling spatial hierarchies, employing dynamic routing to achieve superior performance compared to traditional CNNs.

The proposed method is designed to detect a variety of forgery types, including replay attacks, facial reenactments, and entirely computer-generated media. By leveraging the latent features extracted from part of the pre-trained VGG-19 network, the capsule network scrutinizes these features to effectively differentiate between authentic and manipulated content.

Methodology and Design

The design of the Capsule-Forensics approach involves a sequence of processing phases for both images and videos. For videos, the methodology includes a frame-level analysis where classification probabilities are averaged to generate a final decision. The capsule network architecture consists of three primary capsules and two output capsules directed towards real and fake identifications. An enhancement in this paper is the incorporation of Gaussian random noise during training, which aids in reducing overfitting and enhances detection capability.

Results and Evaluation

The empirical evaluation demonstrates the method's efficacy through comprehensive tests across four major datasets. Notably, the proposed method with noise (Capsule-Forensics-Noise) achieved superior results with an HTER of zero on the Idiap REPLAY-ATTACK dataset. Furthermore, the method exhibited high accuracy rates in detecting deepfake manipulations and facial reenactment, surpassing several existing state-of-the-art techniques.

Particularly impressive was the perfect accuracy in distinguishing between full-size CGI and PI images, underscoring the method's robustness in identifying synthetic content.

Implications and Future Directions

Capsule-Forensics provides a promising direction for future research in multimedia forensics, particularly in environments where new types of forgeries rapidly emerge. Given its broad application scope, this method could significantly impact systems reliant on image and video integrity, such as authentication protocols in security domains and content verification in social media platforms.

Future research should explore the method's resilience to adversarial attacks and consider enhancements to the capsule architecture for even greater robustness. An additional focus will be on tackling mixed attacks that combine multiple forgery types, encouraging further exploration of how capsule networks can be refined for these complex challenges.

In summary, the paper offers a substantial contribution to forgery detection, showcasing the potential of capsule networks in a burgeoning field of digital forensics.