ATSS: Detecting AI-Generated Videos via Anomalous Temporal Self-Similarity

Abstract: AI-generated videos (AIGVs) have achieved unprecedented photorealism, posing severe threats to digital forensics. Existing AIGV detectors focus mainly on localized artifacts or short-term temporal inconsistencies, thus often fail to capture the underlying generative logic governing global temporal evolution, limiting AIGV detection performance. In this paper, we identify a distinctive fingerprint in AIGVs, termed anomalous temporal self-similarity (ATSS). Unlike real videos that exhibit stochastic natural dynamics, AIGVs follow deterministic anchor-driven trajectories (e.g., text or image prompts), inducing unnaturally repetitive correlations across visual and semantic domains. To exploit this, we propose the ATSS method, a multimodal detection framework that exploits this insight via a triple-similarity representation and a cross-attentive fusion mechanism. Specifically, ATSS reconstructs semantic trajectories by leveraging frame-wise descriptions to construct visual, textual, and cross-modal similarity matrices, which jointly quantify the inherent temporal anomalies. These matrices are encoded by dedicated Transformer encoders and integrated via a bidirectional cross-attentive fusion module to effectively model intra- and inter-modal dynamics. Extensive experiments on four large-scale benchmarks, including GenVideo, EvalCrafter, VideoPhy, and VidProM, demonstrate that ATSS significantly outperforms state-of-the-art methods in terms of AP, AUC, and ACC metrics, exhibiting superior generalization across diverse video generation models. Code and models of ATSS will be released at https://github.com/hwang-cs-ime/ATSS.

• The paper introduces ATSS, a multimodal framework that leverages anomalous temporal self-similarity for robust detection of AI-generated videos.
• It employs a triple-similarity representation with dedicated Transformers to model and fuse visual, textual, and cross-modal cues for binary classification of video authenticity.
• Extensive experiments demonstrate high performance and generalization across diverse benchmarks, though limitations exist for highly refined generative models such as Sora.

ATSS: Detecting AI-Generated Videos via Anomalous Temporal Self-Similarity

Motivation and Problem Formulation

The proliferation of high-fidelity AI-generated videos (AIGVs), produced by diffusion and generative frameworks such as Sora, Runway Gen-4, and ModelScope, has rendered traditional video forensics inadequate due to the rapid convergence of authentic and synthetic temporal coherence. Conventional detectors predominantly rely on localized artifacts or short-term inconsistencies, failing to address the systemic generative logic underpinning global temporal evolution. ATSS addresses this gap by positing the anomalous temporal self-similarity (ATSS) phenomenon as a systemic fingerprint inherent in anchor-driven AIGVs, contrasting the stochastic spatiotemporal dynamics of real-world video sequences.

Figure 1: ATSS motivation: Real videos display diffuse, low-magnitude self-similarity; AIGVs exhibit dense, high-intensity, regular cross-modal correlations due to anchor-driven generation.

Framework Design and Technical Implementation

ATSS formalizes the detection of AIGVs through a multimodal triple-similarity representation coupled with a cross-attentive fusion mechanism. Specifically, ATSS leverages frame-wise semantic reconstruction via an image captioning model, extracting visual and textual features per video frame. These representations are used to construct three inter-frame similarity matrices: visual (pixel-level), textual (semantic-level), and cross-modal (visual-textual alignment).

Each matrix quantifies temporal anomalies induced by deterministic anchor-driven generation, resulting in unnaturally regular correlations. Dedicated Transformer encoders process these matrices, modeling higher-order intra-modal and inter-modal temporal dependencies. Subsequently, a bidirectional cross-attentive fusion module integrates branch-specific cues, enabling robust binary classification of video authenticity.

Figure 2: ATSS pipeline: Multimodal similarity matrices are generated and processed by modality-specific Transformers, fused through cross-attentive modules, and aggregated for binary classification.

Quantitative Performance and Generalization

ATSS was evaluated on four large-scale benchmarks (GenVideo, EvalCrafter, VideoPhy, VidProM), spanning 40 generator subsets and diverse open-set scenarios. Strong numerical results include:

• GenVideo: Mean AP = 97.56%, AUC = 97.11%, ACC = 94.32%. ATSS surpasses state-of-the-art (SOTA) D3 by a margin of 5.40% in AP.
• EvalCrafter: AP = 98.61%, AUC = 98.35%, ACC = 96.01%. ATSS outperforms FID by 3.02% AP and 2.93% AUC.
• VideoPhy: AP = 96.38%, AUC = 95.34%. ATSS improves over FID by 1.69% AP and 1.01% AUC.
• VidProM: AP = 95.02%, AUC = 93.45%.

ATSS shows pronounced generalization across unseen generative models and challenging physical commonsense scenarios, although its performance diminishes on Sora and StreamingT2V subsets due to their highly realistic temporal dynamics and refinement strategies. These cases obscure the regular self-similarity artifacts exploited by ATSS, indicating inherent limitations of systemic self-similarity detection under near-physical video synthesis.

Ablation and Visualization

Ablation studies confirm the necessity and effectiveness of ATSS's core components:

• Removing visual, textual, or cross-modal branches induces mean AP/AUC/ACC drops of up to 52.97/47.83/28.18 percentage points, validating the complementary nature of modality-specific artifacts.
• Excluding cross-attentive fusion decreases AP by 0.46% and ACC by 3.67%, demonstrating the importance of bidirectional feature integration.
• Robustness to backbone choice: BLIP2-flan-t5-xl captioning with BLIP2-ITM-ViT-g encoding yields the highest overall metrics, emphasizing critical synergy between semantic granularity and cross-modal representation.

t-SNE projections of fused representations for real and synthetic videos exhibit robust separation across ten generator subsets, including Sora. Attention density visualizations further reveal structured, sustained patterns in AIGVs versus stochastic distributions in real video, confirming ATSS's sensitivity to global anchor-driven fingerprints.

Figure 3: t-SNE projections display strong separation between real and synthetic distributions for each generator subset.

Figure 4: Attention density maps exhibit high regularity in synthetic videos, stochastic variability in real samples across visual, textual, and cross-modal branches.

Implications, Limitations, and Future Directions

ATSS advances the forensic analysis of AI-generated video by shifting from artifact-centric tactics to global temporal modeling. Its multimodal approach ensures robustness across open-set semantic and physical diversity, setting new benchmarks for AP, AUC, and ACC. However, as generative models increasingly emulate stochastic, heterogeneous dynamics, notably in Sora or autoregressive-refinement engines, the efficacy of systemic self-similarity detection will be challenged. This suggests a need for detectors that can adaptively hybridize artifact and generative pattern analysis, possibly incorporating physics-driven or agentic paradigm shifts.

The practical utility of ATSS lies in its generalizability and interpretability across modalities, facilitating deployment in news verification, social media governance, and digital forensics. Theoretically, ATSS provides a framework for quantitative modeling of anchor-driven generative fingerprints, informing future research in adversarial robustness, prompt manipulation analysis, and synthetic media governance.

Conclusion

ATSS proposes a principled, multimodal framework for AIGV detection, operationalizing anomalous temporal self-similarity as a forensic fingerprint. By constructing and encoding triple-similarity matrices and leveraging cross-attentive fusion, ATSS demonstrates superior performance and generalization across extensive benchmarks. Its adaptability and robustness mark a significant development in systemic detection of generative video, with implications for forensic scalability and the evolution of adversarial generative techniques.