DetectZoo: A Unified Toolkit for AI-Generated Content Detection Across Text, Audio, and Image Modalities

Abstract: The growing popularity and capacity of generative models have eroded the distinction between human and machine-generated content, motivating a growing body of work on detection across text, images, and audio. Most available detectors are either commercial software or, if open-source, come with incompatible codebases with bespoke preprocessing, evaluation protocols, and evaluation metrics, which make their adoption, fair comparison, and reproduction quite difficult. To address this critical gap, we introduce DetectZoo, a first-of-its-kind, extensible toolkit designed to provide a unified interface for AI-generated content detection across text, audio, and image modalities. DetectZoo standardizes the complete empirical pipeline, from data ingestion and preprocessing to model assessment, offering researchers a cohesive framework to benchmark state-of-the-art detectors systematically. By integrating diverse public datasets and baseline detection algorithms under a single, unified API, our toolkit facilitates rigorous and reproducible evaluation. DetectZoo provides reference implementations of 61 detectors, native loaders for 22 benchmark datasets, and a standardized evaluation pipeline that reports multiple metrics through a common interface. Each detector is self-contained yet accessible through the same interface, automatically caches pretrained weights, and reproduces the original published results. DetectZoo lowers the barrier to entry for multi-modal AI forensics, enabling researchers to identify performance gaps across domains and accelerating the development of robust, generalizable detection techniques. The open-source repository and comprehensive documentation are publicly available at https://github.com/sadjadeb/DetectZoo, and the package can be installed via pip install detectzoo.

• The paper introduces a unified API that standardizes 61 AI-generated content detectors across text, audio, and image modalities for reproducible evaluations.
• The paper presents a modular evaluation suite with automated pipelines computing metrics like AUROC, AUPR, and EER for systematic comparison.
• The paper highlights current limitations and outlines future directions, including expansion to video modalities and enhanced real-world detection robustness.

DetectZoo: A Unified Evaluation Infrastructure for Multi-Modal AI-Generated Content Detection

Motivation and Problem Statement

The increasing realism of generative models for text, image, and audio has blurred the line between human and synthetic content, raising acute challenges in AI safety, forensics, and security. The corresponding research community has developed a diverse array of detection algorithms, but this proliferation has produced highly fragmented and modality-specific codebases, benchmarks, and evaluation protocols. The absence of standardized APIs, preprocessing, and reproducible metric computation leads to non-commensurate results, significant reproducibility barriers, and hinders cross-modal insights. DetectZoo directly addresses this fragmentation by providing a unified, rigorously validated toolkit for benchmarking AI-generated content detectors across text, audio, and image, facilitating both systematic comparison and reproducibility.

Figure 1: Overview of DetectZoo—standardized benchmarking for AI-generated content detectors across text, image, and audio, via a unified end-to-end API.

Architecture and Technical Contributions

The primary contribution of DetectZoo is a modular, extensible evaluation suite covering 61 detectors over three modalities (text, audio, image) and 22 curated benchmarks. Each detector, regardless of input type or model family, integrates via a single entry-point predict interface with standardized outputs (binary label, confidence, anomaly score, and detector-specific metadata). Dataset ingestion is abstracted—native loaders handle canonical splits, formatting, and automatic downloading, exporting a simple, uniform data structure for seamless evaluation regardless of artifact type.

The core technical features are:

• Unified Multi-Modal API: All detectors are accessed via load_detector and a strictly typed predict method. This abstracts device selection, tokenization, modality-specific normalization, and backend details, enabling agnostic swapping and benchmarking.
• Self-Contained Detector Implementations: All detectors ship with versioned weights and dependency management, validated for numerics against original published results within strict tolerances.
• Automated, Standardized Evaluation Pipelines: Metrics (AUROC, AUPR, EER, accuracy, recall, $F_1$) are computed in threshold-independent and dependent variants, logged to JSON with full experimental metadata for reproducibility.
• Extensible Registration and Dataset Management: Detector and dataset contributions are registered via lightweight decorators (e.g., @register_detector, @register_dataset) with validation hooks, greatly lowering the friction for future expansion.
• Open-Source Release and Distribution: Available as a PyPI package (detectzoo) with modular installation for text, vision, or audio dependencies, released under an open source license and with comprehensive documentation.

Empirical Results and Benchmarking

DetectZoo conducts large-scale reproduction studies validating all integrated detectors against their original publications. For text detection, benchmarked on tasks including paraphrase, polish, and direct generation (across XSum, SQuAD, WritingPrompts, RAID, etc.), ImBD and ReMoDetect yield consistently high AUROC, especially for direct generation. Statistical baselines, e.g., log-likelihood, log-rank, are sensitive to generation semantics and source LLM, with GPT-4o and Qwen2-7B outputs providing the most robust obfuscation. Results confirm that task semantics and model choice are major detection confounds and that aggregate performance metrics mask task-specific vulnerabilities.

For images, cross-dataset benchmarks (ForenSynths, Self-Synthesis, UFD, GenImage, Chameleon) demonstrate that hybrid approaches leveraging both semantic and frequency/pixel-level cues—such as FatFormer, SAFE, Co-SPY, and AIDE—generalize substantially better in both intra- and cross-architecture settings than pure artifact detectors. Notably, reconstruction-based methods (AEROBLADE) show non-uniform transfer; their performance is strong for diffusion-centric datasets (e.g., GenImage SDv1.4, accuracy $> 0.96$), but degrades or collapses for GAN-generated images. All methods experience severe degradation on real in-the-wild benchmarks like Chameleon, underlining the challenge of robust generalization.

In audio, large pretrained SSL models (Wav2Vec2-Large, HuBERT-XLarge, XLS-R-2B) perform near optimally on ASVspoof 2019 (EER $<1\%$), with XLS-R-2B achieving state-of-the-art cross-domain generalization to non-benchmark datasets (In-the-Wild, EER $1.2\%$). However, supervised heads and specialized models frequently overfit, with out-of-distribution EERs increasing sharply ($>10\%$), marking a gap between benchmark and real-world detection needs.

Limitations and Critical Analysis

DetectZoo does not provide end-to-end training pipelines for new supervised detectors, relying on externally trained and released checkpoints for reproducibility. Coverage is empirically much deeper for text than audio, and the current absence of video or more esoteric modalities limits evaluation breadth. For some detectors, especially those requiring proprietary or closed-source language/vision backbones, on-device reproduction may diverge modestly due to frozen reference models. Public benchmarks remain static and may lag behind ongoing advances in generative model architectures and evasion strategies. The toolkit is designed for research benchmarking, not for critical real-time or adversarial-fault-tolerant deployment.

An explicit risk of toolkit centrality is that evasion research could utilize DetectZoo as a surrogate loss, accelerating the development of attack-adaptive generators. Nevertheless, the authors argue—correctly—that security through obscurity is untenable, and transparent, community-driven benchmarking is the only viable research paradigm for robust detector development.

Implications and Future Directions

Practically, DetectZoo accelerates method reusability, head-to-head comparison, and rapid prototyping for detection research, enabling the identification of cross-modal weaknesses and systematic gaps. It also sets a template for the modular, extensible design of evaluation infrastructures in other forensic or veracity tasks.

From a theoretical standpoint, the results expose that most current detection paradigms, especially those trained only on synthetic benchmarks, do not generalize robustly to task or domain shifts (e.g., paraphrased text, high-fidelity or highly compressed images, or in-the-wild speech). The strong performance of hybrid and pretrained foundation models, especially in audio and image, suggests that future research should emphasize representation learning that captures both semantic consistency and low-level statistical oddities. DetectZoo will be instrumental for experimentally validating such transfer across evolving model families.

Imminent roadmap directions include: expansion to video and other time-varying modalities, incorporation of training pipelines for end-to-end supervised method development, dynamic leaderboard integration, fine-grained visual analytics of detection scoring, and continued community-driven expansion of detectors and benchmarks. The architecture's emphasis on modular, open registration will support rapid extension as new methods and adversarial challenges emerge.

Conclusion

DetectZoo provides a rigorously validated, extensible infrastructure that standardizes multi-modal AI-generated content detection benchmarking. By bridging prior fragmentation, it enables reproducible, methodologically consistent evaluation across text, image, and audio, and exposes critical weaknesses in the generalization of prevailing detection methods. As generative models and evasion strategies evolve, community open-source infrastructure like DetectZoo will be essential for maintaining scientific rigor and transparency in detection research.