Simple LLM Baselines are Competitive for Model Diffing

Abstract: Standard LLM evaluations only test capabilities or dispositions that evaluators designed them for, missing unexpected differences such as behavioral shifts between model revisions or emergent misaligned tendencies. Model diffing addresses this limitation by automatically surfacing systematic behavioral differences. Recent approaches include LLM-based methods that generate natural language descriptions and sparse autoencoder (SAE)-based methods that identify interpretable features. However, no systematic comparison of these approaches exists nor are there established evaluation criteria. We address this gap by proposing evaluation metrics for key desiderata (generalization, interestingness, and abstraction level) and use these to compare existing methods. Our results show that an improved LLM-based baseline performs comparably to the SAE-based method while typically surfacing more abstract behavioral differences.

• The paper demonstrates that a simple LLM-based diffing pipeline achieves parity with advanced SAE methods while producing higher-level, actionable hypotheses.
• It introduces a rigorous evaluation framework that quantifies accuracy, frequency, interestingness, abstraction, and acceptance rates in detecting model behavior changes.
• The study underscores the practical implications of API-only diffing for auditing model revisions and ensuring safer, transparent model deployments.

Simple LLM Baselines Are Competitive for Model Diffing

Introduction and Motivation

Conventional LLM evaluation protocols focus on targeted assessments of model capabilities or dispositions, inherently lacking the capacity to reveal unexpected or emergent behavioral shifts—especially between model revisions or across fine-tuning regimes. Model diffing addresses this critical gap by systematically surfacing behavioral differences between models using only black-box (API-only) access. This is highly relevant for the evaluation of closed-source frontier LLMs, as weight access is often unavailable. While both LLM-driven and SAE-driven (sparse autoencoder) model diffing approaches have gained traction, prior research lacks a systematic head-to-head comparison or a robust, multi-faceted evaluation framework quantifying key desiderata such as generalization, interestingness, and abstraction level.

The paper proposes a principled evaluation methodology and delivers a comparative analysis of state-of-the-art API-only model diffing pipelines. Empirically, it demonstrates that a simple LLM-based approach—when implemented with careful engineering—achieves parity with leading SAE-based methods on core metrics, while producing hypotheses at consistently higher abstraction levels.

Model Diffing Methodologies

The study juxtaposes two major classes of API-only model diffing methods:

• LLM-based Diffing: Follows a clustering pipeline—extract qualitative differences between paired prompt–response completions across two models, embed these differences, cluster to identify recurrent patterns, summarize each cluster into testable hypotheses, and assign hypothesis directionality based on prevalence.
• SAE-based Diffing: Leverages a sparse autoencoder over a ‘reader’ LLM; selects activation features with the largest cross-model frequency differences, relabels them using an LLM, and summarizes into human-interpretable hypotheses.

Both methods generate natural language hypotheses of the form "Model A does X more than Model B," but differ in their granularity (token-level versus behavioral abstraction) and capacity for capturing global patterns.

Experimental Evaluation Design

Evaluating diffing methods is non-trivial in the absence of ground-truth, so the paper implements a comparative LLM-judge pipeline to quantify performance. Five core metrics are disentangled:

• Accuracy: Proportion of hypothesis applications where the LLM judge identifies the correct model.
• Frequency: How often a hypothesized behavioral difference manifests in held-out data.
• Interestingness: LLM-based rating reflecting novelty and practical relevance.
• Abstraction Level: LLM-based rating indicating conceptual granularity (token-level to global behavioral strategies).
• Acceptance Rate: Fraction of self-supported hypotheses, indicating generation reliability.

Figure 1, Figure 2, Figure 3, and Figure 4, referenced and displayed below, visualize the distributions of these core metrics across experimental conditions.

Figure 1: Distribution of accuracy metrics across LLM- and SAE-generated hypotheses, revealing similar accuracy profiles with concentration at high values, except on subtler revision tasks.

Figure 2: Frequency distributions demonstrate high-variance, with both methods producing both rare and frequent behavioral hypotheses.

Figure 3: Interestingness distributions according to LLM-raters, showing moderate means with SAE methods sometimes surfacing more non-trivial (but often lower-level) features.

Figure 4: Abstraction level distributions, clearly illustrating that LLM-based diffing produces hypotheses at higher conceptual granularity compared to SAE-based pipelines.

Empirical Findings and Case Studies

Three task settings are investigated, covering both synthetic organism-style fine-tunes with known behavioral ground truths and practical model revision analysis:

Qwen – Emergent Misalignment

Both methods detect the core misalignment (risky/harmful advice) in a fine-tuned Qwen model organism. The LLM-based method surfaces direct characterizations, e.g., "provides advice that is unethical, dangerous, or harmful," while the SAE-based method orients around linguistic or rhetorical features. Additionally, the LLM-based approach uniquely extracts global behavioral signatures such as abnormally brief completions and factually unsupported assertions. Quantitatively, both methods yield comparable accuracy and frequency, with the LLM-based pipeline achieving higher abstraction and acceptance rates.

Gemma – User Gender Assumption

When the ground-truth behavioral shift (implicit female user assumption) is sparsely exhibited in outputs, neither method successfully detects it. Both pipelines instead extract general quality degradations (brevity, repetition, hallucination) arising after fine-tuning. This negative outcome is traced to prompt distribution limitations: only a small fraction of prompts can even elicit the target behavior, so its prevalence in outputs is insufficient for API-based model diffing to succeed (see Section Appendix for extended quantitative analysis and judge inconsistency investigations).

Gemini – Model Revisions

Applying the diffing methods to subsequent revisions of Gemini 2.5 Flash Lite, both pipelines surface undocumented behavioral changes: the preview version exhibits more structured organization (e.g., Markdown tables, section headers, mathematical notation). Here, the LLM-based approach summarizes behavioral formats at the response level, while SAE surfaces token-level syntactic artifacts. Manual validation of output statistics corroborates frequency estimates.

Theoretical and Practical Implications

The study’s methodology and results have several implications:

• Model diffing as a safety and audit tool: API-only approaches enable systematic cross-lab and pre-release comparisons. Higher abstraction-level hypotheses facilitate rapid audit and triage, while lower-level (token or formatting) features from SAE-based pipelines are valuable for forensic or compliance scenarios.
• LLM abstraction advantage: The higher abstraction hypotheses generated by LLM-based pipelines are more actionable and interpretable in practical settings. SAE features, in contrast, often remain bound to token- or phrase-level patterns unless further processed.
• Limits of API-only methods: Subtle or implicit behaviors, unelicited by prompt distributions, are unlikely to be surfaced. For high-confidence safety assignments, diffing must be complemented with targeted prompt design or internal model probes.
• Metric tradeoffs: There are fundamental tensions between accuracy, generality (frequency), and abstraction. High-frequency hypotheses can lack discriminatory power, and high abstraction may sacrifice mechanical verifiability. The proposed metrics enable practitioners to filter and interpret diffing outputs according to operational requirements.

Outlook and Future Directions

Several research directions are suggested by the findings:

• Hybrid pipelines: Integrating SAE feature detection with LLM-based summarization may unify low-level recall with high-level abstraction.
• Prompt selection for domain-specific behaviors: Systematic prompt curation tailored to specific behaviors (e.g., reasoning types, hidden assumptions) can enhance the sensitivity of API-only diffing.
• Comparisons of internal model reasoning: As chain-of-thought output becomes more prevalent, diffing methodologies could disclose divergences in multi-step reasoning or epistemic stance, going beyond completion-level evaluation.
• Metricification and community benchmarks: The evaluation methodology and open-source contributions lay foundations for establishing community benchmarks for model diffing tasks, directly supporting both safety evaluations and deployment studies.

Conclusion

This study demonstrates that simple, carefully engineered LLM-based model diffing pipelines are at least as capable as the most advanced API-only SAE-based techniques across several quantitative and qualitative metrics, exhibiting pronounced advantages in abstraction level and hypothesis acceptance. While both approaches present complementary strengths, the LLM pipeline’s ability to efficiently summarize global behaviors renders it a robust baseline for future model diffing research (2602.10371). Going forward, advances in LLM evaluation, prompt design, and hybridization with mechanistic interpretability tools are likely to further enhance the practical utility of model diffing in responsible model release and alignment.