Autointerpretability Scores Explained

• Autointerpretability scores are scalar or vector-valued metrics that quantify a model's interpretability using weighted expert evaluations and complexity proxies.
• They standardize cross-model comparisons by integrating composite, tri-criteria, latent, and behavioral scoring frameworks across domains like vision, NLP, and program synthesis.
• These metrics reveal trade-offs between interpretability and predictive accuracy, offering actionable insights while addressing limitations and task-specific challenges.

Autointerpretability scores are scalar or vector-valued metrics designed to quantitatively assess the interpretability of machine learning models, their internal representations, or their outputs. These scores facilitate cross-model comparisons, standardize evaluation protocols, and enable practitioners to analyze trade-offs between interpretability and predictive accuracy. The precise definition, computation, and empirical behavior of autointerpretability scores vary according to model class, target domain (vision, NLP, program synthesis), and desired interpretability criteria. The following sections summarize the foundational methodologies, mathematical frameworks, empirical findings, and practical guidance from recent research.

1. Composite and Per-Model Interpretability Scoring

Composite Interpretability (CI) formalizes the interpretability of entire model pipelines by aggregating atomic per-model Interpretability Scores (IS), each computed as a weighted fusion of expert rankings and complexity proxies (Atrey et al., 10 Mar 2025). For a model $m$, IS is:

$$\mathrm{IS}(m) = \sum_{c \in C} w_c \cdot \frac{R_{m,c}}{R_{\max,c}} + w_{\text{parm}} \cdot \frac{P_m}{P_{\max}}$$

where:

• $C = \{$simplicity, transparency, explainability$\}$ (expert-rated criteria),
• $R_{m,c} \in [1,5]$ is the average expert score for criterion $c$ on $m$,
• $P_m$ is parameter count,
• $w_c$, $w_{\text{parm}}$ are nonnegative weights summing to one.

For composite pipelines of $n$ modules, CI is additive:

$\mathrm{CI} = \sum_{i=1}^n \mathrm{IS}(m_i)$

Lower CI indicates higher overall interpretability. Selecting weights to match domain priorities enables tailoring the metric, for example increasing $w_{\text{explainability}}$ in high-stakes settings. Empirical analysis confirms that CI inversely, but non-monotonically, correlates with predictive accuracy—pipelines with lower interpretability may perform better, yet interpretable models can sometimes outperform black-box alternatives.

2. Tri-Criteria AutoInterpretability Scores

A related methodology evaluates interpretability via a triplet of criteria: predictivity, stability, and simplicity (Margot et al., 2020)—yielding a comprehensive autointerpretability score suitable for rule-based and tree-based models:

$$\mathcal{I}(A_i) = \alpha_1 S_{\rm pred} + \alpha_2 S_{\rm stab}^q + \alpha_3 S_{\rm simp}$$

where:

• $S_{\rm pred}$ quantifies accuracy relative to a baseline,
• $S_{\rm stab}^q$ uses the Dice–Sorensen index to measure rule-set stability across bootstrapped samples,
• $S_{\rm simp}$ normalizes rule-set length among competitors,
• $(\alpha_1, \alpha_2, \alpha_3)$ are user-defined, nonnegative weights summing to one.

This scheme enables interpretable comparison across both rule-based and tree-based algorithms, with higher scores denoting better joint performance, robustness, and human simulability. Extensions can adapt the triplet to linear models or clustering by swapping the simplicity/stability proxies.

3. Explanation-Free Latent Scoring for Sparse Representations

For sparse autoencoders (SAEs) and related interpretability-focused representation learning, direct protocols eschew intermediate textual explanations (Paulo et al., 11 Jul 2025). Two notable metrics:

• Intruder-Detection Score: Assesses whether human/LLM evaluators can identify a "non-activating" context among several activating contexts sampled by latent quantile decile. High detection accuracy indicates a latent encodes semantically coherent, easily distinguishable features.
• Example-Embedding Score: Leverages clustering in generic embedding space (e.g., MiniLM sentence vectors) to compute AUROC for distinguishing activating vs. non-activating contexts. Higher AUROC reflects latent monosemanticity.

Empirical results show intruder detection aligns well with human ratings (Spearman $\rho=0.85$), whereas embedding scores provide a fast but less sensitive proxy.

4. LLM-centric Behavioral Interpretation Metrics

Interpretability of programmatic policies and code is quantified via LLM-based round-trip metrics (Bashir et al., 2023). The LINT score computes the average behavioral similarity between an original program $\pi$ and its reconstruction $\pi'$ after:

1. LLM-1 translates $\pi$ (constrained by DSL description and prompt) to natural language explanation $E$.
2. LLM-2 (reconstructor) translates $E$ back to code $\pi'$.
3. Behavioral similarity $B(\pi,\pi') \in [0,1]$ is measured over test cases (perfect match yields $B=1$).

Applications to code obfuscation and policy synthesis confirm LINT is sensitive to injected complexity and reliably ranks original, mildly obfuscated, and heavily obfuscated code according to their interpretability.

5. Representation Inherent Interpretability via Information Retention

For deep vision backbones, Inherent Interpretability Score (IIS) quantifies the fraction of classifiable semantics captured by interpretable concept subspaces (Shen et al., 28 Oct 2025). IIS integrates the accuracy retention rate (ARR) over sparsity schedules:

$$\mathrm{IIS}(f, C; \mathcal{D}) = \int_0^1 \frac{\mathrm{Acc}(\text{interpreted})}{\mathrm{Acc}(\text{full rep})}\; ds$$

Implementing this involves constructing concept libraries (prototype, cluster, end-to-end, or textual), applying varying sparsity, and retraining concept-based classifiers. Notably, IIS exhibits a strong, positive empirical correlation with classifiability. Fine-tuning with IIS-based objectives further boosts both interpretability and classification performance, indicating mutual promotion.

6. Interpretation Quality and Grounded Explanation Scores

Interpretation Quality Score (IQS) evaluates explanation methods against three human-centric axes (Xie et al., 2022):

• Plausibility: Jaccard overlap between method-highlighted and human-highlighted features,
• Simplicity: Penalizes explanations exceeding human cognitive chunk capacity,
• Reproducibility: Measures fidelity between human predictions informed by explanations and true model outputs.

Aggregated as: $$\mathrm{IQS} = \alpha_1\,\text{plausibility} + \alpha_2\,\text{simplicity} + \alpha_3\,\text{reproducibility}$$, IQS supports systematic benchmark comparisons across explanation methods, with established ranking stability across weightings and tasks.

7. Interpretability–Utility Gap and Correlational Analysis

Recent work exposes the weak and non-monotonic link between interpretability scores and downstream utility for feature steering in LLMs (Wang et al., 4 Oct 2025). Using SAEBench’s Automated Interpretability Score $\mu(\theta)$ and AxBench utility metrics, Kendall’s $\tau_b$ rank-analysis yields only a modest association ($\tau_b\approx 0.30$). A novel criterion, Delta Token Confidence ($\Delta TC$), quantifies a feature’s effect on next-token distribution. Selecting features with high $\Delta TC$ boosts steering metrics by over 50%. When restricting analysis to $\Delta TC$-selected features, the interpretability–utility rank correlation vanishes or becomes negative, signifying that easily explainable features are not necessarily the most effective for LLM steering. This suggests that future metrics should be designed with explicit downstream tasks in mind.

8. Limitations, Extensions, and Practical Recommendations

Autointerpretability scores face several open challenges:

• Expert judgment-based scoring can introduce subjectivity and panel bias; algorithmic proxies (e.g. path length, nonzero parameters) may partially ameliorate this.
• Linear or additive aggregation of criteria may miss interaction effects between model components.
• Embedding-based and explanation-free protocols can under- or over-estimate true human interpretability depending on probe design and domain.
• Task dependence remains strong—criteria relevant for vision, NLP, or program synthesis may differ, requiring flexible or extensible scoring formulae.
• All current metrics are diagnostic, not suited as direct training objectives, due to the risk of gaming or superficial optimization.
• Best practice is to use multiple complementary scores, stratify results by domain/task, and cross-validate with expert and human assessment where feasible.

The proliferation of rigorous, formalized autointerpretability scores as described above represents a major advance in the scientific assessment of model human-friendliness, enabling transparent, reproducible, and application-aware selection of ML models for deployment across research and industry.