Biases in the Blind Spot: Detecting What LLMs Fail to Mention

Abstract: LLMs often provide chain-of-thought (CoT) reasoning traces that appear plausible, but may hide internal biases. We call these unverbalized biases. Monitoring models via their stated reasoning is therefore unreliable, and existing bias evaluations typically require predefined categories and hand-crafted datasets. In this work, we introduce a fully automated, black-box pipeline for detecting task-specific unverbalized biases. Given a task dataset, the pipeline uses LLM autoraters to generate candidate bias concepts. It then tests each concept on progressively larger input samples by generating positive and negative variations, and applies statistical techniques for multiple testing and early stopping. A concept is flagged as an unverbalized bias if it yields statistically significant performance differences while not being cited as justification in the model's CoTs. We evaluate our pipeline across six LLMs on three decision tasks (hiring, loan approval, and university admissions). Our technique automatically discovers previously unknown biases in these models (e.g., Spanish fluency, English proficiency, writing formality). In the same run, the pipeline also validates biases that were manually identified by prior work (gender, race, religion, ethnicity). More broadly, our proposed approach provides a practical, scalable path to automatic task-specific bias discovery.

• The paper presents a fully automated black-box pipeline that detects hidden, unverbalized biases by synthesizing counterfactual input variations with robust statistical tests.
• It demonstrates that LLMs can exhibit significant demographic decision shifts not reflected in their chain-of-thought explanations, affecting hiring, lending, and admissions.
• Empirical results reveal both canonical and novel biases—with effects up to 3.7 percentage points—underscoring the need for more reliable bias auditing mechanisms.

Unverbalized Biases in LLMs: Automated Detection Beyond Model Explanations

Introduction

LLMs demonstrate sophisticated reasoning capabilities and are increasingly deployed in decision-critical applications such as hiring, lending, and admissions. These models frequently provide chain-of-thought (CoT) reasoning traces ostensibly elucidating their internal decision processes. However, there is growing evidence that such explanations may be unfaithful—that is, models can rely on hidden, unstated biases in their outputs while presenting plausible yet misleading rationales. This paper, "Biases in the Blind Spot: Detecting What LLMs Fail to Mention" (2602.10117), addresses the detection of these unverbalized (unfaithful) biases: systematic decision shifts caused by input attributes that are never cited as justification in the generated reasoning.

The authors propose a fully automated, black-box pipeline for the detection of such unverbalized biases in LLMs, leveraging LLMs as both concept generators and evaluators, and employing counterfactual variation and robust statistical testing. This essay provides an in-depth analysis of the methodology, evaluation, and implications for both research and deployment of LLMs in sensitive real-world contexts.

Automated Pipeline for Detecting Unverbalized Biases

The core methodological contribution is a multi-stage pipeline that automates the end-to-end process of detecting task-specific hidden biases without requiring pre-defined categories or hand-crafted counterfactual data.

Pipeline overview:

• Concept generation: Input datasets are clustered via embeddings, and LLMs generate candidate bias concepts from cluster representatives. This process is black-box and does not require labeled data or manual hypothesis creation.
• Variation synthesis: For each concept, LLMs create positive and negative input variants—e.g., adding or removing a demographic-signaling sentence—while another LLM judge filters out confounded variations.
• Verbalization filtering: Before statistical testing, the pipeline discards concepts for which the model routinely cites the concept as a decision factor (verbalization threshold, typically 30%). Only concepts that are not overtly discussed by the model are retained as candidates for unverbalized bias.
• Statistical testing: Using McNemar's test on discordant pairs—input pairs where output flips between variations—the pipeline tests for statistically significant decision shifts. Family-wise error is tightly controlled via Bonferroni correction and multi-stage O'Brien-Fleming alpha spending for early stopping.
• Progressive sampling and early stopping: To reduce cost, the process incrementally increases sample size and abandons concepts that show insufficient power (futility analysis).

The pipeline is thus able to scale to large input sets and a wide breadth of concepts, while providing strong guarantees against false positives.

Empirical Results: Unverbalized Biases in Major LLMs

The authors evaluate their pipeline across six major LLMs (Gemma 3-12B, Gemma 3-27B, Gemini 2.5 Flash, GPT-4.1, QwQ-32B, Claude Sonnet 4) and three high-stakes decision tasks: hiring, loan approval, and university admissions. The experimental datasets include both realistic and synthetic data derived from public resources.

Key Findings

• The pipeline automatically rediscovers major demographic biases—such as gender and race/ethnicity—in hiring decisions, previously identified in manual studies. However, critically, these biases are not cited by the models in their CoT explanations.
• Novel biases are uncovered, including biases towards Spanish fluency, English proficiency, and writing formality, which had not been covered in earlier manual analysis (see also Figure 1 for concept survivorship).

Figure 1: Concept category survivorship across seeds, illustrating the consistency and variability in detected bias concepts across pipeline runs.

• The magnitude of these unverbalized biases is substantial: for example, in loan approval tasks, religious affiliation can shift the approval rate by 3.7 percentage points in favor of minority-religion applicants, with an effect size $0.037$ and extremely strong statistical significance ($p = 9.15 \times 10^{-7}$, $n = 2,500$ inputs); the bias is cited in just $12.4\%$ of flipped responses (Figure 2).

Figure 2: Example of an unverbalized bias—changing religious affiliation alters the decision outcome, but religion is never mentioned in the model's reasoning.

• Cross-task and cross-model consistency: Gender and race/ethnicity biases recur across hiring, loans, and admissions, implying these represent entrenched model behaviors rather than dataset or prompt artifacts.
• Faithfulness is not improved by reasoning-model fine-tuning: Comparative analysis of standard supervised and reasoning-trained models (e.g., RLVR versus SFT) shows that unverbalized bias prevalence and verbalization rates are nearly identical.

Injection and Filtering Studies

Intentional injection of secret (concealed) and overt (explicit) biases confirms that the pipeline detects $85\%$ of hidden biases while filtering $100\%$ of overt ones (Figure 3 and Figure 4). False negatives are mainly due to over-sensitive verbalization detection.

Figure 3: Intentional bias injection study results, showing high accuracy in identifying both secret and overt biases.

Figure 4: Per-concept accuracy in detection and filtering; green indicates high accuracy, demonstrating the robustness across diverse injected concepts.

Reliability of Pipeline Components

• Verbalization detection is robust: LLM-based detectors show substantial agreement with human annotators ($\kappa = 0.67$–$0.79$), and errors are mostly false positives (over-filtering).
• Quality checking removes confounded variations: Automated LLM filtering using a $5$-point scale matches human annotation in 80% of cases, ensuring causal isolation of tested concepts.

Implications and Limitations

Practical and Theoretical Implications

• CoT explanations are unreliable for bias auditing: Systematic decision shifts can occur without explicit reasoning. CoT-based oversight and explanation tracing do not suffice to uncover subtle or concealed biases.
• Automated, scalable bias detection: The presented black-box pipeline obviates the need for pre-curated test cases and expert-generated hypotheses, scaling bias discovery to previously unannotated domains and models.
• Impact for safe model deployment: For high-stakes decision support, unverbalized biases present significant risks—e.g., a $3\%$ effect in loan approval represents $30$ out of $1,000$ applicants affected by hidden, unaccounted decision factors.
• Faithfulness training has uncertain effects: Simply training LLMs to provide more detailed or rationale-oriented CoT explanations does not eliminate hidden biases; it may only shift which attributes are verbalized versus left latent.

Limitations

• Concept coverage is bounded by the generating LLM's expressiveness: Although automated, the hypothesis generator can only propose concepts it can infer; rare or subtle biases may thus escape detection without human-in-the-loop or evolutionary augmentation.
• Filtering trades off recall for precision: Conservative thresholds and multi-stage stopping favor minimizing false positives at the cost of missing subtle biases with smaller effect sizes.
• Not all unverbalized factors are inappropriate: The pipeline flags systematic effects regardless of normative judgments about fairness; post-hoc audit is required to differentiate harmful discrimination from unmentioned but legitimate heuristics.

Future Research Directions

• Integrating human oversight with automated hypothesis generation could increase bias coverage, especially for domain-specific applications.
• Evolutionary search or active learning for concept proposal could further improve detection for under-represented or adversarially suppressed biases.
• Improvements in semantic-level verbalization detection, e.g., via entailment models or more nuanced explanation parsing, may reduce over-filtering and increase pipeline recall.
• Incorporating continuous effect metrics (as advocated in recent literature) could provide deeper insight into the gradient of bias for ordinal or continuous input features.

Conclusion

This work presents a rigorous, fully automated, and scalable approach for detecting unverbalized biases in LLMs, fundamentally challenging the view that chain-of-thought explanations reflect true model decision factors. The pipeline successfully identifies both canonical and novel hidden biases across a broad spectrum of contemporary models and tasks, and establishes strong quantitative evidence that explanation-based monitoring is insufficient for robust LLM auditing. As LLMs become ever more integrated into consequential decision-making, such automated, black-box methods constitute an essential addition to the bias and safety evaluation toolbox.