LLM Reasoning Failures: An Overview

• Large Language Model reasoning failures are systematic flaws where models rely on shallow pattern matching rather than robust logical deduction, affecting both formal and intuitive tasks.
• They manifest as compositional, disjunctive, and multilingual breakdowns alongside prompt sensitivity issues, undermining self-consistency and performance in structured domains.
• Mitigation strategies like chain-of-thought prompting, neuro-symbolic hybrids, and explicit constraint modules are crucial for enhancing reliability and safe deployment.

LLMs exhibit remarkable fluency on a wide range of reasoning benchmarks, but they remain fundamentally brittle and error-prone in a variety of real and structured domains. Reasoning failures are persistent, multifaceted, and arise from both architectural limitations and emergent properties of model training. Understanding these failures—ranging from shallow pattern matching and lack of systematic abstraction, to multilingual collapse, breakdowns in self-consistency, and fragile compliance with prompt perturbations—is critical for rigorous evaluation, safe deployment, and theoretically informed advancement of LLMs and derived Large Reasoning Models (LRMs).

1. Taxonomy and Categorization of Reasoning Failures

A unified taxonomic framework distinguishes reasoning failures along two orthogonal axes: (1) the type of reasoning (embodied, non-embodied–informal/intuitive vs. formal/logical) and (2) the nature of failure (fundamental, application-specific, robustness-based) (Song et al., 5 Feb 2026).

• Embodied Reasoning Failures: Breakdown in tasks involving spatial awareness, physical affordances, or grounded motor/action planning, often due to text-/vision-only pretraining without physical interaction signals.
• Non-embodied Reasoning Failures:
  • Informal (Intuitive): Failures in commonsense, social reasoning, or heuristic inference—e.g., inconsistent moral judgments, theory-of-mind errors, cognitive bias recapitulation.
  • Formal (Logical): Deficits in symbolic reasoning, chain-of-thought (CoT) composition, and verified multi-step inference—e.g., logical fallacies, arithmetic errors, loss of variable binding.

Failures further subdivide as:

• Fundamental: Intrinsic to LLM architectures (e.g., self-attention dispersion leading to working-memory collapse or reversal curse).
• Domain-/Application-Specific: Systematic but domain-tied; seen in math word problems, code generation, logic benchmarks, or social inference due to reliance on shallow heuristics.
• Robustness Failures: Sensitivity to semantically irrelevant prompt variations (e.g., option order shuffling, insertion of distractors, domain rewordings).

2. Compositional, Systematic, and Disjunctive Reasoning Deficits

LLMs and LRMs, even after extensive instruction tuning and RLHF, are systematically weak at compositional and disjunctive reasoning:

• Shallow Disjunctive Reasoning: On qualitative relational reasoning benchmarks (e.g., RCC-8, Allen's Interval Algebra), fine-tuned LLMs and advanced LRMs excel in single-path (chain) reasoning but fail at multi-path (disjunctive/intersection) tasks (Khalid et al., 30 Mar 2025). Rather than performing algebraic closure, models either (a) memorize shortcut patterns, (b) treat paths independently (no intersection), or (c) output plausible answers by frequency heuristics.
• Abstract and Causal Reasoning Deficits: Benchmarks designed around strong generalization, abstraction, and unseen composition (e.g., ARC, BIG-Bench-F, pointer-value tasks) reveal that LLMs lack causal-world models and the ability to induce “causal paths” beyond direct observed patterns (Gendron et al., 2023). Chain-of-thought (CoT) strategies offer negligible improvement on true abstraction.
• Self-Consistency Breakdown: LLMs violate both hypothetical consistency (predicting their own answer under semantically equivalent prompt transformations) and compositional consistency (giving the same answer when intermediate steps are replaced with their previously generated sub-answers), even in arithmetic and semantic parsing tasks. Consistency rates for even GPT-4 are <50%–65% depending on the setting (Chen et al., 2023).

3. Multilingual Reasoning Pathologies

Multilingual LLMs and LRMs display systematic breakdowns in reasoning trace fidelity and accuracy, primarily driven by linguistic priors and internal representation limitations.

• Cross-Lingual Collapse: During reinforcement-based fine-tuning, models trained to perform CoT in low-resource or non-English languages revert to their dominant pretraining language (typically English) in internal reasoning traces, a phenomenon termed “Cross-lingual Collapse.” Collapse can be rapid (UK: 98% → 0.3% word ratio in 250 updates) and essentially irreversible; reward shaping for language consistency preserves target language usage but decreases accuracy by up to 10 percentage points (Park et al., 6 Jun 2025). Task difficulty exacerbates collapse and strong pretraining in a non-English language only delays, not prevents, the drift.
• Reasoning-Answer Misalignment Across Languages: Large-scale analyses on GlobalMMLU across six languages show that high accuracy masks reasoning failures: in non-Latin scripts, the trace inconsistency rate (TIR) is more than twice that in Latin scripts (7.2% vs 3.3%), with misalignment rooted in unsupported claims, ambiguous facts, and illogical leaps (Ovalle et al., 27 Dec 2025).
• Understanding Failures as the Root of the Multilingual Reasoning Gap: The majority (70–85%) of the accuracy gap for low-resource languages is attributed to a model’s inability to internally map non-English prompts into its dominant “reasoning space.” Supervised detectors can predict these failures, and “Selective Translation” (translating only flagged cases to English) recovers nearly all of the accuracy improvement otherwise gained by brute-force translation, at ~20% of the cost (Kang et al., 31 Oct 2025).

4. Source and Structure of Reasoning Failures

• Tokenization-Layer Artifacts: Many apparent deficiencies in reasoning arise from the subword tokenizer's non-injective mapping; models may produce “phantom edits” (token ID changes that have no string-level effect) and treat equivalent strings as distinct, making simple replacement or reasoning tasks unreliable until the tokenizer layer is regularized or redesigned (Ayoobi et al., 21 Jan 2026).
• Long-Context Positional Bias and Retrieval-Utilization Disconnect: LLMs encode position and even “know” where the relevant information is within long contexts but frequently fail to surface this information in outputs—especially for items “lost in the middle.” The disconnect between what becomes accessible in hidden states vs. what reaches generation is now quantifiable and can reach accuracy gaps of 20–50 percentage points (2406.14673).
• Geometric Deviation from Reasoning Manifold: The REMA framework demonstrates that correct reasoning traces inhabit a low-dimensional manifold in model representation space. Failures correspond to detectable excursions from this manifold, often emerging at specific layers. Deviance diagnostics (based on k-nearest neighbor distances to the “correct” submanifold) provide an interpretable, architecture-agnostic tool for localizing failure points within a model’s computation (Li et al., 26 Sep 2025).
• Hallucination of Problem Features: RLLMs, especially in constraint-satisfaction tasks like graph coloring, frequently hallucinate critical but non-existent features of the prompt (e.g., inserting edges), directly causing spurious “Impossible” outputs. Over 67%–94% of false “Impossible” errors in state-of-the-art RLLMs are attributable to hallucinated constraints, and the phenomenon persists regardless of prompt framing or problem scale (Heyman et al., 17 May 2025).

5. Limitations in Reflection, Consistency, and Explanation Quality

• Pseudo-Reflection and Goal-Driven Monitoring: When tasked with open-ended generation and self-reflection, LLMs exhibit only minor increases after “reflection” (gain ≈ 0.21 valid items), with repeated constraint violation rates far above chance (85%). Reflection acts as a naive resample rather than active, constraint-aware correction, and reasoning-branded models offer no functional advantage (Weatherhead et al., 21 Oct 2025).
• Consistency-Reasoning Correlation: Consistency is tightly correlated with reasoning quality. Even the best models do not exceed 90% on both dimensions when evaluated on general-knowledge question/explanation pairs; inconsistent answers typically co-occur with spurious or hallucinated explanations, whereas stable models yield better explanations as measured by BLEU, F₁, and BERT scores (Saxena et al., 2024).

6. Robustness Failures under Prompt and Domain Perturbation

• Chain-of-Code Collapse and Prompt Sensitivity: In code-generation, semantically preserving but adversarial prompt transformations—storytelling, gamification, domain shift, example reordering, or distracting constraint injection—can cause accuracy to vary dramatically (–54% to +35% swings for the same logical task), revealing fragile reliance on surface cues (Roh et al., 8 Jun 2025). Low variance (σ) across benign perturbations is proposed as a new metric for reasoning robustness.
• Implicit Reasoning and Alignment-Compliance Gap in Multimodal LLMs: State-of-the-art multimodal LLMs, when faced with real-world tasks that require inferring implicit contradictions, object absence, ambiguous references, or goal infeasibility, often possess the latent reasoning ability but suppress it for user compliance. Explicitly requiring clarifying questions or cautious persona prompts unlocks these hidden skills, increasing performance from ~30% to >90% (Yan et al., 30 May 2025).

7. Empirical Studies in Mathematical and Multi-Hop Reasoning

• Multi-Hop and Structured Mathematical Reasoning: LLMs underperform on complex multi-hop tasks and generate correct answers through flawed logic (e.g., unwarranted assumptions, arithmetic slips, blind pattern fitting). Benchmarks constructed with novel, non-leaked combinatorial and spatial reasoning problems show that, on average, even advanced models only achieve ~75–80% accuracy, with 5% producing the numerically correct answer for unjustifiable reasons (Boye et al., 17 Feb 2025).
• Memory Injections and Failure Correction: For multi-hop failures specifically linked to mid-layer retrieval breakdowns, targeted “memory injection” at attention heads can increase correct-token probability by up to 424%, directly implicating the role of retrieval layers and residual stream manipulation in mitigating reasoning failures during inference (Sakarvadia et al., 2023).

8. Root Causes and Directions for Remediation

• Root Causes:
  • Optimization for next-token likelihood biases models toward locally coherent, statistically plausible continuation, not constraint satisfaction or stepwise deductive logic.
  • Transformer architecture biases (e.g., self-attention, positional encoding) induce surface-level pattern-matching over global compositional structure.
  • Pretraining corpora encode both statistical artifacts and human cognitive biases.
  • Absence of robust symbolic, procedural, or grounding modules limits systematic reasoning, especially across domains and modalities.
• Mitigation Strategies:
  • Chain-of-Thought prompting, reward shaping, retrieval-augmentation, and tool use provide marginal but brittle improvements.
  • Neuro-symbolic hybrids, tokenization-agnostic models, dynamic curriculum and robustness-oriented training, early detection diagnostics, and explicit constraint modules are active research areas requiring systematic integration to achieve robust, verifiable reasoning capabilities (Song et al., 5 Feb 2026, Park et al., 6 Jun 2025, Roh et al., 8 Jun 2025, Ayoobi et al., 21 Jan 2026).
• Future Directions:
  • Modular LLM architectures with explicit symbolic, causal, or embodied reasoning capacities.
  • Unified evaluation protocols that assess both answer quality and underlying reasoning, across perturbations, languages, and modalities.
  • Continual, community-maintained benchmarks and error catalogs to track progress and failure typologies (e.g., “Awesome-LLM-Reasoning-Failures” repository).

LLMs' reasoning failures are as diverse as the cognitive domains they seek to emulate. Systematic categorization, mechanistic interrogation, and rigorous multilingual and multimodal evaluation are essential for diagnosing and mitigating the underlying sources of brittle, surface-level, and unreliable reasoning behavior in these models. Progress will depend on going beyond incremental data and scale, toward architectures and training protocols that enforce both local and global logical rigor.