LLM Reasoning Failures

• Reasoning failures in LLMs are systematic errors in multi-step logic and constraint integration that undermine performance on complex tasks.
• They manifest as compositional breakdowns, inconsistent inference traces, and overreliance on superficial cues, including tokenization-induced artifacts.
• Research indicates that targeted interventions, such as memory injection, enhanced chain-of-thought prompting, and adversarial training, can mitigate these issues.

LLMs exhibit advanced surface-level reasoning but remain systematically vulnerable to a diverse set of reasoning failures, even as their raw capabilities continue to improve. Reasoning failures encompass breakdowns in multi-step logic, misintegration of constraints, inconsistency across similar prompts, susceptibility to superficial cues, and the inability to generalize reliably under even minor task perturbations. Recent research provides both behavioral taxonomies and mechanistic analyses of these deficiencies, illuminating root causes and outlining a taxonomy along both axes of reasoning type (embodied, informal, formal) and failure class (fundamental, application-specific, robustness) (Song et al., 5 Feb 2026).

1. Taxonomy of Reasoning Failures in LLMs

A two-dimensional taxonomy now structures the landscape of LLM reasoning failures (Song et al., 5 Feb 2026):

• By reasoning modality:
  • Embodied reasoning: Inference about the physical world, spatial configuration, and causal action.
  • Informal (intuitive) reasoning: Pattern-based, heuristic, and social inference lacking explicit formalism.
  • Formal (logical) reasoning: Symbolic, rule-based deduction, arithmetic, and algorithmic logic.
• By failure type:
  • Fundamental failures: Architectural or pretraining-level deficiencies inherent to LLM design.
  • Application-specific limitations: Failures in domain-specialized tasks (math, law, social interaction, etc.).
  • Robustness issues: Instabilities with minor or semantics-preserving changes in inputs or context.

A non-exhaustive table of representative failures is given below:

Reasoning Type	Fundamental Failures	Application-Specific Failures	Robustness Failures
Formal	Reversal curse, compositional breakdown, arithmetic collapse	Math word problem hallucination, code trace errors, legal reasoning inconsistency	Option reordering, distractor insertion, adversarial prompt perturbation
Informal	Cognitive biases (anchoring, confirmation)	Theory of Mind, moral/norm inconsistency, groupthink in multi-agent setups	Surface-form/context instability, MCQ phrasing effects
Embodied	Physical intuition limits, lack of grounded planning	Causal scene misprediction, tool use failures	Visual distractor effects, instruction rephrasing

2. Fundamental Mechanisms and Characteristic Failure Modes

Compositionality and Multi-hop Failures

LLMs are proficient on shallow one-hop tasks but degrade sharply on multi-hop or compositional problems—an effect traceable to insufficient persistence and propagation of intermediate entities in hidden states (Sakarvadia et al., 2023, Li et al., 2024, Song et al., 5 Feb 2026). Diagnoses reveal three main causes:

• Missing memory: Failure to represent intermediate facts required for subsequent hops.
• Spurious retrieval: Attention heads focus on high-frequency distractors rather than query-relevant concepts.
• Activation drift: Divergence of hidden-state trajectories between successful and failed multi-hop cases.

Memory injection at critical layers can partially repair such failures, demonstrating that correct intermediate signals are both necessary and reparable within the model's existing architecture (Sakarvadia et al., 2023). CREME, a localized intervention on multi-head self-attention matrices, provides further evidence that compositional errors are localized and amenable to targeted correction (Li et al., 2024).

Self-Consistency and Inference Trace Coherence

Even when LLMs produce correct answers, they often do so with inconsistent or incoherent reasoning chains. Two central forms are recognized (Chen et al., 2023):

• Hypothetical consistency: The model’s answer should be invariant when queried about its own hypothetical reasoning in slightly altered contexts, which fails at rates near random chance even for GPT-4.
• Compositional consistency: When a model's sub-step output is substituted into a subsequent prompt, the final outcome should remain the same. Instead, stepwise decoupling of outputs and trace results occurs ubiquitously.

Legal reasoning evaluations reveal low stepwise soundness—premises in the chain of reasoning frequently contain misinterpretations, irrelevant premises, or factual hallucinations leading to correct conclusions for incorrect reasons (Mishra et al., 8 Feb 2025).

Over-Reliance on Surface Cues and Adversarial Fragility

Evaluation under prompt perturbation, such as narrative reframing, misleading constraint injection, or reordering of examples, reveals that LLMs’ accuracy can drop precipitously (up to –54%) or even increase under certain conditions, reflecting heavy overfitting to prompt surface structure (Roh et al., 8 Jun 2025). These failures are not mitigated by model size or parameter count, and the directionality of performance changes is often unpredictable.

Tokenizer-Induced Artifacts

Recent analysis demonstrates that LLM reasoning can fail due to representational pathologies from non-injective tokenization (Ayoobi et al., 21 Jan 2026). Non-unique mappings lead to phantom-edit artifacts: models perform token-level manipulations believed to be semantic edits, yet the surface string is unaffected. Such artifacts represent a previously underappreciated axis of reasoning brittleness, especially in chain-of-thought and editing tasks.

3. Domain-Specific Failures: Code, Math, and Real-World Reasoning

Code Tracing and Execution

In code simulation and execution reasoning, dominant failure modes are found in basic arithmetic, control-flow splits, index miscalculations, and misinterpretation of native APIs (Abdollahi et al., 28 Nov 2025). Correctness on program outputs can be high (85–98%), but trace-level analysis reveals nine dominant categories of inference error, with computation and indexing errors prevailing. Tool-augmented reasoning (e.g., calculator plugins) can recover a significant fraction (58%) of computational failures, but cannot address high-level logic breakdowns.

Mathematical Word Problems and Symbolic Failures

Evaluations on mathematical word problems demonstrate that even state-of-the-art models combine high answer accuracy with flawed logical justification, a persistent division between solution-step soundness and final answer correctness (Boye et al., 17 Feb 2025). Typical errors include unwarranted assumptions, misapplication of patterns, failures of spatial/physical intuition, and planning deficits.

Symbolic computation exposes an architectural “split-brain syndrome”: the model can articulate correct procedural algorithms (comprehension) but fails to carry them out reliably (competence) due to functional and geometric decoupling between instruction and execution pathways (Zhang, 14 Jul 2025). Controlled tasks reveal no coherent mapping between the two pathways in latent representation space.

Constraint and Feature Hallucination

On core constraint satisfaction tasks such as graph coloring, LLMs systematically hallucinate non-existent problem features (e.g., spurious edges), causing cascading logical failures even when all input data is available (Heyman et al., 17 May 2025). Hallucination rates and error attributable to such phantom constraints scale linearly with problem complexity and are accentuated by chain-of-thought prompting, reflecting profound limitations in factual memory isolation within the model.

4. Multi-Agent and Collective Reasoning Failures

In multi-agent setups, LLM collectives demonstrate classic “shared information bias” and premature consensus, failing to integrate distributed unshared facts even in theoretically solvable Hidden Profile tasks (Li et al., 15 May 2025). Accuracy gains from group discussion are positive but leave large gaps compared to full-information baselines. Prompting strategies (cooperative, debate, explicit asymmetry) yield only modest improvements and do not consistently close the group-vs-individual reasoning gap.

Systematic measurement frameworks quantify the information integration score, lock-in consensus coherence, and comparison to human group dynamics, establishing that collective reasoning failures are not mitigated by scale or coordination protocol alone.

5. Robustness and Generalization Deficiencies

Despite high performance on in-distribution or base-case tasks, LLMs demonstrate catastrophic brittleness to specific stressors:

• Rule/path deletion: Perfect accuracy under redundant rule removal, but 25% (chance) performance if an essential logical path is omitted (Bao et al., 6 Dec 2025).
• Contradictory evidence injection: 0% accuracy when confronted with explicit semantic contradiction; models deterministically complete the chain, ignoring logical impossibility.
• Semantic-preserving rewrites: Invariance is maintained across complex logical equivalence transformations (e.g., contrapositive, double negation), suggesting robust paraphrase generalization, but this masks fragility to omissions and conflicts.

Adversarial perturbations, such as numeric shifts or shuffling distractors, produce massive swings in code and math performance (Roh et al., 8 Jun 2025). In MCQ and knowledge-intensive tasks, option order, distractors, and content cues can shift model choices by 10–20 points or more (Song et al., 5 Feb 2026). Specialized robustness metrics now accompany accuracy in state-of-the-art benchmarks.

6. Mechanistic and Architectural Roots

Root causes, as consolidated in the latest comprehensive surveys (Song et al., 5 Feb 2026), include:

• Next-token objective limitations: Training biases toward local pattern completion over global logical planning.
• Self-attention dispersion: Limitations in working memory and sequential reasoning.
• Tokenization artifacts: Non-injective mappings destabilize token-level manipulations.
• Model-internal separation: Instructional and execution subspaces remain geometrically and functionally dissociated (Zhang, 14 Jul 2025).
• Data and alignment biases: Pretraining and RLHF amplify innate human biases and heuristics.
• Lack of mechanistic introspection: Failure to bind reasoning traces to computation impedes verifiable causality and correction.

Improvements through black-box engineering are possible; for example, human-in-the-loop cascades with selective deferral can drive real error rates below 1% in some risk-sensitive settings without model modification (Zellinger et al., 18 Jul 2025).

7. Mitigation Strategies and Evaluation Paradigms

Remedies under active research span multiple layers:

• Prompt and process-level:
  • Chain-of-thought and stepwise verifiers (Li et al., 2024, Zellinger et al., 18 Jul 2025).
  • Explicit state tracking and prompt-anchoring for constraint problems (Heyman et al., 17 May 2025).
  • Multi-template ensembling and adversarial data augmentation for robustness (Roh et al., 8 Jun 2025).
• Training/data-level:
  • Graph-structured and multi-hop reasoning path pretraining (Song et al., 5 Feb 2026).
  • Explicit reversal and permutation-based data augmentation.
• Architectural/hybrid approaches:
  • External tool integration (calculators, symbolic solvers) (Abdollahi et al., 28 Nov 2025).
  • Neuro-symbolic modules and dual-context memory banks (Li et al., 2024).
  • CREME and memory-injection techniques to patch mid-layer errors (Sakarvadia et al., 2023, Li et al., 2024).
  • Inspector or challenger agents in multi-agent settings (Li et al., 15 May 2025).
• Evaluation and benchmarking:
  • Failure-injection stress tests (rule/path deletion, contradiction) (Bao et al., 6 Dec 2025).
  • Trace-level and self-consistency evaluation (Chen et al., 2023, Abdollahi et al., 28 Nov 2025).
  • Multi-agent scenario benchmarks to expose collective limitations (Li et al., 15 May 2025).

Continuous evaluation using these techniques, combined with unified taxonomies, will be critical for diagnosing and ultimately narrowing the gulf between surface-level fluency and genuine reasoning reliability in future LLM generations.