Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs

Abstract: While reasoning in LLMs plays a natural role in math, code generation, and multi-hop factual questions, its effect on simple, single-hop factual questions remains unclear. Such questions do not require step-by-step logical decomposition, making the utility of reasoning highly counterintuitive. Nevertheless, we find that enabling reasoning substantially expands the capability boundary of the model's parametric knowledge recall, unlocking correct answers that are otherwise effectively unreachable. Why does reasoning aid parametric knowledge recall when there are no complex reasoning steps to be done? To answer this, we design a series of hypothesis-driven controlled experiments, and identify two key driving mechanisms: (1) a computational buffer effect, where the model uses the generated reasoning tokens to perform latent computation independent of their semantic content; and (2) factual priming, where generating topically related facts acts as a semantic bridge that facilitates correct answer retrieval. Importantly, this latter generative self-retrieval mechanism carries inherent risks: we demonstrate that hallucinating intermediate facts during reasoning increases the likelihood of hallucinations in the final answer. Finally, we show that our insights can be harnessed to directly improve model accuracy by prioritizing reasoning trajectories that contain hallucination-free factual statements.

• The paper shows that chain-of-thought reasoning significantly expands pass@k accuracy of LLMs by enabling latent computation and content priming.
• The methodology isolates the benefits of genuine reasoning compared to dummy tokens and quantifies these gains using a new reasoning effectiveness metric.
• Findings reveal that self-retrieved factual context boosts recall in LLMs while also heightening risks of hallucination and inaccurate responses.

Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs

Overview and Motivation

"Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs" (2603.09906) addresses the surprising efficacy of Chain-of-Thought (CoT) reasoning in LLMs, specifically within the context of simple, single-hop factual questions. This domain has traditionally been conceived as requiring only direct recall from model parameters, rendering explicit reasoning apparently superfluous. The paper provides a rigorous investigation into why reasoning traces—despite their lack of logical decomposition—significantly expand the factual recall boundary of state-of-the-art LLMs, rendering correct responses accessible that would otherwise remain unreachable under standard, zero-thought inference.

Expansion of Parametric Knowledge Boundary

The central empirical finding is that enabling reasoning via CoT not only improves standard pass@$1$ accuracy but, more notably, substantially expands the model’s parametric recall boundary at higher sampling rates (pass@$k$). Experiments with Gemini-2.5-Flash, Gemini-2.5-Pro, and Qwen3-32B on closed-book QA benchmarks demonstrate consistently higher pass@$k$ scores when reasoning is enabled, with some setups observing nearly doubled pass@$k$ compared to the non-reasoning regime.

Figure 1: Pass@$k$ curves across two closed-book QA benchmarks and three LLMs, comparing the same models with reasoning OFF vs ON.

To aggregate this effect, a reasoning effectiveness metric $\Omega$ is defined, weighting pass@$k$ improvements more heavily at greater $k$. Analysis reveals that less capable models (e.g., Qwen3-32B) exhibit stronger gains, suggesting that these models store more "hidden" factual knowledge that only becomes accessible via reasoning-driven inference. Counter to intuition, this expansion is largely decoupled from question complexity or multi-hop structure; both simple and “complex” (multi-hop/“requires reasoning”) subsets show similar marginal gains, indicating the effect is not driven by logical decomposition but instead relates to the recall process itself.

Figure 3: Reasoning effectiveness ($\Omega$) as a weighted improvement metric, highlighting greater gains in less capable models and on harder factual QA subsets.

Mechanisms: Beyond Superficial Explanations

The paper advances two mechanistic hypotheses for this phenomenon: (1) a computation buffer effect, and (2) content-dependent factual priming.

Computation Buffer Effect

The computation buffer effect posits that generation of additional “reasoning” tokens—regardless of their semantic content—permits latent computation, enabling models to refine or access representations that would otherwise not inform the final output. This is causally isolated by substituting genuine reasoning traces with repeated, semantically vacuous dummy tokens. Performance improves relative to standard (OFF) inference simply by increasing the “thinking” token budget, but this effect is saturating and never fully recovers the delta achieved with authentic reasoning. An ablation with dummy traces of increasing length reveals a non-monotonic pattern: initial improvements plateau and ultimately degrade as traces become very long, suggesting architectural and context-window constraints on effective compute utilization.

Figure 2: Computation buffer effect on Gemini-2.5-Flash; increasing dummy reasoning trace length yields substantial but bounded accuracy gains.

Figure 5: The effect of increasing compute—dummy_X traces of varying length—showing non-monotonic returns and eventual performance regression.

Factual Priming via Generative Self-Retrieval

Qualitative and controlled quantitative analyses expose a content-based mechanism: factual priming. Reasoning traces seldom feature explicit logical inference steps, but frequently recall or enumerate topically related facts (entities, properties, lists) that are semantically adjacent to the target answer. These recalled items compose a factual context—often not present in the original question—that primes successful answer generation. Careful extraction, filtration, and re-injection of these facts as context under a reasoning-OFF protocol recapitulates nearly all the pass@$k$ improvements of the full reasoning mode. This establishes that the surfaced facts are genuinely useful for recall, independent of their presence in a reasoning trajectory.

Figure 6: Factual priming effect on Gemini-2.5-Flash, showing that contextually injected, model-recalled facts (OFF Facts) almost fully replicate the gains from reasoning.

Direct case studies further validate this effect, with correct answers retrieved only when the model is prompted with the fact list produced during its own reasoning phase.

Figure 8: Case study for the effectiveness of factual priming; injected fact lists derived from the model’s reasoning enable correct recall.

Hallucination Dynamics: The Fragility of Self-Retrieval

While generative self-retrieval (factual priming) is a powerful driver of enhanced recall, it is inherently susceptible to error propagation. The study conducts a rigorous, automated audit of all intermediate facts produced during reasoning using a search-enabled LLM classifier. It demonstrates that hallucinated (incorrect) facts massively increase the probability of a hallucinated final answer—down from 41.4% correctness in “clean” traces to 26.4% when any intermediate fact is incorrect (SimpleQA-Verified). This effect persists after controlling for intrinsic question difficulty by within-question stratification.

Figure 7: Within-question analysis of correct final answer rates for clean (x-axis) vs. hallucinated (y-axis) reasoning traces, showing consistent degradation when hallucinations occur.

Operationalizing Insights: Towards More Reliable LLM Reasoning

The findings have direct practical implications: test-time inference strategies that prefer reasoning trajectories containing factual statements, and especially those in which all retrieved facts are verified as correct, yield significant gains in accuracy (up to 12.2% relative improvement). This suggests that downstream reliability in open-ended QA may be enhanced by factual audit and trace selection or, prospectively, by process-hallucination-aware reward shaping during training.

Implications and Prospects for Future Research

This work provides compelling evidence that the boundary of an LLM’s parametric knowledge is not fixed by its static parameterization, but is substantially influenced by the generative inference protocol. The strong effects of latent computation and primed context recall invite further study into optimization of computational graphs and prompt engineering for knowledge extraction.

The hallucination dynamic exposes fragility in generative self-retrieval mechanisms, indicating a potential avenue for model improvement via hallucination-minimizing process supervision or auxiliary factual consistency penalties. It remains an open question whether differentiated architectural modifications could further disentangle compute/recall and control hallucination propagation.

Conclusion

Enabling reasoning in LLMs substantially expands their effective factual recall boundary, even for simple, single-hop questions that, prima facie, require no explicit reasoning. This effect arises through both increased latent computation and, more critically, semantic priming by self-recalled, related facts. However, the benefits of generative self-retrieval are offset by increased risk of factual hallucination, which is shown to directly degrade final answer reliability. Operationalizing these findings via factual-trace prioritization or hallucination audits yields meaningful accuracy improvements, with significant implications for the development of more capable and reliable LLMs in closed-book knowledge tasks.