Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers

Published 30 Jun 2026 in cs.LG and cs.CL | (2606.31779v1)

Abstract: LLMs typically reason via explicit chain-of-thought (CoT), generating intermediate steps token-by-token. Latent CoT offers an alternative: it performs multi-step reasoning in the model's hidden states, replacing decoded tokens with continuous representations for greater efficiency. However, existing latent CoT methods underperform explicit CoT beyond 1B parameters, and the gap widens with scale. Looped, or recurrent-depth, Transformers, which reuse their weights to increase computation depth without adding parameters, are a natural fit for latent reasoning. We therefore ask whether looped Transformers can bridge this gap. We answer affirmatively with a simple recipe: a looped padded Transformer that processes K latent blocks in parallel for R iterations, with a cross-entropy loss on each latent position's gold CoT-step token, similar to explicit CoT supervision. We instantiate it as LOTUS (Looped Transformers with parallel supervision on latents). LOTUS is, to our knowledge, the first latent-CoT method to bridge the gap to explicit CoT at the 3B scale, while cutting thought-phase latency by 2.5x-6.9x from compact math expressions to natural language. Projecting LOTUS's post-loop latents through the base LM head recovers the gold reasoning steps and even surfaces alternative valid intermediate steps, evidence that its latent space is interpretable and CoT-aligned. Ablations confirm that both the looped backbone and the parallel supervision on gold CoT tokens are essential.

Abstract PDF Upgrade to Chat

Authors (3)

Summary

The paper introduces LOTUS, a looped transformer that marries explicit chain-of-thought supervision with latent reasoning to achieve near-parity in accuracy at scale.
The methodology employs a recurrent-depth design with parallel latent supervision, cutting inference latency by processing latent steps in parallel.
Experimental results on benchmarks like GSM8K reveal LOTUS's robustness and efficiency, demonstrating significant improvements over prior latent methods.

Bridging Latent and Explicit Reasoning: LOTUS and Looped Transformers

Introduction and Motivation

Explicit chain-of-thought (CoT) reasoning protocols for LLMs, wherein intermediate steps are produced as sequentially generated tokens, have demonstrated strong improvements on tasks necessitating multi-step reasoning. However, the decoding cost inherent in explicit CoT—each intermediate step requiring an additional autoregressive generation step—yields substantial inference overhead, particularly as solution lengths grow with backbone model size. Latent CoT methods attempt to mitigate this by performing intermediate computation directly in continuous hidden states. Despite early success on small architectures, prior latent CoT approaches lag significantly behind explicit CoT as model scale increases, exposing a widening performance gap beyond the billion-parameter regime.

This paper addresses the challenge of achieving both the accuracy ceiling of explicit CoT and the efficiency of latent reasoning. Leverage is found in recurrent-depth ("looped") Transformers, which can increase computational depth without additional parameters by reusing weights across multiple iterative passes. The central question is whether such looped architectures, combined with structured latent supervision, can close the accuracy gap while offering significant latency advantages.

LOTUS Architecture

The proposed architecture, LOTUS (Lo*oped **Transformers with parallel supervision on latents*), is a looped padded Transformer that processes $K$ fixed latent blocks in parallel for $R$ loop iterations, with each block intended to capture the latent representation of a gold CoT step.

A learnable latent prefix is appended to the input, and looped forward computation is applied: the input is passed through the backbone Transformer $R$ times, with each iteration refining the set of latent embeddings while attending to a fixed prefix key–value cache. Supervision is imposed via parallel cross-entropy loss: after $R$ iterations, each latent block is aligned with the corresponding gold CoT token through the model’s original LM head. This design crucially decouples sequential step count from the number of forward evaluations, as all latent blocks are processed in parallel.

Figure 1: LOTUS architecture. (a) Looped forward: the looped Transformer iterates $R$ times over the input with multiple learnable latent blocks, producing post-loop hidden states for each latent position, which are supervised via the LM head against CoT tokens.

The LOTUS framework supports an auxiliary variant ("LOTUS-aux"), which routes supervision through a separate autoregressive decoder conditioned on each latent block (used only at training). Ablation demonstrates that both the looped backbone and direct, parallel supervision are essential for robust scaling.

Experimental Results

Accuracy–Efficiency Tradeoff

LOTUS is the first latent-reasoning method to achieve near-parity with explicit CoT at scale, matching within 1.5 points on GSM8K in-domain accuracy at the 3B parameter scale, and even exceeding explicit CoT on the out-of-domain average across benchmarks such as GSM-Hard, MultiArith, and SVAMP. Strong performance is maintained across model sizes, in contrast to prior latent methods (such as CODI, KaVa, and SIM-CoT), which display increasing divergence from explicit CoT as model capacity increases.

Critically, LOTUS reduces "thought-phase" inference latency by $2.5\times$ on compact math-expressions and $6.9\times$ on natural-language rationale stress-tests, representing substantial walltime gains without loss of accuracy. LOTUS-aux performs comparably at large scales, but is less robust on smaller backbones.

Figure 2: LOTUS bridges the latent–explicit CoT accuracy gap at scale on GSM8K test. Accuracy of latent methods degrades with backbone size, while LOTUS matches explicit CoT and yields significant latency improvements.

Ablation and Architectural Insights

Both width (tokens per block, $c$ ) and depth (iterations, $R$ ) are crucial hyperparameters. Increasing $c$ improves accuracy up to a moderate value, after which returns diminish; increasing $R$ 0 continues to yield gains until saturation. Notably, inference latency is nearly invariant to block width due to parallel processing of latents.

Comparison with Jacobi-style latent refinement (e.g., KaVa, PCCoT) reveals that LOTUS’s fixed block alignment and direct token-level supervision are decisive in preventing training collapse at larger model scales.

Latent Representation Analyses

Readouts of the post-loop latent blocks via the LM head recover gold CoT steps with high per-position top-1 accuracy (over 70%), and significant probability mass is placed on alternative valid reasoning chains—indicative of genuinely CoT-aligned, interpretable latent spaces as opposed to brittle, memorized trajectories.

Figure 3: Multi-path readout probability analysis — unseen, valid intermediate steps not present in the training chain still surface with high probability, evidencing that latents capture more than a single CoT trajectory.

Further, explicit ablation of the two supervision losses in LOTUS (step-aligned and answer losses) demonstrates their complementary nature: the parallel latent supervision ensures support coverage over correct intermediate steps, while the answer loss steers the global latent configuration toward answer-consistent states.

Theoretical and Empirical Implications

The empirical evidence confirms that looped Transformers, when paired with structured, parallel step-supervision, can yield latent computation that is both efficient and competitive with explicit methods. Theoretically, this advances the understanding of how recurrent depth and latent-space computation enable the modeling of complex reasoning chains, as backed by recent analyses on the power and limitations of latent reasoning and looped architectures (Zhu et al., 29 Oct 2025, Geiping et al., 7 Feb 2025, Wei et al., 24 Sep 2025).

LOTUS’s grounding in gold CoT tokens directly addresses the instability and drift that affect prior latent approaches at scale (Zou et al., 1 Feb 2026, Pellat-Finet, 25 Sep 2025), and the block-wise alignment strategy facilitates interpretability and flexible adaptation to task structure.

Outlook and Future Directions

While evaluated in the context of math reasoning benchmarks, these findings suggest that bridging latent and explicit reasoning via looped latent computation is a scalable paradigm for efficient reasoning in LLMs. This raises promising avenues for extension:

Adaptive Control: Making block count $R$ 1, block width $R$ 2, and loop depth $R$ 3 fully dynamic and input-dependent, potentially integrating RL-based stopping criteria or gating (Ning et al., 26 Nov 2025, Song et al., 2 Mar 2026).
Task Transferability: Assessing efficacy on non-mathematical, compositional tasks, or formal language recognition (Altabaa et al., 15 Oct 2025).
Pretraining and Generalization: Transferring the approach to large-scale, multi-task pretraining for broad reasoning capabilities (Zhu et al., 29 Oct 2025, Zeng et al., 27 Sep 2025).
Latent Space Modelling: Further enhancing interpretability and support for parallel exploration in latent space, possibly integrating diffusion-based or mixture-of-depth approaches [UrM4MNRYZm, (Raposo et al., 2024)].

Conclusion

This work demonstrates that looped Transformers equipped with parallel, token-aligned supervision can overcome the expressivity and efficiency limitations of prior latent reasoning architectures. LOTUS achieves explicit CoT-level performance at large model scales with a drastic reduction in inference cost and transparent latent representations. The methodology and observations herein set a new standard for scalable, interpretable, and efficient latent reasoning, motivating further research on adaptive and domain-general latent computation in LLMs.

References: