Imagination Helps Visual Reasoning, But Not Yet in Latent Space

Abstract: Latent visual reasoning aims to mimic human's imagination process by meditating through hidden states of Multimodal LLMs. While recognized as a promising paradigm for visual reasoning, the underlying mechanisms driving its effectiveness remain unclear. Motivated to demystify the true source of its efficacy, we investigate the validity of latent reasoning using Causal Mediation Analysis. We model the process as a causal chain: the input as the treatment, the latent tokens as the mediator, and the final answer as the outcome. Our findings uncover two critical disconnections: (a) Input-Latent Disconnect: dramatic perturbations on the input result in negligible changes to the latent tokens, suggesting that latent tokens do not effectively attend to the input sequence. (b) Latent-Answer Disconnect: perturbations on the latent tokens yield minimal impact on the final answer, indicating the limited causal effect latent tokens imposing on the outcome. Furthermore, extensive probing analysis reveals that latent tokens encode limited visual information and exhibit high similarity. Consequently, we challenge the necessity of latent reasoning and propose a straightforward alternative named CapImagine, which teaches the model to explicitly imagine using text. Experiments on vision-centric benchmarks show that CapImagine significantly outperforms complex latent-space baselines, highlighting the superior potential of visual reasoning through explicit imagination.

• The paper demonstrates that latent tokens in MLLMs lack strong causal linkage between input and output, questioning their role as effective semantic mediators.
• Causal mediation analysis shows that interventions on latent tokens have minimal impact on outputs, highlighting limitations in current latent visual reasoning approaches.
• The proposed CapImagine method replaces latent tokens with explicit textual captions, significantly improving both performance and interpretability on multiple benchmarks.

Authoritative Summary of "Imagination Helps Visual Reasoning, But Not Yet in Latent Space" (2602.22766)

Motivation and Background

The paper investigates latent visual reasoning (LVR) in Multimodal LLMs (MLLMs), specifically questioning whether the hidden states (latent tokens) used in current LVR approaches genuinely mediate visual reasoning akin to human imagination. While LVR has empirically performed well on vision-centric tasks, its mechanistic validity—whether latent tokens causally link input to output—remains unclear. The authors interrogate this paradigm using Causal Mediation Analysis, systematically perturbing both input and latent states to measure causal connections and semantic encoding.

Causal Mediation Analysis and Empirical Findings

The core analytical framework models reasoning as a causal chain: input $X \rightarrow$ latent tokens $Z \rightarrow$ answer $Y$. The authors execute perturbation experiments to diagnose dependencies:

Input-Latent Disconnect: Alterations to $X$ (input sequence) result in minimal change in $Z$ (latent tokens), as measured by inter-instance cosine similarity. Latent tokens across instances and tasks collapse into highly similar representations, indicating a loss of input-dependent semantics and rapid degeneration during autoregressive generation. Monet, LVR, and Mirage models, despite distinct supervision protocols, all exhibit this degenerative pattern.

Latent-Answer Disconnect: Direct interventions on $Z$ (e.g., setting all latent tokens to fixed tensors, Gaussian noise injection, or extreme value substitution) yield only marginal changes to $Y$ (answers). Performance either remains unchanged or slightly improves, suggesting that $Z$ has minimal causal effect on $Y$. Only severe interventions (stage-2 variant in Mirage with extreme token collapse) reduce performance, confirming a weak coupling.

Semantic Probing: Directly using latent embeddings as input for auxiliary VQA or compositional reasoning fails to achieve meaningful accuracy; latent tokens encode insufficient task-relevant visual semantics compared to standard visual or textual features.

Overall, these results indicate that current LVR implementations do not maintain effective causal mediation nor meaningful semantic representations in latent space. Instead, latent tokens act more as placeholders or soft prompts rather than vehicles for visual imagination.

CapImagine: Explicit Text-Space Imagination

To address the shortcomings of LVR, the authors propose CapImagine, a method that replaces latent tokens with explicit textual descriptions of visual manipulations (zoom, highlight, mark). Intermediate reasoning images are verbalized as text captions, thus enabling the model to imagine in natural language, grounded in concrete visual evidence.

Dataset Construction: Monet-SFT-125K is rewritten by generating textual captions for manipulated images using Qwen3-VL-4B. Rigorous filtering and refinement are applied to ensure logical coherence and to mitigate ambiguity and answer conflicts, resulting in a high-quality subset for training.

Ablation Studies: Removing text-space imagination (replacing captions with a generic token) and filtering both result in significant performance drops, confirming the causal role of text-driven imagination and the necessity of data quality control.

Results: Benchmark Comparison and Causal Dependency

CapImagine significantly outperforms latent-space models (Monet, LVR) across multiple benchmarks:

• HR-Bench-8K: +4.0% over Monet
• MME-RealWorld-Lite: +4.9% over Monet
• TableVQA: +6.1% improvement
• BLINK (compositional/multi-view): >10 point improvement over Monet and LVR

Causal dependency analysis confirms that text-form imagination tokens in CapImagine are highly sensitive to input and intervention, with low cross-instance similarity and strong causal influence on final answers. Perturbing the imagination trace produces pronounced performance degradation, confirming the central role of explicit imagination.

Efficiency evaluation shows CapImagine achieves comparable inference speed to Monet and is nearly twice as fast as tool-based approaches (DeepEyes), supporting practical utility.

Implications and Future Directions

This work decisively questions the necessity and effectiveness of latent visual reasoning in its current form. The findings imply that LVR methods, as presently implemented, lack interpretable and causally effective mediation. The explicit text-space imagination approach not only enhances interpretability but also aligns more closely with human reasoning patterns, albeit at the cost of potential granularity loss due to linguistic abstraction.

Pragmatically, CapImagine provides a more faithful and efficient reasoning mechanism, challenging the community to rethink the reliance on latent tokens and to explore novel ways of encoding visual imagination. Theoretically, this exposes the challenge of constructing high-quality, information-rich latent chains—future research may focus on designing more discriminative, semantically dense, and causally grounded latent representations, potentially leveraging advances in continuous space modeling or hybrid reasoning chains.

Conclusion

The paper systematically demonstrates that imagination can improve visual reasoning in MLLMs, but latent-space paradigms do not yet realize this benefit. Explicit text-driven imagination (CapImagine) yields superior effectiveness, stronger causal relationships, and competitive efficiency. The work guides both practical model design and theoretical exploration of reasoning mechanisms, highlighting a fundamental gap in current LVR and motivating further research in causal and interpretable visual imagination (2602.22766).