Visual Tree Search (VTS)

• Visual Tree Search (VTS) is a planning framework that operates directly on high-dimensional visual data using deep generative and predictive models.
• It integrates modules for visual state encoding, transition modeling, action sampling, and reward evaluation to facilitate decision-making in robotics and multimodal reasoning.
• VTS improves robustness to noisy and occluded observations while offering sample efficiency and scalability through modular, data-driven planning.

Visual Tree Search (VTS) refers to a class of planning algorithms that combine tree search paradigms—most commonly Monte Carlo Tree Search (MCTS)—with visual or pixel-based representations of state, often powered by deep generative or predictive models. VTS underpins a range of recent advances in vision-based decision-making, robotics, and multimodal reasoning, where future state predictions, action proposals, and value assignments are performed directly in high-dimensional image spaces. Its central insight is that the planning process should operate on visually grounded, data-driven world models, leveraging composable modules or learned transitions, rather than purely symbolic or hand-engineered simulators.

1. Formal Definitions and General Framework

VTS generalizes classical tree search to operate over observation or image spaces. States are encoded as high-dimensional arrays (e.g., RGB-D images or latent representations thereof), and state transitions are given by data-driven models—either deterministic predictors, conditional generative models, or diffusion-based dynamics networks. At each node in the tree, possible actions (physical, linguistic, or multimodal) are enumerated; successors are constructed via learned predictors; and candidate rollouts are evaluated using either learned or heuristic reward models.

Common formalizations include:

• Object-retrieval/robotics VTS: where $s_t$ is an RGB-D workspace image, $a_t$ is a parametrized manipulation (e.g., push/grasp), and the transition function $f_\theta(s_t, a_t)$ is a neural predictor of object poses or next images (Huang et al., 2021).
• POMDP-based VTS: where the search occurs over beliefs of latent states, observations are high-dimensional images, and transitions and observation models $(T, Z_\theta, G_\xi)$ are realized as deep generative networks (Deglurkar et al., 2021).
• LVLM reasoning VTS: where $s_t = (V_t, A_{t-1})$ captures the current visual canvas and chain-of-thought context, and actions span both external visual tools and textual steps (Wang et al., 12 Apr 2025).
• Model-based robotic planning VTS: where VTS is realized via action-conditioned diffusion world models, pixel-level state rollouts, and MCTS planners, coupled to behavior-executing MPC (Khorrambakht et al., 4 Nov 2025).

The key differentiator of VTS versus conventional planning is the centrality of vision-based representations and the compositional integration of generative models for next-state or observation synthesis during tree expansion and rollout.

2. Algorithmic Components

VTS implementations typically consist of several tightly coupled modules:

1. Visual State Representation: The state at each search node is either a raw or preprocessed image, a scene graph/mesh, or a tuple comprising both visual and textual context (for LVLMs) (Wang et al., 12 Apr 2025, Huang et al., 2021).
2. Transition and Observation Modeling:
   • Pixel-to-pixel predictive networks (e.g., DIPN, U-Nets, VAEs, CVAEs, DDPMs) for deterministic or stochastic transitions (Huang et al., 2021, Khorrambakht et al., 4 Nov 2025, Deglurkar et al., 2021).
   • Conditional observation generators to predict likely sensory outcomes from hypothesized states (Deglurkar et al., 2021).
3. Action Proposal and Sampling:
   • Discrete or continuous parameterizations—e.g., planar pushes, end-effector velocities—sampled via learned diffusion priors or enumeration around object contours (Huang et al., 2021, Khorrambakht et al., 4 Nov 2025).
   • LVLM multimodal action generation: textual tokens, tool/visual function calls (Wang et al., 12 Apr 2025).
4. Search Procedure:
   • MCTS with UCT or progressive widening for continuous/high-dimensional spaces (Huang et al., 2021, Deglurkar et al., 2021, Khorrambakht et al., 4 Nov 2025).
   • Beam search combined with rollout simulation for multimodal reasoning (Wang et al., 12 Apr 2025).
   • In all cases, rollouts are performed by chaining generative models and actions to synthesize visual trajectories.
5. Reward/Scoring Models:
   • Task-specific or generalizable (e.g., geometric error, DINOv2 embedding, video-ranked “Rand2Reward”) (Khorrambakht et al., 4 Nov 2025).
   • LVLM “vote” mechanisms for self-supervised reward assignment (Wang et al., 12 Apr 2025).

Table: Representative VTS Algorithmic Building Blocks

Module	Typical Realization	Reference
Transition Model	DIPN, U-Net, DDPM, VAE	(Huang et al., 2021, Khorrambakht et al., 4 Nov 2025, Deglurkar et al., 2021)
Action Sampler	Contour sampling, diffusion	(Huang et al., 2021, Khorrambakht et al., 4 Nov 2025)
Reward Model	Grasp predictor, embedding	(Huang et al., 2021, Khorrambakht et al., 4 Nov 2025)
Search Algorithm	MCTS, beam search	(Huang et al., 2021, Wang et al., 12 Apr 2025, Khorrambakht et al., 4 Nov 2025)
Observation Gen.	CVAE, generative networks	(Deglurkar et al., 2021)

3. Methodological Instantiations

3.1 Object Retrieval with Visual Foresight Trees

Visual Foresight Trees (VFT) perform VTS for nonprehensile rearrangement in cluttered scenes. The algorithm formulates the scene state as a 224×224×4 (RGB-D) image, masks, and target labels. Push and grasp actions are enumerated, and a deep push-outcome predictor (DIPN) $f_\theta$ predicts per-object pose shifts for each candidate push. MCTS is then executed in visual space: nodes represent post-action images, rolled out up to a fixed depth $D^*$, and scored against a learned graspability function. The UCT selection statistic uses a top-m average to robustly select promising expansions. Empirically, VFT achieves near-perfect success rates and reduced action counts in both simulation and real-robot settings relative to prior model-based and model-free approaches (Huang et al., 2021).

3.2 Vision POMDP Planning via Compositional VTS

VTS for vision-based POMDPs employs particle filtering to represent state uncertainty and advances the belief with differentiable particle filtering (DPF) updates. Deep generative models realize the observation density $Z_\theta(o|s)$ and conditional observation generator $G_\xi(o|s)$. MCTS is performed over belief nodes, and tree expansion/simulation involves sampling from the transition, generating synthetic observations, and computing likelihoods. All models are trained offline, enabling fast and robust POMDP planning that generalizes to significant visual noise and novel reward structures, maintaining >97% task success across diverse test-time distribution shifts. Comparative results show superior performance and efficiency over DualSMC, DVRL, and PlaNet (Deglurkar et al., 2021).

3.3 Multimodal Tree Search for LVLM Reasoning

In the VisuoThink framework, VTS underpins stepwise visual-verbal reasoning. State nodes are (visual canvas, textual chain-of-thought), actions are textual or tool-based, and the search is a D-step beam expansion with B-width progressive candidate sampling per step. Each candidate is subjected to a “rollout” (full answer simulation), and the LVLM self-votes on the result’s promise. This procedure enables “slow visual thinking,” substantially boosting geometry and spatial reasoning benchmarks (e.g., +50pp with increased D, peak gains at B=3). The planner leverages cross-modal transformer fusion and exploits test-time lookahead with no model retraining (Wang et al., 12 Apr 2025).

3.4 Action-Conditioned Visual World Models in Robotic Planning

WorldPlanner integrates a conditional pixel-level diffusion world model, a learned action-diffusion policy, and MCTS. Tree search nodes are pixel images; transitions and expansions use denoised outputs of the world model under the action prior. MCTS’s selection statistic is UCB1, and reward functions can be geometric, embedding-based, or learned from video ranking. Planning outputs are tracked via a CEM-based MPC for closed-loop execution. This model-based VTS framework outperforms behavior cloning by 15–30 percentage points on robotic manipulation tasks, supports unstructured play data training, and scales efficiently on a single GPU for moderate horizons (Khorrambakht et al., 4 Nov 2025).

4. Advantages, Robustness, and Limitations

VTS affords several practical and theoretical benefits:

• Robustness to Observational Shift: Training observation and proposer models on high-entropy, diverse data lends VTS greater invariance to noise, occlusion, and test-time distribution drift. Task success rates remain >97% even under substantial unseen visual corruption (Deglurkar et al., 2021).
• Sample Efficiency: Offline modular training decouples model learning from online planning, reducing dependence on interaction-heavy end-to-end RL (Deglurkar et al., 2021).
• Generalizability: Reward and rollout components may be swapped without full retraining, facilitating re-planning under new objectives or dynamics.
• Interpretability: Explicit visual rollouts aid in tracing decision pathways, debugging, and error analysis.
• Scalability: Diffusion-based world models and modular policies allow planning at pixel-level granularity, supporting open-ended visual tasks (Khorrambakht et al., 4 Nov 2025).

However, limitations remain:

• Model Capacity and Distribution Shifts: Planning performance degrades when predictions fall far outside the training data, with rare semantics (e.g., complex tool usage) not robustly represented (Khorrambakht et al., 4 Nov 2025).
• Computational Overhead: In LVLM and high-resolution visual settings, each tree node expansion may require multiple rollouts, incurring $O(D \cdot B)$ black-box model calls per search (Wang et al., 12 Apr 2025).
• Tool and Executor Dependence: Multimodal VTS (e.g., VisuoThink) assumes reliable and consistent tool chaining (matplotlib, physics engines) (Wang et al., 12 Apr 2025).
• Limited Search Horizon: Effective lookahead depends on the predictive horizon and stability of generative models; longer horizons or deeper trees increase cost and compounding error.

5. Empirical Performance and Benchmarks

A summary of empirical results as reported in the key references:

Setting	VTS Variant	Baseline(s)	Success Rate (%)	Steps/Actions
Dense clutter obj. retrieval (Huang et al., 2021)	VFT	VPG, PGN, DIPN	100	2.00 (sim)
Vision POMDP (3D LD) (Deglurkar et al., 2021)	VTS	DualSMC, DVRL, PlaNet	99.6	18.2
LVLM geometry (Geomverse-109) (Wang et al., 12 Apr 2025)	VTS	CoT, VisualSketchpad	28.9	n/a
Pixel-based manipulation (≤5cm) (Khorrambakht et al., 4 Nov 2025)	MCTS+VTS	BC, ACT, diff. policy	92	n/a

Contextually, VTS consistently outperforms model-free or policy-based alternatives across object manipulation, sensorimotor POMDPs, and LVLM-driven reasoning, with particular robustness to observation corruption and reward redefinition.

6. Applications and Extensions

VTS provides a general principle for high-dimensional visual planning, with demonstrated applications in:

• Robotic Manipulation: Planning nonprehensile rearrangements, deformable object pushing, selective retrieval from clutter (Huang et al., 2021, Khorrambakht et al., 4 Nov 2025).
• Vision-based POMDPs: Navigation, light-dark problems, occlusion-robust reward search (Deglurkar et al., 2021).
• Multimodal Reasoning: Math/geometry problem solving, visual navigation, tiling, and text-tool integration via LVLMs (Wang et al., 12 Apr 2025).
• Open-loop and closed-loop control: Through integration with MPC for robust real-world tracking (Khorrambakht et al., 4 Nov 2025).

Potential extensions include explicit value-function learning for deeper search, adaptive tree parameters (beam-width, depth), richer tool-sets (physics, 3D rendering), and foundation model-enhanced planners to expand beyond empirical data regimes (Khorrambakht et al., 4 Nov 2025, Wang et al., 12 Apr 2025).

7. Theoretical and Practical Significance

VTS marks a transition in planning algorithms from symbolic or feature-based state spaces toward end-to-end data-driven models directly in observation space. By unifying generative learning with online decision search, VTS has demonstrated that robust, adaptable, and interpretable planning is possible in highly complex, noisy, real-world environments without requiring extensive hand-engineering or reward-dependent retraining (Deglurkar et al., 2021, Khorrambakht et al., 4 Nov 2025). The approach provides a foundation for future intelligent agents that leverage both vision and compositional reasoning to operate in dynamic, structured, or multimodal domains.