Traversability-Oriented Manipulation

• Traversability-oriented manipulation is a paradigm integrating perception, planning, and control to actively modify obstacles—such as repositioning or stabilizing objects—to improve robot navigation.
• The framework employs reinforcement learning guided by visual affordance maps and manipulability priors to steer robots toward kinematically favorable configurations for large base displacements.
• Empirical evaluations demonstrate improved sample efficiency and successful simulation-to-reality transfer, with policies converging up to 12× faster than conventional methods.

Traversability-oriented manipulation is a paradigm within mobile robotics focused on learning and executing manipulation strategies that directly enhance the subsequent navigability of the robot in dynamic environments. Instead of treating manipulation and navigation as isolated subproblems, traversability-oriented manipulation integrates perception, planning, and control to select and execute actions that maximize the robot's ability to traverse an environment, particularly when movable obstacles block the path. This approach prioritizes interactive behaviors such as clearing, repositioning, or stabilizing environment elements to enable safe and efficient navigation. The methodology centers on reinforcement learning with explicit visual affordance segmentation and kinematic manipulability priors that guide exploration toward body poses amenable to large base displacements after manipulation (Zhang et al., 18 Aug 2025).

1. Reinforcement Learning Formulation for Manipulate-to-Navigate

The traversability-oriented manipulation framework employs a Markov decision process (MDP) formulation:

• State space: At each timestep $t$, the robot receives an RGB-D observation $I_{rgbd}\in\mathbb{R}^{480\times640\times4}$. Two encoders extract:
  • Policy state $s_t = \psi(I_{rgbd}) \in \mathbb{R}^{348}$ (Dinov2 features).
  • Affordance state $$s_t^{aff} = \Phi(I_{rgbd}) \in \{0,1\}^{480\times640}$$ (binary affordance from Mobile-SAM).
• Action space: Pixel coordinates in the image $$a_t = (I_x, I_y) \in \{0\ldots639\} \times \{0\ldots479\}$$, back-projected to a 3D end-effector target via camera intrinsics $K$ and extrinsics $[R|t]$.
• Dynamics: Transitions $P(s_{t+1}|s_t,a_t)$ use detailed Spot+Isaac Sim simulation, incorporating arm IK and base motion constraints.
• Reward function: Weighted sum of four terms:

  $$R(s,a)=w_1 r_{ik}(s,a)+w_2 r_{balance}(s,a)+w_3 r_{arm}(s,a)+w_4 (r_{move}(s,a)\cdot\text{reach})$$

  Key components include IK feasibility, base stability penalties, arm-task proximity, and the base traversal distance $r_{move}(s,a) = \text{Dist}(s,a)$, gated by a binary reach flag.

• Learning algorithm: Double DQN (DDQN) regularized with a KL-divergence prior:

  $$L(\theta)=\mathbb{E}_{(s,a,r,s',s'^{aff})}[(Q_\theta(s,a,s^{aff})-y)^2]+\lambda\sum_{t=0}^T\gamma^t KL[\pi(a_t|s_t)\|p(a_t|c_r)]$$

  Here, $\pi(a|s)$ is the learned policy and $p(a|c_r)$ is the manipulability prior.

This formulation allows the policy to efficiently discover manipulation actions that unlock large feasible base movements, instead of merely focusing on task-specific end-effector placements.

2. Manipulability Priors for Dexterous Arm Configurations

Manipulability priors bias the policy towards kinematic configurations that facilitate extensive base traversal post-manipulation.

• Yoshikawa index: For candidate 3D end-effector pose $x=(x,y,z)$, manipulability is defined as $w(x) = \sqrt{\det(J(x) J(x)^T)}$, where $J(x) \in \mathbb{R}^{6\times n}$ is the robot arm's geometric Jacobian.
• Offline map generation: The workspace is sampled, discarding infeasible (IK-failed, self-colliding) poses. $w(x)$ is computed for all feasible $x$, which are then projected into the camera image plane. Pixel-level manipulability maps are normalized to form $p(a=(u,v)|c_r)$, a discrete prior over actions, reflecting expected traversability given placement.
• Algorithmic integration: The KL-divergence penalty in the RL loss constrains policy $\pi(a|s)$ not to deviate significantly from $p(a|c_r)$, empirically guiding exploration towards regions associated with higher post-manipulation traversability.

A plausible implication is that employing manipulability priors can systematically accelerate policy discovery in environments where arm pose quality determines navigation feasibility.

3. Visual Affordance Maps for Action Selection

Pixel-level affordances are used to identify meaningful and feasible manipulation sites.

• Affordance extraction: The Mobile-SAM segmentation model produces a binary map $s^{aff} = \Phi(I_{rgbd})$, where $s^{aff}(u,v)=1$ indicates visually identified manipulable surfaces (e.g., suitable for grasping or pushing).
• Policy pruning: In the Q-network $Q_\theta(s,a,s^{aff})$, only affordance-positive pixels are considered valid actions (non-affordance pixels receive a large negative Q-value). The affordance mask is concatenated or used as attention, ensuring $\pi(a|s,s^{aff})$ selects meaningful contacts.
• Sample efficiency: Empirical results indicate that visual affordance guidance is the largest single driver of improved sample efficiency, allowing policies to avoid extensive random exploration of non-contact regions.

This suggests that real-time visual affordance segmentation is a critical component for rapidly focusing exploration and promoting effective manipulation in traversability-oriented tasks.

4. Algorithmic Workflow and Integration

The learning and execution loop is defined as follows (see Algorithm 1 from (Zhang et al., 18 Aug 2025)):

Initialize manipulability prior p(a|c_r) Initialize Q-network parameters θ, target network θ′←θ Initialize replay buffer D←∅, ε←ε₀ for episode=1…N do observe I_t s_t=ψ(I_t), s_t^aff=Φ(I_t) with prob ε: a_t∼Uniform(valid affordance pixels) else: a_t=argmax_a Q_θ(s_t,a,s_t^aff) execute a_t → observe r_t, next image I_{t+1} store (s_t,s_t^aff,a_t,r_t,s_{t+1},s_{t+1}^aff) in D sample minibatch B from D compute targets y via DDQN compute L(θ)=MSE+λ·KL[π(·|s)‖p(·|c_r)] over B gradient-step θ←θ−η∇_θL(θ) occasionally update θ′←θ, ε←decayed(ε) end return π*(a|s,s^aff)=argmax_a Q_θ(s,a,s^aff)

By leveraging both the manipulability prior and visual affordance mask, the policy rapidly converges on arm configurations that maximize traversability, efficiently integrating perception, kinematics, and RL optimization.

5. Traversability-Oriented Manipulate-to-Navigate Tasks

Two canonical tasks are used to validate the framework:

• Spot-Reach (simulation and real):
  • Goal: Place the Spot arm's end-effector in a specified tabletop goal region (green rectangle) to enable maximum forward base movement while maintaining the grasp.
  • Metrics: Success is defined by hand entry into the region ("reach" = 1); base traversal distance is recorded thereafter.
• Spot-Door (simulation):
  • Goal: Employ the arm to push open a spring-loaded sliding door, maintaining contact to create sufficient navigation clearance, then driving forward.
  • Metrics: Success requires door's projected image-length $L_{door}$ to fall below threshold $L_{threshold}$, plus base advancement.

Both experiments rely on NVIDIA Isaac Sim for high-fidelity Spot dynamics including joint/IK feasibility, collision, and balancing constraints.

6. Empirical Evaluation and Policy Transfer

Quantitative results demonstrate the efficacy of traversability-oriented manipulation:

Variant	Success (Reach, Sim)	Steps to 85% Success	Success (Door, Sim)	Success (Reach, Real, 10 trials)
DDQN	~85%	~12,000	~60%	1/10
DDQN-P	–	–	~75–80%	1/10
DDQN-A	–	–	~75–80%	8/10
DDQN-AP	~85%	~4,000 (full conv)	~75–80%	8/10

• Sample efficiency: DDQN-AP (both guidance methods) converges 3×–12× faster than baseline DDQN. Visual affordance (A) confers the largest efficiency gains.
• Base traversal: Solutions with priors/affordances achieve +0.03 m greater traversal in simulation.
• Door task: Visual and prior guidance yield success rates substantially above baseline, but with smaller average traversal due to necessary backward balancing.
• Real-world transfer: Unaltered DDQN-AP network achieves 8/10 success on Spot hardware, with base advancements up to 0.5 m post-grasp. Failures are attributed to perception or camera alignment errors.

A plausible implication is that synergistic affordance-prior integration enables robust simulation-to-reality transfer in complex manipulate-to-navigate settings.

7. Significance and Context within Mobile Manipulation

Traversability-oriented manipulation offers a principled, empirically validated framework for mobile robot policy learning in environments demanding active interaction for path clearing. By fusing manipulability priors and online visual affordance segmentation into a KL-regularized RL pipeline, the methodology achieves rapid convergence and successful policy transfer—key limitations in traditional decoupled navigation/manipulation approaches. The empirical findings demonstrate that such guidance can accelerate learning by an order of magnitude and generalize to physical robots, provided robust perception and kinematic modeling are maintained (Zhang et al., 18 Aug 2025). This framework is applicable wherever active environment interaction modulates path traversability, with future directions likely including additional affordance modalities, richer priors, and more complex physical tasks.