Papers
Topics
Authors
Recent
Search
2000 character limit reached

FurnitureVLA: Learning Long-Horizon Bimanual Furniture Assembly with Vision-Language-Action Model

Published 1 Jul 2026 in cs.RO and cs.AI | (2607.01212v1)

Abstract: Current work on robot furniture assembly mostly focuses on toy-scale settings or single-arm manipulation. We introduce FurnitureVLA, the first systematic study of real-scale bimanual furniture assembly using Vision-Language-Action models (VLAs). We formalize the task, develop a scalable simulation pipeline for expert data generation and evaluation, and build a VR teleoperation system for single-operator bimanual control to collect high-quality real-world demonstrations. To address extreme long-horizon assembly with up to 7 subtasks and 1550 control steps, we propose a progress-enhanced VLA, finetuned on semantically grounded subtasks, that jointly predicts actions and a continuous progress signal, enabling automatic subtask transitions and reducing compounding errors during inference. We further study perception and control design factors that critically affect precision in real-scale assembly. FurnitureVLA improves average simulation success from 48% to 80% compared to baselines across three furniture types, with an additional 21% gain from our design factor study. We validate on a real Kinova Gen3 platform with only 16% drop on the hardest task.

Summary

  • The paper introduces a progress-enhanced VLA model that decomposes complex furniture assembly into semantically grounded subtasks for improved stability.
  • It achieves an average 80% assembly success rate in simulation by leveraging coordinated dual-arm operations, refined geometric alignment, and multiple camera viewpoints.
  • Real-world validation on Kinova Gen3 demonstrates the model's efficacy in mitigating compounding errors over long-horizon bimanual tasks with minimal performance drop.

Authoritative Essay on "FurnitureVLA: Learning Long-Horizon Bimanual Furniture Assembly with Vision-Language-Action Model"

Introduction and Motivation

Vision-Language-Action models (VLAs) promise generalist robotic capabilities beyond short-horizon manipulation. However, end-to-end VLA policies for real-scale, long-horizon, and contact-rich tasks, especially those requiring bimanual coordination, have remained unresolved due to compounding error, high geometric precision demands, and covariate shift. "FurnitureVLA: Learning Long-Horizon Bimanual Furniture Assembly with Vision-Language-Action Model" (2607.01212) presents the first systematic study tackling these challenges in full-scale, dual-arm furniture assembly scenarios.

Distinct from prior work focusing on single-arm, toy-scale, or highly modularized settings, this work formalizes real-scale furniture assembly requiring up to 7 sequential subtasks, over 1500 control steps, and high-precision geometric alignment. It contributes a scalable simulation and data collection pipeline and introduces a progress-enhanced VLA architecture explicitly designed for robust subtask transition and long-horizon stability.

Task Formalization and System Design

The study formalizes the task of bimanual assembly for three complex IKEA items: LACK side table, KALLAX shelf, and IVAR chair. These tasks demand coordinated dual-arm operations, strict temporal sequencing, and fine geometric tolerances beyond prior benchmarks.

The experimental system comprises two core elements:

  1. Simulation Pipeline: Expert demonstrations are generated in simulation via motion planning, enabling diverse, validated trajectories compatible with bimanual reachability and avoiding kinematic singularities.
  2. VR Teleoperation for Demonstration Collection: A bespoke VR interface enables a single operator to perform precise, temporally efficient, and reproducible bimanual manipulation on a real Kinova Gen3 platform with mirrored control, preset grasp primitives, and synchronized arm modes. Figure 1

    Figure 1: Schematic breakdown of assembly task steps, illustrating subtask compositionality and complexity across LACK, KALLAX, and IVAR furniture models.

    Figure 2

    Figure 2: Hardware and observation setup, marking the placement of Kinova Gen3 arms, distinct grippers, and multi-view camera configuration (front, rear, wrist-mounts) for robust pose estimation and occlusion mitigation.

Progress-Enhanced VLA Architecture

Compounding error and distribution drift represent fundamental obstacles in monolithic long-horizon VLA finetuning. FurnitureVLA addresses this by decomposing demonstrations into semantically grounded subtasks, each associated with concise natural language instructions; importantly, subtask transitions are defined at post-retreat, contact-free arm states, which stabilizes subsequent policy rollouts by narrowing subtask-wise initial state distributions.

A key architectural innovation is the augmentation of the action output space with a scalar continuous progress signal. During inference, this enables the policy to trigger subtask transitions automatically via a threshold-based filter on the predicted progress, obviating the need for external phase estimators or computationally expensive reasoning modules. Figure 3

Figure 3: End-to-end VLA data pipeline from motion planning and VR teleoperation through policy training, chunked action prediction, and continuous progress modeling.

Experimental Evaluation

Long-Horizon Assembly Performance

Extensive simulation experiments reveal substantial gains:

  • FurnitureVLA achieves an average assembly success rate of 80% across the three furniture types, up from the 48% obtained by monolithic VLA finetuning baselines, with zero success from zero-shot π0.5\pi_{0.5} policies.
  • Success rates decrease with task complexity: LACK (98%) > KALLAX (85%) > IVAR (56%), consistent with increasing action horizon and subtask count.
  • The progress-enhanced architecture specifically mitigates failure modes concentrated around challenging mid-sequence subtasks requiring tight bimanual coordination and multi-surface alignment, as observed in the IVAR assembly sequence. Figure 4

    Figure 4: Subtask-wise assembly success rates, showing dropoff points and distribution of failures across the assembly sequence for all evaluated policies and configurations.

Perception and Control Factors

Targeted ablations examine the impact of:

  • Action horizon: Frequent replanning (5 or 10 steps) enhances recovery from local errors, while longer horizons (25) benefit smoothing for large/heavy parts (KALLAX).
  • Temporal ensembling: Online averaging of predicted action chunks with discount weighting (λ=−0.1\lambda = -0.1) consistently stabilizes bimanual motion and boosts overall success (up to +15%).
  • Camera viewpoints: Addition of a rear camera markedly improves performance by reducing occlusion-induced ambiguity, outperforming both single-front-view and depth-only setups.
  • Input resolution: Visual input at 448×448448 \times 448 in the SigLIP vision encoder achieves the best geometric alignment.

A design factor optimization yields 21% additional gain over the default configuration.

Data Efficiency and Progress Signal Ablations

  • Success scales with demonstration number, but plateaus after 50% of maximal data budget, suggesting diminishing returns past a moderate threshold.
  • Discretized progress signals fail to produce feasible controllers for assembly, demonstrating the necessity of continuous, monotonic progress modeling for robust subtask transition.

Real-World Validation

On the Kinova Gen3 hardware, using data from VR teleoperation:

  • The system achieves up to 40% full-sequence success on the IVAR chair in the real world, with only a 16% drop from simulation, despite additional noise, hardware inaccuracies, and visually challenging conditions.
  • Per-part (single subtask initialized) success metrics remain considerably higher, confirming that accumulated compounding error across subtasks is the principal source of long-horizon failures.

Emergent behaviors such as self-corrective regrasping and fine-tuned magnet alignment are observed, indicating the policy's capacity to learn subtle manipulation corrections from demonstrator bias. Figure 5

Figure 5: Real-system demonstration sequence highlighting coordinated bimanual grasps, corrective maneuvers, and final insertion steps on the IVAR chair assembly.

Figure 6

Figure 6: Quantitative real-world performance: full-sequence and per-part assembly success rates across the IVAR chair's seven subtasks.

Implications and Future Outlook

FurnitureVLA demonstrates that progress-aware action modeling and subtask-level decomposition are essential for scalable VLA deployment on long-horizon, high-precision, bimanual tasks. The findings have direct implications for:

  • Generalist robot design, where routines are explicitly staged and progress feedback is intrinsic, supporting continual operation without iterative human correction.
  • Task and motion planning integration for future VLA pretraining datasets, motivating systematic multi-phase annotation.
  • Design of teleoperation systems and data pipelines for data-efficient acquisition of complex dual-arm demonstrations, particularly for large, occlusion-prone assemblies.

Scaling to broader object categories, relaxing assumptions (e.g., eliminating magnets), and integrating tool use remain open challenges. A core limitation is workspace constraint due to fixed-base arms, which may be addressed by future mobile manipulator integration. Figure 7

Figure 7: Quantitative and visual survey of real-scale furniture models and their subcomponent dimensions, emphasizing the diversity and complexity in assembly scenarios.

Conclusion

This work establishes a rigorous benchmark and methodology for real-scale, long-horizon, bimanual furniture assembly with VLAs. It introduces architectural and data-centric advances for subtask stabilization, perception, and action control that collectively yield robust long-sequence policy execution. The empirical findings and tools provided set a strong foundation for future research in generalist, compositional manipulation using vision-language-action paradigms. Figure 8

Figure 8: Detailed breakdown of per-subtask success rates and ablation outcomes, reaffirming the value of continuous progress modeling, rear-view fusion, and high-resolution visual inputs for stable long-horizon assembly.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Explain it Like I'm 14

What is this paper about?

This paper shows how to teach a robot with two arms to build real, full-size IKEA-style furniture. The tasks are long (many steps) and require very precise movements, like lining up holes and slots so pieces snap together. The authors introduce FurnitureVLA, a system that gives the robot “eyes” (cameras), “language” (step-by-step instructions), and “actions” (arm and gripper moves) all in one model. They also create tools to practice safely in simulation and to collect high-quality human demonstrations using virtual reality.

What questions are the researchers trying to answer?

  • How can a robot with two arms assemble full-size furniture safely and accurately over many steps?
  • How can we train a single general model to handle different furniture types, not just one?
  • How do we decide when the robot should move from one step to the next without a human telling it?
  • Which design choices (like camera views, image resolution, or control smoothing) actually make assembly more reliable?

How did they do it?

Building a practice world (simulation)

The team built a large “video game-like” simulator where the robot can practice assembling furniture. They used motion planning (a method to compute safe, smooth paths) to generate expert demonstrations. This lets them:

  • Try many furniture types and starting positions quickly.
  • Check if the robot can reach parts without getting stuck.
  • Measure success using strict rules (for example, parts must be within about 1–2 cm and ~4° of the correct final position).

To make evaluating success clear, they compare where each part ends up relative to a chosen “base” part (like the main panel). The assembly only counts as a success if every piece is in the right place and angle.

Teaching by example (VR teleoperation)

They also built a VR system so a single human can control both robot arms at once and show the right way to do each task. Helpful features include:

  • Separate controls for moving and rotating the hands to make delicate insertions easier.
  • Predefined “grip poses” the robot can snap to for consistent grasps.
  • A synchronized mode where both arms move together like mirrored hands, useful for lifting big panels.

In the real world, they use magnets to connect parts once aligned (instead of screws). Screwing is a separate, harder problem they left for future work.

The brain: a Vision‑Language‑Action model with a progress meter

The robot uses a Vision-Language-Action (VLA) model. You give it:

  • Images from cameras (front, wrist, and a helpful rear view to see around big parts).
  • A short language instruction for the current step (for example, “Pick up the seat panel and align it with the frame”).
  • It outputs the next arm and gripper moves.

Long tasks can fail if tiny mistakes pile up over time. To reduce this “snowball effect,” the authors:

  • Break each build into clear subtasks (like pick up, align, insert, then retreat to a safe position).
  • Train the model on these shorter, labeled steps instead of the whole long task at once.
  • Add a “progress meter” (a number from 0 to 1) that the model predicts along with its actions. This tells the robot how far it is within a step, so it can automatically switch to the next step when the progress is high enough.

A clever detail: they end each step after the robot retreats to a stable, contact-free pose, not right at the moment two parts touch. That way, small mistakes don’t carry over and get worse in the next step.

Making the robot steady and precise (design factors)

They studied which choices improve precision:

  • Action horizon: the model predicts a short plan for the next few moves at once, then replans often. This can help fix small mistakes quickly.
  • Temporal ensembling: average recent action predictions to smooth out jittery movements (like stabilizing a camera video).
  • Camera setup: adding a rear camera helps see around big parts and reduces occlusion.
  • Image resolution: higher-resolution images help the robot see details for tight alignments.

What did they find?

Overall, their approach made a big difference in both simulation and the real world.

Highlights:

  • In simulation, success at finishing the entire build improved from 48% (baseline) to 80% with FurnitureVLA across three furniture types (LACK table, KALLAX shelf, IVAR chair).
  • Careful design choices (smoothing, camera views, image resolution) gave an extra average gain of about 21%.
  • In real robot tests on the hardest furniture (the IVAR chair), performance only dropped by about 16% compared to simulation, which is strong for real-world robotics.
  • A continuous “progress meter” worked well; a simpler, “step-is-done or not” signal failed because the robot couldn’t reliably detect the exact switch point in visually similar scenes.
  • More demonstrations helped, especially when going from a small to a medium dataset.
  • The robot sometimes showed “emergent corrections,” like regrasping a part or making tiny alignment fixes on its own, likely learned from the human demonstrations.

For clarity, here are a few simple takeaways from their tests:

  • Breaking long tasks into sensible steps and adding a progress meter keeps the robot on track.
  • Smoothing the control commands prevents shaky motions that can ruin tight insertions.
  • A rear camera and higher image resolution help the robot see what matters.

Why does this matter?

Getting robots to handle long, careful jobs like furniture assembly is a big step toward useful home and factory helpers. The ideas here—splitting tasks into steps, using language to guide the robot, giving it a built-in sense of progress, and tuning cameras and control smoothing—can transfer to many other long, precise tasks, such as:

  • Setting up lab equipment
  • Assembling electronics
  • Helping in warehouses or workshops

Limitations and what’s next

  • The setup uses fixed robot arms on a table, so very large furniture might be out of reach; a mobile base could help.
  • They skip screws by using magnets for now; adding tools like screwdrivers is a harder challenge for future work.

In short, FurnitureVLA shows a practical path to reliable, two‑arm, real‑size furniture assembly by combining smart training (subtasks + progress), good sensing (extra viewpoints, higher resolution), and smooth control. This moves robots closer to handling complex, real-world chores without constant human supervision.

View Paper Prompt  View All Prompts 

Knowledge Gaps

Knowledge gaps, limitations, and open questions

Below is a concise list of what remains missing, uncertain, or unexplored in the paper, framed to guide concrete next steps for future research:

  • Generalization to unseen furniture: The model is trained/evaluated on three specific IKEA items in simulation and one in the real world; it is unknown how well it transfers to new, unseen furniture geometries, materials, connectors, and assembly flows without additional finetuning.
  • Real-world coverage across tasks: Real-robot validation is only conducted on the IVAR chair; the gap to deploy on LACK, KALLAX, and other items outside the training set remains unquantified.
  • Larger state perturbations: Robustness to larger initial part misplacements and orientations (beyond ±3 cm and ±5°) and to significant scene disturbances (e.g., bumped parts, occlusions, partial assemblies shifting) is untested.
  • Screwing and tool-use: Magnets substitute for screw-based fastening, avoiding contact-rich fastener insertion and torqueing; the method’s ability to perform tool-enabled operations (e.g., drilling, screwdriving) is not addressed.
  • Physics realism in demonstration generation: Bimanual motion planning in simulation disables physics and treats the object as rigidly attached to both end-effectors, potentially misrepresenting contact, friction, and force constraints; how to generate demonstrations that respect real contact dynamics is open.
  • Contact and force sensing: The policy relies on vision and proprioception; the role of force/torque or tactile sensing for precise, compliant insertions and alignment is unexplored.
  • Progress supervision scalability: Subtask and primitive boundaries (and uniform progress interpolation) are manually defined; automatic discovery of subtasks/primitives and self-supervised progress estimation is not investigated.
  • Progress thresholding and calibration: The transition threshold (τ_p = 0.95) and filtering logic are fixed and task-tuned; sensitivity, calibration, and adaptive thresholding under varying conditions are not studied.
  • Failure detection and recovery: Rollouts terminate on failure; strategies for online error detection, recovery, and backtracking (e.g., progress rollback, re-planning, or self-correction beyond emergent behaviors) are not incorporated or evaluated.
  • Multi-view and 3D perception: While a rear RGB camera helps, the paper does not explore fused multi-view depth, stereo, or explicit scene reconstruction to mitigate occlusions and improve pose/geometry estimation in cluttered assemblies.
  • Marker dependence: Teleoperated demonstrations in the real world use colored markers for human alignment assistance; the impact of these markers on learned policies and generalization without markers is unknown.
  • Control modality and compliance: The policy drives absolute Cartesian poses; effects of impedance/admittance control, compliant insertion strategies, and hybrid force/position control for tight fits are not analyzed.
  • Bimanual coordination constraints: There is no explicit constraint modeling or coupling during policy inference to ensure synchronized, complementary arm motions; principled coordination mechanisms (e.g., shared latent commands or coupling losses) are not explored.
  • Action representation invariances: Actions are absolute in the robot frame; invariance to workspace shifts, base placement changes, or cross-embodiment transfer (to different arms or grippers) is untested.
  • Latency and real-time performance: The compute budget (L40S GPU) and inference latency are not quantified; the impact of control-loop rate, network delays, and ensembling-induced lag on precision and safety remains unclear.
  • Data efficiency and active learning: Simulation requires 500 demos/furniture and real-world IVAR uses 100 demos; how few demonstrations are sufficient, and whether active selection, on-policy data collection, or synthetic augmentation can reduce this is open.
  • Sim-to-real transfer at scale: Only one real furniture item is tested; systematic strategies (domain randomization, feature alignment, photorealistic rendering, tactile simulation) for broader sim-to-real generalization are not evaluated.
  • Robustness to sensing/lighting/camera calibration: Sensitivity to camera extrinsic/intrinsic errors, lighting changes, motion blur, or occluders is not characterized or stress-tested.
  • Automatic camera placement/active perception: Camera viewpoints are fixed; optimizing sensor placement or actively moving cameras to reduce occlusions and improve alignment is not addressed.
  • Language grounding breadth: Subtask prompts are concise and curated; robustness to natural, noisy, or manual-derived instructions (e.g., parsing IKEA manuals) and the ability to follow diverse language styles are untested.
  • Assembly quality beyond pose thresholds: Success is defined by pose error; mechanical fit quality, load-bearing performance, and long-term stability (especially without magnets) are not evaluated.
  • Progress modeling beyond uniform interpolation: Progress is linearly interpolated between primitive boundaries; learning non-linear progress dynamics, uncertainty-aware progress, or multi-signal progress (e.g., combining vision and force cues) is not explored.
  • Design-factor generality: Optimal temporal ensembling λ and action horizons are task-specific; automated tuning or adaptive scheduling across assemblies and environments is absent.
  • Dataset and benchmark scale: Demonstrations rely on handcrafted initial placements and primitive libraries; automated generation of diverse assemblies, placements, and failure cases for broad benchmarking is an open need.
  • Safety and collision handling: Beyond planning feasibility, runtime collision avoidance with partially assembled structures and self-collision handling under perception errors is not formally analyzed or benchmarked.
View Paper Prompt  View All Prompts 

Practical Applications

Below are actionable applications derived from the paper’s findings, methods, and system design. For each item, we indicate sector(s), potential tools/products/workflows, and key assumptions/dependencies that affect feasibility.

Immediate Applications

  • Bimanual cobot “co-assist” for flat-pack assembly
    • Description: Use the progress-enhanced VLA to hold, align, and insert large furniture panels while a human performs fastening, reducing fatigue and improving throughput.
    • Sectors: manufacturing, retail operations, construction fit-out
    • Tools/products/workflows: “Co-Assist Flow” (progress-driven phase handoffs), synchronized bimanual control primitives, rear-camera perception pack
    • Assumptions/dependencies: No screwing by the robot (magnets used in study); requires two 7-DoF arms in a shared workspace, camera placement (front + rear), and tuned temporal ensembling/action horizon
  • Remote robot-as-a-service for on-demand assembly and rework
    • Description: Single-operator VR teleoperation (with predefined grasp presets and mirrored bimanual control) to assemble, rework returns, or refurbish items at micro-fulfillment hubs, with the VLA providing assistive autonomy and progress-based stage indicators.
    • Sectors: robotics services, retail, reverse logistics
    • Tools/products/workflows: Quest2ROS teleop stack, “Operator HMI with progress bar,” fallback-to-teleop workflow for edge cases
    • Assumptions/dependencies: Reliable low-latency connectivity, operator training, safe collaboration spaces, hardware similar to Kinova Gen3 setup
  • Digital-twin feasibility checks and offline programming for assembly cells
    • Description: Use the simulation pipeline to verify reachability, avoid singularities, and generate expert trajectories/demonstrations for new SKUs before deploying to the floor.
    • Sectors: robotics integration, industrial engineering, furniture OEMs
    • Tools/products/workflows: Isaac Gym-based FurnitureVLA Simulator, “Reachability & Singularity Audit,” CAD-to-dataset pipeline
    • Assumptions/dependencies: Accurate CAD models and mass/size estimates; fidelity of simulator-to-real transfer
  • Rapid skill authoring with post-retreat subtask boundaries
    • Description: Apply the paper’s subtask segmentation and continuous progress supervision to collect fewer, higher-quality demos per phase, improving long-horizon robustness and data efficiency.
    • Sectors: robotics software, system integration
    • Tools/products/workflows: “Subtask Studio” (language-conditioned subtask templates), grasp primitive libraries (90° variants), demo harvesting/playbook
    • Assumptions/dependencies: Consistent subtask language prompts; post-retreat boundary design; camera coverage to mitigate occlusion
  • Progress head retrofit for existing VLA/vision-based policies
    • Description: Add a continuous progress head to trigger automatic transitions across phases in packaging, palletizing, or fixture loading tasks.
    • Sectors: robotics software, warehousing
    • Tools/products/workflows: “Progress Head Kit” (PyTorch module + eval scripts), thresholding/filtering utilities for robust transitions
    • Assumptions/dependencies: Labeled or easily inferred action primitives per phase; tuning of progress thresholds (τp) and spike filters
  • Precision-tuning checklist for real-scale manipulation
    • Description: Adopt empirically validated factors—temporal ensembling (λ≈−0.1), action horizon (10–25), higher RGB resolution (≥300×300), and rear camera—to improve tight tolerance insertions.
    • Sectors: robotics integrators, system engineering
    • Tools/products/workflows: “Precision Pack” (configurations, camera placements, QA scripts)
    • Assumptions/dependencies: Slight latency trade-offs from ensembling; camera mounting feasibility; compute headroom for higher resolutions
  • Cobots for alignment/holding in interior fit-outs and lab assembly
    • Description: Robots align/hold panels/frames while humans fasten, leveraging synchronized dual-arm motions and progress-triggered human handoffs.
    • Sectors: construction fit-out, research labs, prototyping shops
    • Tools/products/workflows: Progress-aware HMI prompts, mirrored bimanual control mode
    • Assumptions/dependencies: Human-robot safety policies; ergonomics and fixture design; non-screw steps handled by robot or deferred to human
  • QA and operator assistance via progress-aware HMI
    • Description: Use predicted progress to signal completion, request inspection, or trigger recovery scripts; expose status to MES/HMI dashboards.
    • Sectors: manufacturing IT/OT, quality assurance
    • Tools/products/workflows: “Progress-to-MES Bridge,” pose-error-based assembly checks (relative pose tolerance from the paper)
    • Assumptions/dependencies: Pose estimation or fiducials to compute assembly tolerances; HMI integration
  • Academic benchmarking and teaching modules
    • Description: Use the tasks, subtask structure, and evaluation metrics to teach long-horizon, bimanual control and to benchmark VLA variants.
    • Sectors: academia, education
    • Tools/products/workflows: Course labs with VR teleop, action-primitive progress labeling, sim-to-real assignments
    • Assumptions/dependencies: Access to dual-arm robots (or sim-only alternatives); reproducible environments/assets
  • Bimanual handling for kitting of bulky components (non-assembly)
    • Description: Apply synchronized bimanual control and temporal smoothing to lift, rotate, and place large parts into kits or fixtures with fewer collisions.
    • Sectors: warehousing, manufacturing logistics
    • Tools/products/workflows: “Dual-Arm Kitting Skills,” rear-view occlusion mitigation
    • Assumptions/dependencies: Stable grasping primitives; calibrated workspace and collision models

Long-Term Applications

  • Full end-to-end furniture assembly (including screwing and tool use)
    • Description: Extend beyond magnets to autonomous screwing, cam locks, and fasteners via force/torque sensing, compliance, and tool-use skills learned in VLA frameworks.
    • Sectors: manufacturing, retail assembly services, home robotics
    • Tools/products/workflows: Tool-change workflows, force/impedance controllers, contact-rich insertion libraries
    • Assumptions/dependencies: Tactile/FT sensors, precise force control, robust contact modeling, stronger sim2real for tool dynamics
  • Generalist home assistant for DIY assembly and repairs
    • Description: Mobile, bimanual robot that navigates cluttered homes, perceives novel furniture, and performs long-horizon assembly with progress-driven autonomy.
    • Sectors: consumer robotics, home services
    • Tools/products/workflows: Mobile base + dual arms, home-scale mapping, object discovery + SKU recognition, teleop fallback
    • Assumptions/dependencies: Robust navigation and safety in unstructured spaces, low-cost hardware, privacy and liability frameworks
  • Mobile bimanual platforms for room-scale assemblies
    • Description: Assemble closets, cabinets, or racks beyond a fixed table workspace; coordinate base motion with dual-arm manipulation.
    • Sectors: construction robotics, retail fit-out, data center infrastructure
    • Tools/products/workflows: “Mobile Dual-Arm Planner,” progress-synchronized navigation/manipulation
    • Assumptions/dependencies: Tight base-arm coordination, richer scene understanding, collision-aware mobility under load
  • Cross-industry generalist assembly model
    • Description: A foundation VLA trained across furniture lines, fixtures, and modular products, leveraging progress heads for phase management and few-shot adaptation for new SKUs.
    • Sectors: robotics software, OEMs
    • Tools/products/workflows: “Assembly Skill Packs,” SKU onboarding pipeline with minimal demos, synthetic data augmentation
    • Assumptions/dependencies: Large, diverse datasets; standardization of action spaces; scalable training infra and on-prem inference
  • Autonomous error recovery and failure-aware replanning
    • Description: Combine progress prediction with reasoning/planning modules to detect stalls, backtrack to post-retreat states, and reattempt insertions.
    • Sectors: robotics software, manufacturing
    • Tools/products/workflows: Failure detectors, recovery policy library, state rollback buffers
    • Assumptions/dependencies: Reliable state estimation; consistent post-retreat waypoints; reasoning engines integrated with low-latency control
  • Multi-robot and human-robot team orchestration via shared progress signals
    • Description: Use progress as a synchronization primitive across multiple arms/robots and human operators for complex assemblies.
    • Sectors: manufacturing, construction
    • Tools/products/workflows: “Progress Bus” coordination layer, task graph schedulers
    • Assumptions/dependencies: Inter-robot comms latency bounds; safety-certified coordination policies
  • Surgical and medical bimanual manipulation analogs
    • Description: Adapt progress-guided, phase-based bimanual control to delicate, long-horizon procedures (e.g., suturing, tissue retraction) in simulation-first pipelines.
    • Sectors: healthcare, medtech
    • Tools/products/workflows: High-fidelity simulators, teleop-to-autonomy pipelines, regulatory-grade logging
    • Assumptions/dependencies: Extremely high precision and safety standards; domain-specific sensing/tools; regulatory approvals
  • Standardization, testing, and certification of VLA-enabled assembly
    • Description: Develop benchmarks and acceptance tests (pose-tolerance metrics, progress curves) for certifying long-horizon, bimanual tasks and HRI safety.
    • Sectors: policy/regulation, standards bodies, insurance
    • Tools/products/workflows: “Progress-curve Test Suite,” assembly tolerance scorecards
    • Assumptions/dependencies: Consensus on metrics; reproducible testbeds; traceability/audit of learning systems
  • Self-supervised data scaling with motion-planned seeds and teleop corrections
    • Description: Bootstrap large demonstration corpora by blending planner-generated trajectories with human teleop corrections, then distill into VLAs with progress heads.
    • Sectors: robotics R&D, OEMs
    • Tools/products/workflows: Planner-to-demo generator, human-in-the-loop correction UI, continuous retraining loops
    • Assumptions/dependencies: Planner realism and diversity; efficient labeling/segmentation of primitives; data governance
  • Edge-friendly deployment of progress-enhanced VLAs
    • Description: Compress and quantize models for industrial PCs while retaining precise control and smooth ensembling, enabling wider factory adoption.
    • Sectors: industrial automation, embedded AI
    • Tools/products/workflows: ONNX/TensorRT pipelines, mixed-precision/quantization-aware training, real-time profiling kits
    • Assumptions/dependencies: Latency budgets compatible with ensembling; camera bandwidth; thermal/power constraints on the floor

Notes on key global assumptions and dependencies from the paper:

  • Magnets replace screws in experiments; true fastener handling requires tool-use and force control.
  • Two fixed-base 7-DoF arms with specific grippers; reach and payload limit furniture scale.
  • Vision design matters: rear camera mitigates occlusion; higher resolution improves precision; temporal ensembling trades slight latency for stability.
  • Continuous progress supervision and post-retreat boundaries are central to reducing compounding errors; thresholds need tuning per task.
  • Simulation-to-real depends on CAD accuracy, textures, and realistic motion plans; some sim planning disabled physics for bimanual object-following.
View Paper Prompt  View All Prompts 

Glossary

  • Action chunking: Grouping low-level motor commands into short contiguous sequences to improve stability and reduce error accumulation. "ACT~\cite{aloha23} uses action chunking and temporal ensembling to reduce compounding error and smooth control,"
  • Action horizon: The number of predicted control steps executed before replanning to balance responsiveness and smoothness. "Action horizon: π0.5\pi_{0.5} predicts chunks of H=50H{=}50 actions; we execute only the first 5, 10, or 25 actions before replanning"
  • Action primitives: Discrete, semantically meaningful control segments (e.g., pick, place, retreat) used to structure tasks and define progress. "Each subtask consists of NkN_k action primitives (Fig.~\ref{fig:tasks}), such as pick up, place, and retreat."
  • Action tokenization: Representing continuous actions as discrete tokens for sequence models, which can limit precision in manipulation. "discretized action tokenization can fail on dexterous, high-frequency data"
  • Bimanual: Involving the coordinated use of two robot arms/hands during manipulation. "real-scale bimanual furniture assembly using Vision-Language-Action models (VLAs)."
  • Contact-rich: Involving sustained or complex physical contacts that impose force/constraint sensitivity during manipulation. "Contact-rich post-assembly states are also highly sensitive to execution errors,"
  • Covariate shift: The mismatch between training and test state distributions that degrades policy performance over long horizons. "Long-horizon execution remains brittle due to error accumulation and covariate shift~\cite{RossGB11, sun2024arch, YangZDBS25, ma2025coopera}."
  • Cross-embodiment standardization: Normalizing action/observation formats across different robot bodies, which can reduce control fidelity. "cross-embodiment standardization can coarsen action representations~\cite{ONeillRMGPLPGMJ24}"
  • Distribution drift: Gradual deviation from the training distribution during rollout due to small compounding errors. "Long-horizon trajectories induce distribution drift, where small deviations can drive the policy into out-of-distribution states and lead to compounding errors~\cite{RossGB11, YangZDBS25}."
  • End-effector: The robot’s tool or gripper whose pose is controlled to execute tasks. "For single-arm actions, we plan end-effector pose trajectories directly."
  • Failure detector: A module that identifies execution failures to trigger recovery or replanning. "that replan and recover under a failure detector~\cite{lu2025kitchenvla, ma2026cyclevla}."
  • Flow matching: A generative modeling approach used to decode sequences of future actions in a single pass. "The policy uses flow matching~\cite{pi0_5_pi} to decode a chunk of HH future actions in a single forward pass"
  • Imitation learning: Learning policies by mimicking expert demonstrations at scale. "learning-based methods study large-scale imitation learning~\cite{AnkileSS024}"
  • Isaac Gym: A high-performance physics simulator for large-scale robot learning and evaluation. "We use Isaac Gym~\cite{MakoviychukWGLS21}"
  • Kinematic singularities: Robot configurations where certain motions become unattainable or ill-conditioned due to Jacobian rank deficiency. "avoiding kinematic singularities throughout the assembly process~\cite{FuZF24, li2025robotmover}."
  • Kinematic workspace: The spatial region reachable by a robot given its geometry and joint limits. "within the robot's kinematic workspace;"
  • Motion planning: Computing feasible, collision-free trajectories that satisfy kinematic and task constraints. "We use motion planning to generate expert demonstrations in simulation."
  • Post-retreat states: Contact-free configurations reached after completing a contact-rich action, used as robust subtask boundaries. "post-retreat states are free of contact and force constraints,"
  • Proprioception: Internal robot state measurements (e.g., joint angles, gripper state) used alongside vision. "maps an observation oto_t (RGB images and proprioception) and a language instruction gg to robot actions."
  • Progress-enhanced VLA: A VLA augmented to predict a continuous progress signal alongside actions to manage long-horizon execution. "we propose a progress-enhanced VLA to tackle long-horizon assembly."
  • Reachability: Whether a robot can physically attain a target pose given its kinematics and limits. "respecting reachability limits and avoiding kinematic singularities"
  • Relative pose error: Deviation between the achieved and ground-truth relative transformations of assembled parts. "evaluate assembly by measuring the relative pose error of every other part with respect to this base."
  • Residual policy: A controller that learns corrective adjustments on top of a base policy to improve precision. "residual policies for precision~\cite{AnkileSSTA25}"
  • Rigidly attached end-effectors: A constraint where end-effectors move as if rigidly connected to the manipulated object during planned motions. "constraining both arms to follow as rigidly attached end-effectors."
  • SE(3): The Lie group of 3D rigid body transformations (rotations and translations) used to represent object/robot poses. "Let Tb,Tk∈SE(3)\mathbf{T}_b, \mathbf{T}_k \in SE(3) denote the poses of the base part and the kk-th part, respectively."
  • Sim-to-real transfer: Techniques to deploy policies trained in simulation to real robots with minimal additional data. "sim-to-real transfer with limited real data~\cite{TangLAHS0FN23, ZhangWSWZT23, VTRefine_huang}"
  • Subtask decomposition: Splitting a long task into semantically grounded segments for training and execution. "We define subtask decompositions with corresponding language instructions that are concise, descriptive, and semantically grounded"
  • Subtask progress signal: A continuous scalar that tracks completion within a subtask and gates transitions. "We propose a progress-enhanced VLA that jointly predicts actions and a subtask progress signal"
  • Subtask transition: Switching from one subtask policy/instruction to the next based on progress or detection logic. "automatic subtask transitions"
  • Task-and-motion planning: Integrated high-level task sequencing with low-level motion planning for feasible execution. "task-and-motion planning~\cite{ZhouP18}"
  • Teleoperation: Controlling a robot remotely by a human operator, often in VR, to collect demonstrations or execute tasks. "VR teleoperation system"
  • Temporal ensembling: Averaging overlapping action predictions across time to smooth control and reduce errors. "temporal ensembling~\cite{aloha23}"
  • Vision-Language-Action model (VLA): A generalist policy class that conditions on images and language to output robot actions. "Vision-Language-Action models (VLAs)."
  • Zero-shot: Evaluating a model without task-specific finetuning or additional training. "π0.5\pi_{0.5} (zero-shot) achieves zero success across all furniture,"
View Paper Prompt  View All Prompts 

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 3 tweets with 103 likes about this paper.

Sign Up for Free