2000 character limit reached
Touch begins where vision ends: Generalizable policies for contact-rich manipulation
Published 16 Jun 2025 in cs.RO and cs.CV | (2506.13762v1)
Abstract: Data-driven approaches struggle with precise manipulation; imitation learning requires many hard-to-obtain demonstrations, while reinforcement learning yields brittle, non-generalizable policies. We introduce VisuoTactile Local (ViTaL) policy learning, a framework that solves fine-grained manipulation tasks by decomposing them into two phases: a reaching phase, where a vision-LLM (VLM) enables scene-level reasoning to localize the object of interest, and a local interaction phase, where a reusable, scene-agnostic ViTaL policy performs contact-rich manipulation using egocentric vision and tactile sensing. This approach is motivated by the observation that while scene context varies, the low-level interaction remains consistent across task instances. By training local policies once in a canonical setting, they can generalize via a localize-then-execute strategy. ViTaL achieves around 90% success on contact-rich tasks in unseen environments and is robust to distractors. ViTaL's effectiveness stems from three key insights: (1) foundation models for segmentation enable training robust visual encoders via behavior cloning; (2) these encoders improve the generalizability of policies learned using residual RL; and (3) tactile sensing significantly boosts performance in contact-rich tasks. Ablation studies validate each of these insights, and we demonstrate that ViTaL integrates well with high-level VLMs, enabling robust, reusable low-level skills. Results and videos are available at https://vitalprecise.github.io.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
What is this paper about?
This paper is about teaching robots to do very precise, touch-heavy tasks in the real world—things like plugging in a charger, sliding a credit card, inserting a USB, or putting a key in a lock. The authors introduce a method called ViTaL that helps a robot first find the right place to work using sight, and then complete the tricky, close-up part using both sight and touch. The goal is to be accurate to the level of millimeters while still working in many different rooms and setups.
What questions are the researchers trying to answer?
They’re mainly asking:
- How can a robot do super-precise actions without needing thousands of human demonstrations?
- How can it stay reliable in new places with different lighting, clutter, and backgrounds?
- Does combining cameras (vision) with touch sensors (tactile sensing) make a big difference?
- Can a small amount of practice (around 45 minutes) make the robot noticeably better?
How does their method work?
Think of the robot’s job as happening in two phases: “find, then finesse.”
- Phase 1: Find (global reaching) The robot uses a smart vision-LLM (a tool that understands images and text) to locate the object it needs, like “USB port” or “card slot.” That gets it to the right spot.
- Phase 2: Finesse (local interaction)
- A camera on the robot’s wrist (an egocentric view aligned with the robot’s movements).
- A touch sensor in the gripper (tactile sensing) to feel contact, pressure, and slip.
To train this local strategy, they use two steps:
- Behavior cloning (learning from demonstrations):
- They collect just 32 human demonstrations per task using VR teleoperation.
- They train the robot to copy these actions.
- To make the robot robust across different rooms, they use a green screen and then swap in lots of different backgrounds during training. They use powerful image tools (foundation models) to cut out the robot and the objects cleanly, so only backgrounds change. This teaches the robot to focus on what matters (the object and the robot’s gripper), not the messy room.
- Residual reinforcement learning (fine-tuning with small corrections):
- After copying the demonstrations, the robot practices and learns to make small “fixes” to its actions (like tiny nudges) to be more precise.
- Think of it like having training wheels (the demo policy), and then learning to steer more smoothly with practice.
- They keep the same background-swapping during this practice to preserve generalization.
At run time, the robot:
- Uses the vision-LLM to find the target area in the scene.
- Moves near it.
- Switches to the local touch+vision policy to finish the precise part.
Simple analogies for key terms:
- Behavior cloning: like watching a coach do the task and then copying them.
- Reinforcement learning: like practicing and getting rewards for doing it better each time.
- Residual learning: you keep your original plan, then add tiny corrections to improve it.
- Tactile sensing: giving the robot a sense of touch, so it can “feel” contact and adjust.
- Foundation models for vision: big pre-trained tools that can quickly segment and understand images, helping create realistic training data.
What did they find, and why does it matter?
Here are the standout results and why they’re important:
- High success with very little data: With just 32 demos and about 45 minutes of practice per task, ViTaL succeeds around 90% of the time on hard, contact-rich tasks. That’s unusually data-efficient.
- Generalizes to new places: It works well even in new rooms, with new backgrounds, clutter, or different object positions. This shows the robot isn’t “overfitted” to one lab setup.
- Touch is crucial: Removing the touch sensor drops performance by about 40% on average. When the camera view is blocked (for example, by the object in the gripper), touch helps the robot continue accurately.
- Stronger than common baselines: ViTaL outperforms other methods by about 40% on average across four precise manipulation tasks.
- Works well with a vision-LLM: The “find, then finesse” strategy—using a general model to locate the target, then a trained local skill to finish—proves effective.
These results matter because precise, contact-heavy actions are some of the hardest things for robots to do, especially in changing environments. ViTaL shows you don’t need huge datasets or perfect lab conditions to get robust, accurate performance.
What could this change in the future?
If robots can learn precise skills that transfer to new homes, offices, and factories without massive retraining, they become much more useful:
- Home help: plugging in devices, operating appliances, or handling delicate items.
- Retail and service: using payment terminals or handling cards and keys.
- Light manufacturing: inserting parts or cables accurately, even when the workspace looks different.
The authors also note some current limits:
- They mostly tested on flat tabletops; handling vertical or angled surfaces is future work.
- The “find” step assumes a clear path (no complex obstacle avoidance yet).
- Most tests are in a controlled lab; testing in truly messy, real homes is a next step.
In short, ViTaL shows a practical recipe: let a general vision model find the right spot, then rely on a reusable local touch+vision skill to do the delicate work. This makes precise robot manipulation more reliable, adaptable, and efficient.
Knowledge Gaps
Below is a consolidated list of knowledge gaps, limitations, and open questions that remain unresolved and could guide future research.
- Spatial generalization beyond horizontal tabletops: assess performance on vertical, angled, and curved surfaces, including re-targeting strategies and orientation initialization for non-planar contacts.
- End-to-end autonomy: the method assumes the tool/object is already grasped; extend ViTaL to include reliable grasp planning and object acquisition before local manipulation.
- Obstacle-aware global navigation: VLM-based reaching currently assumes obstacle-free paths; integrate mapping, collision avoidance, and trajectory planning, and quantify end-to-end failure modes due to navigation errors.
- Dependence on external calibrated RGB-D camera: evaluate robustness to calibration drift, moving cameras, occlusions, and elimination of third-person sensors (e.g., wrist-only or proprioceptive-only setups).
- VLM localization fidelity: quantify Molmo’s 2D point accuracy, error distributions, and their impact on downstream success; develop uncertainty-aware strategies and recovery behaviors when coarse localization is wrong.
- Object instance/category generalization: test across diverse connectors/ports (USB variants, different sockets), locks, and card readers with varying tolerances, materials, and geometries; measure failure modes across unseen instances.
- Cross-robot transfer: validate whether end-effector-frame conditioning suffices across arms/grippers with different kinematics, compliance, camera mounting, and morphology; define adaptation procedures and limits.
- Cross-tactile-sensor transfer: study generalization to different tactile skins, layouts, sampling rates, and noise profiles; establish calibration-free or self-calibrating approaches for new sensors.
- Control rate and latency constraints: the policy runs at 6 Hz; quantify the effect of higher-frequency control on precision and slip handling, and conduct sensitivity analyses to communication delays and jitter.
- Force/impedance control: the approach appears position-centric; assess integration of force/torque sensing and impedance control to mitigate damage risk and improve contact stability in tight-tolerance tasks.
- Quantitative precision metrics: replace binary success counts with mm-level alignment, insertion depth, contact forces, cycle time, and repeatability metrics to substantiate claimed precision.
- Reward specification and generality: clarify “dense L1 distance to goal” definitions per task and devise task-agnostic, self-supervised tactile/visual success signals to reduce human labeling and task-specific shaping.
- Residual RL algorithm choice and consistency: the paper references DrQ-v2 and n-step DDPG; systematically compare residual SAC/DrQ-v2/DDPG, ablate regularization (e.g., L2 actor penalty), UTD ratios, and stability.
- Encoder fine-tuning vs freezing: investigate whether partial or full encoder adaptation during residual RL improves performance without sacrificing generalization; study catastrophic forgetting under augmentations.
- Semantic augmentation pipeline scalability: the multi-stage DIFT/SAM2/XMem pipeline needs per-task manual keypoint annotation and green-screen captures; evaluate automation, robustness in natural scenes, and removal of green-screen dependence.
- Online augmentation practicality: clarify how semantic augmentations are applied during on-robot RL (masking, replacement latency, segmentation errors), and quantify overheads and benefits in real time.
- Modality fusion design: beyond separate encoders + transformer, benchmark fusion architectures (early/late fusion, cross-attention, confidence weighting) and adaptive reliance on tactile vs vision under occlusion.
- Robustness under severe occlusion: test tactile-only fallback strategies when the wrist camera view is fully blocked; introduce modality confidence estimation and dynamic policy gating.
- Longer-horizon, multi-contact tasks: extend to sequences requiring multiple precise contacts and tool reorientations; evaluate skill composition, termination conditions, and hierarchical controllers for chaining ViTaL skills.
- Real-world “in-the-wild” validation: run in homes/offices with dynamic lighting, reflective surfaces, moving humans, and unstructured layouts; report distribution shift failures and recovery strategies.
- Failure analysis and taxonomy: systematically categorize errors (mislocalization, misalignment, slip, over-insertion, force overshoot) to inform targeted improvements in sensing, control, and training.
- Data-efficiency boundaries: test whether 32 demos and 45 minutes of interaction suffice across harder tasks; derive scaling laws for contact-rich precision tasks, and explore active demonstration selection.
- Baseline breadth and fairness: include stronger modern vision-RL baselines (e.g., DrQ-v2, SAC with augmentations, AWR, RLPD variants), standardized hyperparameter sweeps, and cross-method augmentations for fair comparison.
- Action space sensitivity: evaluate robustness of end-effector-relative actions to kinematic errors and calibration noise; develop adaptive control that compensates for model inaccuracies.
- Safety constraints and damage avoidance: incorporate safe RL or constraint-based control to cap forces/insertions, detect misalignment early, and prevent damage to ports/locks/card readers.
- Multi-task training and skill reuse: investigate whether a shared visuotactile policy library improves transfer and data efficiency; benchmark zero-/few-shot adaptation to new tasks via residuals.
- Autonomizing success signals: remove human terminal success labels by using tactile/visual signatures for insertion/engagement; study reliability and false positives in cluttered settings.
- Temporal credit assignment: examine alternative shaping signals, hindsight relabeling, or tactile-based dense rewards to improve learning stability in sparse-success tasks.
- Prompting and instruction ambiguity: assess VLM robustness to vague or ambiguous language, multi-object scenes, and distractor semantics; develop prompt engineering or grounding strategies.
- Domain randomization vs semantic augmentation: directly compare classical domain randomization to the proposed semantic pipeline; quantify which variations truly matter for generalization.
- Computational footprint: report and optimize the training/inference cost of segmentation, augmentation, and residual RL on limited robot-side hardware; study real-time constraints and scalability.
- Distractor proximity and false segmentations: analyze failures when distractors resemble target objects or are in close proximity; enhance segmentation reliability and target identity preservation.
- Sim-to-real pathways: explore simulation for large-scale visuotactile data generation, tactile simulation fidelity, and transfer methods that preserve contact-rich precision through reality gaps.
Practical Applications
Immediate Applications
Below are concrete, near-term applications that can be deployed with current hardware and software stacks (e.g., xArm-class cobots, wrist cameras, tactile skins like AnySkin), using the ViTaL pipeline (visuotactile behavior cloning + residual RL + VLM-based reaching + semantic augmentation).
- Plug-and-play connector insertion for electronics assembly
- Sector: Manufacturing (electronics, automotive wire harnesses, appliance assembly)
- What: Automate insertion of board-to-board connectors, USB/Type-C, RJ45, coax, and keyed cable terminals where tolerances are millimeter-level.
- Tools/workflows/products:
- “ViTaL Local Policy Pack” for standard connector families (USB, RJ, JST, Molex) trained with 32 demos + 45 minutes residual RL.
- Wrist-camera and tactile-skin retrofit for existing cobots; ROS2 nodes for visuotactile inference.
- VLM-driven coarse localization to handle fixtured but variable workcells.
- Assumptions/dependencies: Horizontal or gently curved surface placements; obstacle-free reach for the global phase; quality depth/camera calibration; consistent connector geometries and seating depth; safe force-limits.
- Cable plugging/unplugging for lab and data-center operations
- Sector: IT/Ops, Data Centers, Lab Automation
- What: Plugging and removing Ethernet/USB/power cables; test-bay wiring for device provisioning.
- Tools/workflows/products:
- ViTaL skills for standardized cable end types, combined with rack-level VLM localization from fixed RGB-D.
- Test harness integration to validate network or power continuity after insertion.
- Assumptions/dependencies: Repeatable port layout or QR-coded labeling to assist coarse localization; ESD-safe end-effectors; power isolation or compliance logic for live circuits.
- Automated test of card readers and ports (QA in retail/embedded)
- Sector: Retail QA, Embedded device QA
- What: Swipe/insert payment cards to validate terminal acceptance; plug test dongles into device ports for functional checks.
- Tools/workflows/products:
- ViTaL card-swipe/insert skills and port-insertion skills; batch-scripted test orchestration.
- Semantic augmentation to handle terminal bezels, lighting variations, or countertop clutter.
- Assumptions/dependencies: Standard terminal geometry; safe, non-public environments; card/port wear-level considerations.
- Mobile robot self-charging via outlet or docking plug
- Sector: Facilities/Service Robotics
- What: Robots that plug into wall outlets or charge points in brownfield facilities without precision docking stations.
- Tools/workflows/products:
- ViTaL outlet-plug insertion skill; VLM localization from an external RGB-D camera; safety interlocks for electrified hardware.
- Assumptions/dependencies: Electrically safe connectors; grounded, compliant gripper; conservative force control and stop thresholds; clear access to outlets.
- Assistive tasks in controlled care or lab settings
- Sector: Assistive robotics (rehabilitation centers, labs)
- What: Plugging phone chargers, inserting keys into locks, placing pods into machines—tasks requiring tactile feedback.
- Tools/workflows/products:
- Low-cost arm with wrist camera + tactile skin; ViTaL skills fine-tuned with user-specific fixtures; teleop fallback via VR for edge cases.
- Assumptions/dependencies: Supervised/controlled environments; fixtures to standardize target pose; trained operators for setup and oversight.
- Brownfield integration for cobots with minimal re-calibration
- Sector: Systems integration, Manufacturing SMEs
- What: Deploy skills that generalize across benches/backgrounds using egocentric vision + semantic augmentation rather than relying on external camera calibration or environment-specific retraining.
- Tools/workflows/products:
- ViTaL-based drop-in skills; segmentation/augmentation pipeline (DIFT + SAM2 + XMem + RoboEngine) as a turnkey training add-on.
- Assumptions/dependencies: Short, one-time annotation for target instances; green-screen or adequate segmentation achievable; stable wrist-camera mount.
- Academic courseware and benchmarking for contact-rich robotics
- Sector: Academia, R&D labs
- What: Reproducible visuotactile manipulation curricula and benchmarks (USB insertion, key-in-lock, card swipe) with open models and datasets.
- Tools/workflows/products:
- Open-source ViTaL training code; student labs on semantic augmentation and residual RL; tactile-vision ablation studies.
- Assumptions/dependencies: Access to a mid-range robot arm and tactile skin; VR teleop or kinesthetic teaching for 32 demos per task.
- Rapid skill authoring for integrators using VR teleop + augmentation
- Sector: Robotics services, SI/consulting
- What: Collect ~1 minute of high-quality demos and auto-augment scenes to produce robust, transferable skills with minimal data collection.
- Tools/workflows/products:
- “Skill authoring” workflow: VR teleop → semantic augmentation → BC → residual RL → packaged skill; internal skill repository with tags by geometry and contact profile.
- Assumptions/dependencies: VR teleop setup; access to foundation model pipelines for augmentation; stable training compute.
- Tactile-sensor value demonstration and retrofits
- Sector: Tactile hardware vendors, Cobot OEMs
- What: Showcase immediate, measurable uplift (≈40% success increase in ablations) from adding tactile skins to grippers for contact-rich tasks.
- Tools/workflows/products:
- Vendor-provided ViTaL-compatible tactile encoders and ROS drivers; demo kits for precise insertion tasks.
- Assumptions/dependencies: Robust tactile skin attachment; stable sampling at ~6–30 Hz; minimal latency fusion with vision.
- Robust local skills for VLA stacks
- Sector: Software/AI for robotics
- What: Combine high-level VLM/VLA planners for “what/where” with ViTaL local skills for “how,” improving success on precise subtasks within broader missions.
- Tools/workflows/products:
- Skill APIs with offset-based RL refinement; plug-ins for existing VLA agents; composition graphs that call ViTaL skills after coarse waypointing.
- Assumptions/dependencies: Reasonable VLM localization accuracy; clear path to the local skill’s operational envelope; task-space control available.
Long-Term Applications
These applications require additional research and engineering (e.g., obstacle-aware global planning, vertical/angled surface generalization, higher-rate tactile sensing, regulatory approvals).
- Fine medical procedures with tactile guidance
- Sector: Healthcare
- What: Catheter/IV insertion, cannulation, delicate probe manipulation, assistive surgical subtasks where tactile cues dominate near-contact phases.
- Tools/workflows/products:
- Medical-grade tactile skins; high-frequency control; compliance controllers; FDA-approved workflows integrating residual RL safely.
- Assumptions/dependencies: Rigorous safety/regulatory approval; sterile design; sub-millimeter sensing; fault detection and human-in-the-loop oversight.
- Electrical panel wiring and switchgear operation
- Sector: Energy, Utilities, Industrial maintenance
- What: Tactile, precise insertion of wires into terminals, flipping and latching mechanisms, PV connector handling in field conditions.
- Tools/workflows/products:
- Ruggedized visuotactile modules; obstacle-aware VLM-based global planners; weatherproof sensors; skill library for connector variants.
- Assumptions/dependencies: Vertical/angled surface robustness; compliant manipulation under safety lockout/tagout; environment mapping; shock protection.
- General-purpose home robots for assistive living
- Sector: Consumer robotics, Elder-care
- What: Reliable “local skills” for key insertion, charger plugging, appliance-actuation (e.g., loading pods, inserting/removing accessories).
- Tools/workflows/products:
- Household skill marketplace trained via minimal demos; multi-room VLM navigation fused with SLAM; safe physical HRI.
- Assumptions/dependencies: Affordable hardware platforms with wrist cameras + tactile skins; robust global navigation with obstacle/crowd awareness; fail-safe contact control.
- Delicate mechanical assembly (watchmaking, miniature gearboxes)
- Sector: High-end manufacturing
- What: Insert tiny pins, align gears, snap-fit micro-parts requiring combined vision-tactile precision and generalization to part/batch variation.
- Tools/workflows/products:
- High-resolution tactile arrays; micro-grippers; adaptive residual RL with tight force thresholds; environmental isolation for vibrations.
- Assumptions/dependencies: Upgraded sensing/actuation precision; cleanroom compatibility; cycle-time optimization.
- On-orbit servicing and in-space assembly
- Sector: Aerospace
- What: Mating electrical/data connectors, fastening latches, handling microgravity-specific contact dynamics with minimal visibility.
- Tools/workflows/products:
- Radiation-hardened visuotactile hardware; force/torque integration; long-horizon planners orchestrating ViTaL local skills.
- Assumptions/dependencies: Space-qualified hardware; microgravity dynamics modeling; zero-error tolerance safety layers.
- Underwater maintenance and research tooling
- Sector: Marine robotics
- What: Connect wet-mate connectors, insert probes into ports, manipulate valves where visibility is poor and tactile feedback is crucial.
- Tools/workflows/products:
- Waterproof tactile skins; sonar/fused perception for coarse localization; pressure-compensated end-effectors.
- Assumptions/dependencies: Environmental sealing; domain-randomized training for turbidity and currents; safe contact-force limits.
- Agricultural delicate handling and grafting
- Sector: AgTech
- What: Insert sensors into stems/soil, perform grafting/tying actions requiring tactile finesse and visual occlusion robustness.
- Tools/workflows/products:
- Task libraries with plant-specific local policies; seasonal adaptation via residual RL; mobile platforms with VLM navigation in fields.
- Assumptions/dependencies: Outdoor robustness; compliance in deformable materials; safe tool-switching.
- Obstacle-aware VLM navigation and 3D surface generalization
- Sector: Robotics software
- What: Extend global “localize-then-execute” to cluttered scenes and non-horizontal targets (vertical panels, angled sockets).
- Tools/workflows/products:
- VLMs augmented with 3D mapping and collision-aware trajectory planning; training on vertical/angled datasets; policy-conditioned waypoints.
- Assumptions/dependencies: Better 3D reasoning and motion planning; richer demonstrations for non-planar interactions.
- Self-serve cloud platform for skill creation and deployment
- Sector: Software/Platforms
- What: Cloud service that takes short teleop demos, runs semantic augmentation + BC + residual RL, and deploys a packaged visuotactile skill to fleets.
- Tools/workflows/products:
- Web UI for annotation/keypoint seeding; automated segmentation and augmentation; model versioning; fleet rollout with monitoring.
- Assumptions/dependencies: Reliable upload of visuotactile logs; device fleet management; on-premise options for IP-sensitive factories.
- Standards and certification for contact-rich robot tasks
- Sector: Policy/Regulation
- What: Safety guidelines for tactile-equipped cobots performing electrified or public-facing contact tasks; data standards for visuotactile logging.
- Tools/workflows/products:
- Test suites (connector insertion repeatability, force thresholds, fail-safe behaviors); audit trails for tactile/vision data during incidents.
- Assumptions/dependencies: Multi-stakeholder consensus (manufacturers, regulators, insurers); harmonization with existing ISO/ANSI robot safety standards.
Notes on feasibility and dependencies across applications:
- Core assumptions from the paper:
- Access to a wrist-mounted egocentric camera and tactile sensing; 6 Hz control loop in the reported setup.
- Global (VLM) reaching assumes obstacle-free path; spatial generalization primarily validated on horizontal surfaces.
- Semantic augmentation pipeline may need green-screen or high-quality segmentation; minimal manual keypoint annotation.
- Task-specific training still required (≈32 demos + 45 minutes of residual RL per task); simple reward shaping or success signal collection.
- Risk factors:
- Regulatory and safety constraints (especially for electrified/medical tasks).
- Hardware reliability of tactile skins in harsh environments.
- VLM localization accuracy in visually challenging or cluttered scenes without obstacle-aware planning.
Glossary
- Ablation study: A research method for testing hypotheses by altering one variable to determine its effect on a particular outcome. Example: "Ablation studies validate each of these insights, and we demonstrate that ViTaL integrates well with high-level VLMs."
- Behavior cloning: A technique in machine learning where a policy model learns to replicate a set of expert demonstrations. Example: "foundation models for segmentation enable training robust visual encoders via behavior cloning."
- DrQ-v2: Data-Reinforcement Learning method for visual continuous control tasks that uses Q-learning with augmented environment data for improved policy learning. Example: "Rather than learning policies from scratch, we apply DrQ-v2 to refine behavior-cloned policies by predicting small corrective actions."
- Egocentric vision: A method of visual sensing that captures images from the perspective of the robot's end effector or body, offering a consistent viewpoint aligned with the robot's movements. Example: "a reusable, scene-agnostic policy performs fine-grained, contact-rich manipulation using egocentric vision."
- Fisheye camera: A type of lens that captures wide panoramic or hemispherical images, often used in robotic perception. Example: "observations for policy learning include RGB images captured by a fisheye camera mounted on the robotâs wrist."
- Foundation models: Large-scale deep learning models pre-trained on vast datasets that provide robust baselines for downstream tasks in specific domains. Example: "foundation models for segmentation enable training robust visual encoders via behavior cloning."
- Residual reinforcement learning (RL): A reinforcement learning technique where a residual policy refines the behavior of a base policy by learning corrections or fine-tuning actions. Example: "these encoders improve the generalizability of policies learned using residual RL."
- Scene-agnostic: A quality of a method or model that allows it to be effective in a variety of environments without relying on scene-specific features or configurations. Example: "a reusable, scene-agnostic policy performs fine-grained, contact-rich manipulation."
- Semantic augmentation: Data augmentation technique that employs semantic understanding (such as object segmentation) to create modified versions of data while preserving essential content. Example: "task success depends primarily on the visual features of task-relevant objects...we introduce a semantic, task-aware data augmentation pipeline."
- ViTaL: VisuoTactile Local policy learning framework designed for learning generalizable and precise manipulation in robots using a combination of vision and tactile feedback. Example: "We introduce VisuoTactile Local (ViTaL) policy learning, a framework that solves fine-grained manipulation tasks."
- Vision foundation model (VLM): A model incorporating vision and language to enhance reasoning about scenes and objects, aiding in complex tasks in robotics and AI. Example: "a vision-LLM (VLM) enables scene-level reasoning to localize the object of interest."
Collections
Sign up for free to add this paper to one or more collections.