Papers
Topics
Authors
Recent
Search
2000 character limit reached

Hallucination in World Models is Predictable and Preventable

Published 25 Jun 2026 in cs.LG, cs.CV, and cs.RO | (2606.27326v1)

Abstract: Modern generative world models render increasingly realistic action-controllable futures, yet they frequently hallucinate: rollouts remain visually fluent while drifting from the ground-truth dynamics. We hypothesize that hallucination concentrates in low-coverage regions of the state-action space, where lightweight data-centric signals can both detect it and guide mitigation. To test this, we introduce MMBench2, a 427-hour, 210-task dataset for visual world modeling with ground-truth actions, rewards, and live simulators, and train a 350M-parameter world model on it. We identify three distinct hallucination modes: perceptual, action-marginalized, and scene-diverging -- each anchored to a different stage of the pipeline, and develop three signals that accurately predict where the model will fail. To close coverage gaps at training time, we develop a coverage-aware sampling technique; to close them online, our hallucination predictors serve as curiosity rewards for targeted data collection, yielding a data-efficient finetuning recipe that adapts the pretrained world model to entirely unseen environments with as few as 50 real environment trajectories. Overall, our findings reveal that hallucination in world models is inherently a data coverage issue, and that the same signals used to detect it can also be used for mitigation. An interactive web version of our paper is available at https://www.nicklashansen.com/mmbench2

Authors (2)

Summary

  • The paper introduces a taxonomy of hallucination by systematically identifying predictive signals from internal model metrics such as tokenizer round-trip residual, flow instability, and inter-seed denoising variance.
  • It demonstrates that coverage gaps in the diverse MMBench2 dataset are strongly correlated with rollout errors, highlighting the role of data distribution in model failures.
  • The research validates that both coverage-aware sampling and curiosity-driven data collection efficiently mitigate hallucination, thus enhancing robust model-based control.

Predictable and Preventable Hallucination in World Models: An In-Depth Analysis

Introduction

The phenomenon of hallucination in generative world models—where rollouts appear visually plausible but deviate from ground-truth environment dynamics—presents a significant obstacle to robust model-based control. This work systematically investigates the underlying causes, identifiable predictive signals, and data-centric mitigations of hallucination in world models, advancing the discussion beyond model-induced limitations toward an evidence-based, coverage-driven perspective. The authors introduce MMBench2, an extensive multitask dataset with dense action and reward annotation, and empirically validate the role of data coverage in hallucination, proposing practical collection and sampling remedies.

Taxonomy and Characterization of Hallucination

The analysis yields a stage-wise taxonomy of hallucination, each mapped to distinct points in the generative pipeline:

  1. Perceptual Hallucination: Emanates from the tokenizer (frozen encoder-decoder); out-of-distribution (OOD) states are projected onto near neighbors in the tokenizer's latent space, often causing structural misalignment in reconstructed frames. This effect is isolated to input stage corruption and is invariant to rollout horizon.
  2. Action-Marginalized Hallucination: Dependent on the action-conditioning dynamics model, these hallucinations manifest as insensitivity to the provided actions; resulting rollouts are visually coherent yet not controllable, failing to respect the causal effect of interventions.
  3. Scene-Diverging Hallucination: Occurs during multi-step rollouts when compounding errors in low-coverage state regions produce physically implausible predictions, such as objects reappearing or state resets inconsistent with the environment’s rules. Figure 1

Figure 1

Figure 1

Figure 1

Figure 1

Figure 1

Figure 1: Hallucination taxonomy, visualizing perceptual, action-marginalized, and scene-diverging failure modes.

This taxonomy provides diagnostic clarity; interventions must be targeted at data and mechanisms at the respective pipeline stages.

MMBench2: A Diverse Dataset for Visual World Modeling

The MMBench2 dataset is engineered to facilitate the identification and mitigation of hallucination sources. It comprises 65,600 trajectories (~427 hours, 23 million frames at 224×224224\times 224, 15 fps) across 210 continuous-control tasks spanning ten domains, including manipulation, navigation, classic games, and open-loop control, with dense action and reward annotations and access to live simulators. Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2

Figure 2: Representative sample of 36 out of 210 tasks in MMBench2, demonstrating extensive morphological and perceptual diversity.

Distributional skew is highlighted in Figure 3, where heavy tails (20 tasks comprising 26% of all frames) expose the risk of persistent coverage gaps in lower-task-count regions. Figure 3

Figure 3: Dataset composition reveals the highly non-uniform distribution of state coverage by task.

Predictors of Hallucination: Mechanistic and Statistical Grounding

Three mechanistically distinct hallucination predictors are derived, each requiring only internal model signals—no additional labels or networks:

  • Tokenizer Round-trip Residual (uru_r): Quantifies divergence upon decoding and re-encoding predicted latents, flagging failures in perceptual fidelity (i.e., drift from the tokenizer’s manifold).
  • Flow Instability (ufu_f): Measures variance between denoising predictions during Euler integration substeps; action-marginalized or weakly-conditioned predictions correspond to high ufu_f.
  • Inter-seed Denoising Variance (usu_s): Computes epistemic uncertainty via latent prediction variability across random noise seeds, pinpointing regions likely to exhibit scene divergence.

A critical empirical finding is that all three predictors are tightly correlated with realized rollout error (Spearman ρ≈0.80\rho \approx 0.80), validating their operational use as runtime hallucination detectors. Figure 4

Figure 4: All three predictors robustly track realized rollout error, supporting their use as label-free detectors.

Coverage-Aware Training and Targeted Data Collection

Two principal mitigation strategies are explored:

  • Coverage-aware Sampling: Adjusts pretraining data distribution to uniformly sample across tasks (rather than raw frames), thus addressing inter-task coverage bias. Empirically, coverage-aware sampling improves both low-level reconstruction (PSNR), action sensitivity, and rollout fidelity, confirming the centrality of distributional support over model capacity.
  • Targeted Online Collection via Curiosity: The proposed predictors serve as curiosity signals to guide data collection toward historically low-coverage, high-hallucination regions, substantially improving performance with minimal new data. This mechanism is effective on both seen and entirely novel tasks, as evidenced by adaptation to unseen domains with as few as 50 real trajectories. Figure 5

Figure 5

Figure 5

Figure 5

Figure 5

Figure 5

Figure 5

Figure 5

Figure 5

Figure 5

Figure 5

Figure 5

Figure 5: Hallucinations (high uru_r) cluster in low-density state regions, mapping directly onto coverage gaps.

Figure 6

Figure 6

Figure 6

Figure 6

Figure 6

Figure 6

Figure 6: State density comparison under expert, curiosity-driven, and human data collection across tasks, highlighting the ability of hallucination-predictor-driven data collection to target coverage gaps.

Ablations show that, for zero-shot generalization, even strong pretrained models exhibit notable hallucination in OOD regimes; however, minimal targeted collection using model-internal uncertainty signals rapidly narrows the transfer gap, approaching human- or expert-collected data efficacy.

Comparative Analysis and Practical Implications

A controlled evaluation demonstrates that off-the-shelf tokenizers, pretrained on massive corpora, achieve superior perceptual generalization in the truly OOD regime, yet task-specific fine-tuning allows in-domain tokenizers to recover and surpass baselines. Thus, in-domain adaptation remains relevant until sufficient scale and heterogeneity are achieved in generic, foundational tokenizers.

Theoretical implications are clear: hallucination is fundamentally a coverage problem, not a capacity or architectural limitation, at the examined scales. These findings dovetail with ongoing work on uncertainty estimation, offline RL coverage theorems, and exploration in RL, emphasizing the importance of data curation, coverage-driven sampling, and active, curiosity-based acquisition policies.

Broader Impacts and Future Directions

Practically, these results motivate the adoption of coverage diagnostics as first-order components in world model pipelines, informing both training and deployment. The open-sourcing of MMBench2, the pretrained model, and interactive data collection interfaces provides an immediate platform for further research, including systematic scaling studies, exploration of real-world robotic perception, and probing extensibility to partially observable or highly stochastic domains.

Potential extensions include the integration of advanced OOD detectors, structured uncertainty quantification for risk-aware planning, and the development of more annotation-efficient collection protocols.

Conclusion

This work establishes that hallucination in modern world models is both predictable and preventable. Coverage-oriented metrics—extractable directly from model internals—enable both robust detection and efficient online mitigation. The MMBench2 dataset, alongside coverage-aware and curiosity-driven collection protocols, provides the empirical foundation to close coverage gaps and reliably adapt large-scale world models to previously unseen environments with minimal additional data.

The methodology reframes hallucination from an intractable nuisance into an actionable failure mode whose incidence—and solution—is fundamentally data-centric. The practical and theoretical trajectories outlined here pave the way for more reliable, adaptive, and scalable generative world models for downstream control and interactive tasks.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Explain it Like I'm 14

Big picture: what this paper is about

This paper studies “world models,” which are AI systems that try to imagine what will happen next in a video-like world when you take certain actions (like pressing a game controller’s buttons). These models can make very realistic-looking futures, but sometimes they “hallucinate”: the video looks fine, yet it doesn’t follow the real rules of the world. The authors show that these hallucinations aren’t random—they usually happen in parts of the world the model hasn’t seen much—and they explain simple ways to predict and reduce them.

The main questions

  • When do world models hallucinate, and why?
  • Can we spot (predict) hallucinations using signals available inside the model while it runs?
  • Can we reduce hallucinations by changing what data we train on or by collecting better data?

How they studied it

A big playground to test ideas (the dataset)

They built a huge, varied dataset called MMBench2:

  • 427 hours of video (23 million frames), 210 different tasks (like games, robot sims, physics worlds), and “live” simulators you can actually play.
  • Each video frame comes with the action taken and the reward earned, so the model can learn how actions change the world.

This gave them full control: they know exactly what the model saw during training and can test new situations to see where it fails.

The world model they trained

Think of the model as a two-part system:

  1. A tokenizer (compressor + reconstructor):
  • It takes an image frame and compresses it into a short code (like summarizing a scene).
  • Then a decoder turns that code back into an image.
  • If the decoder can’t rebuild the original well, it means the model struggles to represent that scene.
  1. A dynamics model (the “what happens next” brain):
  • It looks at the current compressed code and your action (e.g., “move left”), and predicts the next code.
  • Chaining this step predicts a whole future video.
  • If the predictions drift from reality, you get hallucinations.

They trained a large (about 350 million parameters) model on MMBench2, then ran many tests.

Three kinds of hallucinations

  • Perceptual: The tokenizer misrepresents the scene from the start. It reconstructs an unfamiliar situation as something it has seen before (like confusing a new maze layout with a known one).
  • Action-marginalized (ignoring actions): The model’s next-frame prediction looks fine but barely changes when you change the action—like a video player, not an interactive world.
  • Scene-diverging: As it predicts multiple steps into the future, things become physically impossible (for example, a ball “teleports” back into play). Errors snowball over time.

How to predict hallucinations in real time

The authors use three simple signals (no extra training needed), each like a “warning light”:

  • Tokenizer round-trip residual: Compress a predicted frame and immediately reconstruct it. If the result changes a lot, the model likely doesn’t truly “understand” that scene—watch out for perceptual mistakes.
  • Flow instability: When the model refines its next-frame guess over a few tiny steps, a stable guess settles quickly. If it keeps wobbling, the model is uncertain—hallucination risk is higher.
  • Inter-seed variance: Run the next-frame prediction multiple times with different tiny noises. If the answers disagree a lot, the model is unsure—future rollouts can split apart and go weird.

In everyday terms: if your calculator gives very different answers to the same problem after tiny nudges, don’t trust the result.

How to reduce hallucinations

Two data-centric strategies:

  • Coverage-aware training: Rebalance which training clips you sample so the model doesn’t over-practice a few popular tasks and under-practice rare situations. This “fills in the map” of what the model has seen.
  • Curiosity-driven data collection: Use the warning signals above as a “curiosity score.” When exploring a new environment, ask the model where it is most unsure, then go collect real data there. This efficiently patches the gaps.

What they found (and why it matters)

  • Hallucinations cluster in low-coverage regions—the “edges” of what the model has seen. This supports the idea that hallucination is mostly a data coverage problem, not just an architecture problem.
  • The three warning signals strongly track real errors during rollouts. In short, you can predict where the model will fail while it’s running.
  • Coverage-aware training improves all three hallucination types at once, without extra cost.
  • Curiosity-driven data collection is very efficient. Using just about 50 real trajectories in brand-new environments, the model adapts much better—approaching the performance of expert or human-collected data.
  • Off-the-shelf visual compressors (tokenizers) trained on huge internet-scale data can help on unfamiliar tasks, but fine-tuning on in-domain data still wins in the end.

Why this is important: If planners and robots trust hallucinated futures, they make bad decisions. Being able to predict and prevent these mistakes makes world models more reliable for control, planning, and robotics.

What this could change going forward

  • Build safer, more reliable planning systems: If we know when the model is likely wrong, we can avoid risky actions or gather the right data first.
  • Make training more efficient: Instead of endlessly scaling models and datasets, we learn where the coverage gaps are and target them.
  • Faster adaptation to new places: Robots and agents can enter a new environment, quickly identify what they don’t know, and learn with minimal real-world data.

In short, the paper shows that world-model hallucinations are not mysterious—they’re predictable, and with smart data choices, preventable.

View Paper Prompt  View All Prompts 

Knowledge Gaps

Knowledge gaps, limitations, and open questions

Below is a focused list of concrete gaps and open questions that remain unresolved:

  • External validity beyond simulators: Does the “coverage causes hallucination” claim, the three failure modes, and the proposed predictors/mitigations hold on real robots with sensor noise, latency, occlusions, non-stationarity, and partial observability?
  • Scaling behavior: How do hallucination rates, predictor calibration, and mitigation effectiveness change with model scale (e.g., >1B parameters) and data scale (orders-of-magnitude larger or more diverse datasets)?
  • Architecture generality: Do the three predictors (tokenizer round-trip residual, flow instability, inter-seed variance) and failure taxonomy transfer to other world-model classes (diffusion, autoregressive video models, latent variable models), alternative dynamics backbones, and different tokenizers (VQ-VAEs, quantized latents)?
  • Predictor robustness and calibration:
    • Sensitivity to training hyperparameters (noise schedule, number of Euler substeps, denoiser architecture) and to inference settings (number of seeds for usu_s, temperature, step size).
    • Stability over finetuning (calibration drift), across tasks/domains, and under distribution shift.
    • Principled thresholding for runtime decisions (e.g., stopping rollouts or switching strategies) with calibrated false positive/negative rates.
  • Computational overhead: Can the predictors be computed in real-time for control (especially usu_s requiring multiple seeds and uru_r requiring decode–encode round-trips)? Are there low-cost surrogates or approximations that preserve detection power?
  • Normalization by scene motion: The current per-task latent-motion normalization may fail in non-stationary or multi-phase tasks. Are adaptive, per-state or learned normalizers more reliable across varying dynamics?
  • Action-marginalization detection: The one-step “action shuffle ratio” may miss multi-step or context-dependent insensitivity. Can multi-horizon causal/gradient-based measures, counterfactual action interventions, or Fisher information diagnostics yield more reliable detection?
  • Failure taxonomy completeness: Are there additional hallucination modes (e.g., reward hallucination, object permanence/occlusion failures, contact dynamics violations, long-horizon semantic drift, partial observability-induced ambiguities) that need explicit detection/mitigation?
  • Metrics adequacy: PSNR/LPIPS and a “repeat-last-frame” baseline are crude proxies for rollouts. Which task-aware, physics-consistency, and safety-critical metrics better capture control-relevant errors (e.g., contact violations, conservation laws, collision checks, goal attainment)?
  • Coverage-aware sampling granularity: Uniform sampling across tasks does not directly target low-coverage states/actions within tasks. Can density-estimation or uncertainty-weighted state–action reweighting (e.g., prioritized rare transitions) improve coverage without overfitting or catastrophic forgetting?
  • Targeted data collection limitations:
    • Reliance on live simulators; applicability to offline logs and real-world data constraints.
    • Bias-amplification risk when using a flawed model to guide collection; how to enforce diversity and avoid feedback loops?
    • Safety constraints for curiosity-driven exploration on physical systems.
  • Curiosity objective design: Only urnormu_r^{\text{norm}} was used for exploration; are ufnormu_f^{\text{norm}} or usnormu_s^{\text{norm}} better for certain domains? What is the best way to combine predictors (e.g., learned mixtures) and allocate budgets adaptively across tasks?
  • When curiosity underperforms: Which task properties (stochasticity, sparse rewards, deceptive dynamics) cause curiosity-based collection to fail or plateau? What stopping criteria or curriculum strategies should be used?
  • Off-the-shelf tokenizer integration: The comparison is limited to reconstruction; does end-to-end training of dynamics on off-the-shelf latents reduce hallucination and improve control? How do latent dimensionality and code semantics affect dynamics learnability and predictor efficacy?
  • Temporal tokenization: The tokenizer is frame-wise MAE-style; do video tokenizers with temporal latents reduce perceptual hallucinations and improve long-horizon consistency?
  • Planning-time risk awareness: Can predictors be integrated directly into MPC (e.g., risk-aware cost terms, uncertainty-constrained horizon selection, fallback to shorter horizons or real rollouts) to improve closed-loop safety and performance?
  • Reward modeling and hallucination: How often are rewards hallucinated, how does reward error correlate with the visual/dynamics predictors, and what is the impact on planning and evaluation?
  • Multimodal inputs: How do additional modalities (proprioception, depth, tactile) affect hallucination incidence and predictor reliability? Are predictors easily extendable to multimodal latents?
  • Long-horizon behavior: The evaluation uses horizon 32; how do hallucinations and predictor reliability evolve for much longer horizons and in hierarchical planning?
  • OOD shift types: How robust are predictors and mitigations to systematic shifts (lighting, camera pose, textures), dynamics parameter changes, or morphology changes? Can the framework detect and adapt to such shifts online?
  • Dataset composition and quality: The corpus is heavy-tailed across tasks and contains mixed-quality trajectories. How should sampling/weighting account for variable data quality and noisy labels, and how does this interact with coverage and hallucination?
  • Catastrophic forgetting and regressions: Does reweighting or targeted finetuning degrade performance on well-covered regions or other tasks? Are there principled mechanisms (e.g., replay, regularization) to prevent regressions?
  • Theoretical grounding: Can we formalize the link between state–action coverage and hallucination (e.g., bounds relating density to rollout error accumulation or model uncertainty), and justify when the predictors are sufficient statistics for hallucination risk?
  • Granular failure analysis and benchmarks: A standardized benchmark and labeled corpus for each hallucination type (with triggers and contextual factors) would enable controlled comparisons of detectors, mitigations, and planners.
  • Transfer across action spaces: How well do methods adapt to tasks with different action dimensionalities/types (e.g., discrete, hybrid), constraints, or different control frequencies and delays?
  • Real-world deployment practices: What engineering practices (e.g., safety envelopes, monitoring, human-in-the-loop overrides) are needed when deploying predictor-driven exploration and planning on physical systems?
  • Reproducibility and accessibility: The approach requires substantial compute (8×H100, 58 GPU-days). What reduced-cost recipes, smaller models, or distilled predictors can make the methods broadly reproducible?
  • Ablations vs. standard uncertainty methods: Direct comparisons to ensembles, MC-dropout, bootstrapped models, or disagreement-based exploration are missing; do the proposed predictors add complementary signal or supersede prior methods?
View Paper Prompt  View All Prompts 

Practical Applications

Immediate Applications

Below are actionable use cases that can be deployed with modest engineering effort using the paper’s dataset (MMBench2), signals, and training/finetuning recipes.

  • Runtime hallucination monitoring for world-model-based control (sectors: robotics, autonomous vehicles, manufacturing)
    • What: Add lightweight online monitors that flag when a world model’s rollouts are likely to drift from reality.
    • How: Compute the paper’s motion-normalized predictors inline during inference: tokenizer round-trip residual urnormu_r^{norm}, flow instability ufnormu_f^{norm}, and inter-seed variance usnormu_s^{norm}.
    • Tools/workflow: Integrate a “WM-Health” dashboard that logs these signals per step and triggers alerts; expose thresholds for safety policies.
    • Assumptions/dependencies: Access to model internals (tokenizer/dynamics), tolerance for small compute overhead (extra denoising seeds, encode–decode round-trip), and a world-model-based control stack.
  • Safe fallback and replanning policies (sectors: robotics, autonomous driving, logistics)
    • What: Automatically shorten MPC horizon, increase replanning frequency, or switch to a conservative controller when hallucination risk rises.
    • How: Set thresholds on ufnormu_f^{norm} (low conditioning confidence) and usnormu_s^{norm} (high epistemic uncertainty) to gate planning horizon H and CEM parameters.
    • Tools/workflow: Add a policy-gating layer that modifies planner hyperparameters or escalates control to a safety controller/human-in-the-loop.
    • Assumptions/dependencies: Planner supports runtime parameter changes; validated thresholds per task/environment.
  • Coverage-aware data loaders for world-model training (sectors: software/ML infrastructure, gaming, robotics)
    • What: Reduce hallucination by rebalancing data sampling without collecting more data.
    • How: Adopt uniform-across-task (rather than frame) sampling and optional per-region reweighting using the provided signals.
    • Tools/workflow: “CovSampler” plug-in for PyTorch/JAX data pipelines; run A/B pretraining/fine-tuning with and without coverage-aware sampling.
    • Assumptions/dependencies: Access to dataset-level task labels or density estimates; sufficient batch diversity.
  • Targeted data collection for fast adaptation to new environments (sectors: robotics, simulation platforms, AR/VR)
    • What: Collect the most informative trajectories to close coverage gaps with minimal budget.
    • How: Use urnormu_r^{norm} (or usnormu_s^{norm}) as a curiosity reward to select trajectories in closed-loop (plan in world model; execute top-scoring in the real/sim environment).
    • Tools/workflow: “CurioCollect” module that rolls out candidate plans for H steps in the model, scores them, and executes the best; fine-tune for 30–50k steps.
    • Assumptions/dependencies: Live environment access; planning loop (e.g., CEM) in place; small extra compute for scoring candidates.
  • Action-sensitivity audits to prevent “action-marginalized” models (sectors: QA/compliance for control software, robotics)
    • What: Ensure the world model truly conditions on actions (i.e., does not ignore inputs).
    • How: Use the action-shuffle ratio (teacher-forced one-step flow MSE with vs. without shuffled actions) as a recurring regression test.
    • Tools/workflow: CI test that fails if action-sensitivity drops below a threshold; reporting per-task or per-scenario.
    • Assumptions/dependencies: Access to action-labeled batches; established bounds for acceptable sensitivity.
  • Tokenizer health checks and dynamic selection (sectors: ML ops, software, gaming)
    • What: Detect perceptual hallucination at the perception stage and select the best tokenizer for the task.
    • How: Monitor urnormu_r^{norm} per frame; if high, switch to a more robust/tokenizer (e.g., an off-the-shelf VAE) or adapt via quick finetuning.
    • Tools/workflow: Runtime tokenizer registry; simple bandit or rule-based switcher driven by urnormu_r^{norm}.
    • Assumptions/dependencies: Multiple tokenizers available; minimal switching overhead; fit-for-purpose licenses.
  • Dataset triage and curation (sectors: dataset engineering, autonomous systems validation)
    • What: Identify low-coverage regions to prioritize additional logging, human play, or expert demos.
    • How: Aggregate urnorm,ufnorm,usnormu_r^{norm}, u_f^{norm}, u_s^{norm} by state clusters or scene coordinates; highlight periphery regions (as in the paper’s density plots).
    • Tools/workflow: Data-report generator that surfaces “under-covered” slices for collection or augmentation.
    • Assumptions/dependencies: Basic state/proxy-state extraction for binning (e.g., object/agent position); reproducible hashing of scenes.
  • Benchmarking and teaching materials using MMBench2 (sectors: academia, edtech, R&D labs)
    • What: Standardize evaluation of hallucination and mitigation methods across 210 tasks with live simulators.
    • How: Use provided code, checkpoints, and browser UI for labs on world modeling, MPC, and active data collection.
    • Tools/workflow: Course modules and reproducible eval scripts; public leaderboards on hallucination metrics and downstream MPC.
    • Assumptions/dependencies: GPU access for students; conformance with dataset licenses.
  • Game AI and simulation QA (sectors: gaming, simulation platforms)
    • What: Improve physics plausibility and action responsiveness of learned simulators and non-player agents.
    • How: Apply coverage-aware sampling for diverse game mechanics; use usnormu_s^{norm} to detect scene-divergent sequences (e.g., “teleporting” artifacts).
    • Tools/workflow: Automated acceptance tests that reject builds with elevated hallucination rates on held-out levels.
    • Assumptions/dependencies: Access to level/task taxonomy; integration hooks for model-driven gameplay.
  • Digital twin drift detection (sectors: manufacturing, energy)
    • What: Detect when a predictive twin’s imagined trajectories diverge from sensor-grounded dynamics.
    • How: Compute ufnormu_f^{norm} and usnormu_s^{norm} on rolling forecasts; flag periods where uncertainty spikes and initiate recalibration or data collection.
    • Tools/workflow: Monitoring service within the twin’s MLOps; scheduled retraining using targeted logs.
    • Assumptions/dependencies: Synchronized action/observation streams; allowances for incremental retraining.
  • Human-in-the-loop data collection via the provided browser interface (sectors: robotics research, edtech)
    • What: Quickly collect high-value demonstrations in unseen tasks and combine with curiosity-selected data.
    • How: Use the UI to record human play for rare states; merge with urnormu_r^{norm}-guided exploration for breadth.
    • Tools/workflow: Session management, data versioning, and post-hoc selection by coverage metrics.
    • Assumptions/dependencies: Moderate operator time; stable interface to live simulators or real devices.

Long-Term Applications

These use cases require further validation in real-world domains, scaling beyond simulation, or integration with larger embodied or multimodal systems.

  • On-device, self-improving household and warehouse robots (sectors: consumer robotics, industrial automation)
    • What: Robots that monitor their own world-model hallucinations and opportunistically collect data where uncertainty is highest to self-correct.
    • How: Run urnorm/usnormu_r^{norm}/u_s^{norm} as background monitors; periodically execute curiosity-selected micro-routines; fine-tune during off-hours.
    • Tools/workflow: Federated/edge training pipeline; safety constraints baked into exploration policies.
    • Assumptions/dependencies: Safe exploration in human environments; battery/compute budgets; robust sim-to-real bridging of the predictors.
  • Hallucination-aware planning in autonomous driving (sectors: automotive)
    • What: Planning stacks that adapt horizon and aggressiveness based on online hallucination risk, reducing rare-event failures.
    • How: Integrate normalized predictors into trajectory scoring; prioritize data collection from low-coverage maneuvers (e.g., odd intersections).
    • Tools/workflow: Joint training across fleets using coverage-aware sampling by scenario taxonomy.
    • Assumptions/dependencies: High-fidelity sensor tokenizers; verification that signals transfer to complex road scenes; regulatory acceptance.
  • Regulatory standards for model-based control with uncertainty gating (sectors: policy, safety certification)
    • What: Require deployment-time uncertainty monitors and data-coverage audits for systems relying on learned world models.
    • How: Define conformance tests using action-sensitivity audits, usnormu_s^{norm} thresholds, and coverage reports across scenario catalogs.
    • Tools/workflow: “Hallucination Audit Suite” for certification authorities and OEMs.
    • Assumptions/dependencies: Community consensus on metrics; reproducible reporting; test scenario availability.
  • General-purpose embodied agents with cross-domain tokenizers (sectors: AI platforms, robotics)
    • What: Agents that dynamically select and adapt tokenizers per environment, guided by round-trip residuals and performance.
    • How: Multi-tokenizer ensembles with routing based on urnormu_r^{norm}; periodic in-domain finetuning to close perceptual gaps.
    • Tools/workflow: Tokenizer registries/marketplaces; lightweight distillation to unify latents.
    • Assumptions/dependencies: Efficient switching; stable interfaces across tokenizers; large-scale pretraining corpora beyond MMBench2.
  • Real-world digital twins with active data governance by coverage (sectors: energy, process industries, smart buildings)
    • What: Twins that steer sensor deployment and logging budgets toward under-covered dynamics regimes.
    • How: Spatial/temporal coverage heatmaps driven by ufnormu_f^{norm} and usnormu_s^{norm}; suggest sensor placements or new operating points.
    • Tools/workflow: Planning tools for instrumentation and experiment design; feedback loops into retraining schedules.
    • Assumptions/dependencies: Cost-effective sensing; safe operation in under-explored regimes.
  • Healthcare and assistive robotics with uncertainty-aware interaction (sectors: healthcare)
    • What: Assistive devices that reduce autonomy or request clarification when hallucination risk increases, especially around novel contexts.
    • How: Online monitors trigger “ask-for-help” behaviors; targeted data collection during supervised sessions to expand safe operating envelopes.
    • Tools/workflow: Clinical supervision workflows; auditable logs of uncertainty-driven decisions.
    • Assumptions/dependencies: Strong safety guardrails; domain-shifted tokenizers for medical environments; regulatory clearance.
  • Multimodal foundation models for control with shared hallucination signals (sectors: AI research, edge AI)
    • What: Extend the predictors to audio/tactile/3D streams within unified world models to inform planning and data acquisition.
    • How: Generalize round-trip residuals to modality-specific tokenizers; maintain inter-seed variance across multimodal denoisers.
    • Tools/workflow: Cross-modal flow-matching architectures; standardized normalization for scene activity across modalities.
    • Assumptions/dependencies: Scalable multimodal datasets with actions; reliable motion/dynamism estimates per modality.
  • Standardized evaluation suites for hallucination robustness (sectors: academia, standards bodies)
    • What: Community benchmarks expanding MMBench2 with real-robot tasks and field datasets; leaderboards on hallucination detection/mitigation.
    • How: Release task packs with live simulators + real logs; require reporting of action-sensitivity, coverage-aware improvements, and MPC downstream metrics.
    • Tools/workflow: Open measurement protocols; reproducibility toolkits.
    • Assumptions/dependencies: Shared investment in dataset curation; clear licensing for real-world data.
  • Marketplace of coverage-aware training services (sectors: MLOps vendors)
    • What: Managed services that retrofit existing world-model training with coverage-aware sampling and data-collection loops.
    • How: Drop-in data loaders and active-collection SDKs; dashboards for coverage heatmaps and ROI on new data.
    • Tools/workflow: Cloud/GPU orchestration; integrations with simulators and robot fleets.
    • Assumptions/dependencies: Customer datasets with action labels; secure connectors to online environments or devices.

Cross-cutting assumptions and dependencies

  • Access to action-labeled data and/or live environments (for both training and targeted data collection).
  • Compute overhead: predictors require additional encode–decode passes and multi-seed denoising for usnormu_s^{norm}; acceptable latency must be confirmed in deployment.
  • Architecture dependence: signals were validated on a Dreamer-4-like visual world model; transfer to other architectures should be tested.
  • Motion normalization: accurate estimates of scene dynamism are needed for reliable normalized predictors.
  • Sim-to-real transfer: effectiveness in noisy, partially observable real settings remains an open empirical question.
  • Governance and safety: exploration policies for data collection must respect operational constraints and human safety.
View Paper Prompt  View All Prompts 

Glossary

  • Action-marginalized hallucination: A failure mode where the model’s predicted future is largely insensitive to the input action, effectively ignoring control. "Action-marginalized (ignored) hallucination. Conditional on a context, the predicted next latent is largely insensitive to the input action."
  • Action shuffle ratio: A diagnostic metric that compares model error with true actions versus shuffled actions to assess action sensitivity. "Action shuffle ratio ↑\uparrow evaluates action sensitivity in the dynamics model by measuring one-step teacher-forced flow MSE relative to batch-shuffled actions."
  • Action-conditioned: Describes a model that conditions its predictions on action inputs. "an action-conditioned generative model that follows an increasingly common two-stage training recipe"
  • AUROC: Area under the ROC curve; a summary measure for binary classification performance across thresholds. "We additionally measure their per-task aggregated AUROC against two binary hallucination labels"
  • Autoregressive rollout: Sequential generation where each step conditions on previous model outputs, often leading to error accumulation. "where an autoregressive rollout will hallucinate, and why, is poorly understood."
  • Behavior cloning (BC): Supervised imitation learning that trains a policy to reproduce demonstrated actions. "behavior cloning (BC) policy heads"
  • Block-causal Transformer: A Transformer with causal structure applied in blocks, supporting temporally ordered modeling. "a dynamics model implemented as a block-causal Transformer"
  • CLIP-ViT/B: A specific CLIP vision transformer variant used to obtain embeddings conditioned on language. "The reward head and BC policy are conditioned on CLIP-ViT/B embeddings of per-task language instructions."
  • Closed-loop MPC: Model Predictive Control that replans using feedback from the current state at each step. "closed-loop MPC using the Cross-Entropy Method (CEM) with horizon H=32H=32 and replanning every $16$ steps."
  • Compounding error: The phenomenon where small prediction errors accumulate over time in sequential predictions. "autoregressive rollouts accumulate compounding error as the prediction horizon increases."
  • Context length: The number of timesteps of history the model conditions on. "using a context length of T=24T=24."
  • Cross-Entropy Method (CEM): A stochastic optimization/planning algorithm that iteratively refines action distributions. "closed-loop MPC using the Cross-Entropy Method (CEM) with horizon H=32H=32 and replanning every $16$ steps."
  • Curiosity rewards: Intrinsic rewards that encourage exploration of uncertain or novel states. "our hallucination predictors serve as curiosity rewards for targeted data collection"
  • Denoiser: The network component in diffusion/flow models that maps noisy inputs to clean targets. "the denoiser's clean-target prediction x^1\hat{x}_1"
  • Discrete regression: Predicting discretized/quantized targets using classification-like losses. "multi-step discrete regression (using symlog⁡\operatorname{symlog} two-hot encoding)"
  • Dreamer 4: A specific world-model architecture and training recipe for model-based RL/control. "introduced in Dreamer~4"
  • Epistemic uncertainty: Uncertainty due to limited knowledge or data, reducible with more information. "by measuring epistemic uncertainty across noise seeds"
  • Euler integration substeps: Numerical integration sub-steps used during sampling in flow/diffusion-based models. "between successive Euler integration substeps"
  • Flow instability: A measure of how much the denoiser’s clean-target prediction fluctuates across integration substeps, indicating poor conditioning. "the flow instability of the dynamics head at a given (context,action)(\text{context}, \text{action}) pair"
  • Flow matching: A training objective that matches model dynamics to a target probability flow field. "trained with the shortcut flow-matching objective"
  • Hallucination: Visually plausible but incorrect predictions that diverge from ground truth. "We use the term hallucination to refer to any such failure: an output that is fluent and visually plausible, yet decoupled from reality."
  • Inter-seed variance: Variation in predictions across different random seeds in stochastic denoising, reflecting epistemic uncertainty. "usu_s, the inter-seed variance of the next-latent prediction across NN independent denoising trajectories"
  • LPIPS: A learned perceptual image similarity metric used to assess reconstruction fidelity. "the loss (pixel MSE + LPIPS \citep{zhang2018unreasonable}) is computed only on masked positions."
  • Masked auto-encoding: A self-supervised pretraining task where masked parts of an input are reconstructed. "a video tokenizer is trained via masked auto-encoding of visual observations"
  • Mean squared error (MSE): A standard loss function measuring average squared differences between predictions and targets. "The BC policy is a deterministic Gaussian policy trained to predict ground-truth actions via an MSE loss"
  • Open-loop rollout: Predicting future sequences without feeding back the true next observation, using model outputs instead. "between open-loop rollout Δ\DeltaPSNR and each of our predictors"
  • Out-of-distribution (OOD): Inputs that differ significantly from the training distribution. "projects out-of-distribution scene structure onto the closest in-distribution exemplar"
  • Patchified: Processing images as sequences of patches for Transformer-based tokenization. "frames ``patchified'' at stride $14$"
  • Perceptual hallucination: A failure originating in the encoder-decoder where reconstructions already deviate from the input. "Perceptual hallucination. The tokenizer's reconstruction of an observation already differs from the observation itself"
  • PSNR: Peak signal-to-noise ratio, a reconstruction quality metric measured in decibels. "Reconstruction PSNR ↑\uparrow evaluates encoder/decoder quality"
  • Register tokens: Special tokens used within the Transformer sequence for internal state or readout. "and $4$ register tokens"
  • Round-trip residual (tokenizer): The reconstruction error after a decode–encode cycle in latent space; used as a hallucination predictor. "Proposed predictor 1: tokenizer round-trip residual."
  • Scene-diverging hallucination: A failure where rollouts predict physically implausible events as horizons grow. "Scene-diverging hallucination. It is well understood that autoregressive rollouts accumulate compounding error as the prediction horizon increases."
  • Self-consistency bootstrap: A training technique distilling coarser-step predictions into finer-step targets for stability. "with a self-consistency bootstrap that distills two coarser-step predictions into one finer-step target."
  • Shortcut-conditioning token: An auxiliary token encoding noise level and step size for conditioning the dynamics. "a shortcut-conditioning token (encoding noise level σ\sigma and step size dd)"
  • Spearman correlation: A rank-based correlation measure assessing monotonic relationships. "we compute Spearman correlations between open-loop rollout Δ\DeltaPSNR and each of our predictors"
  • Symlog two-hot encoding: A discretization scheme using a symmetric log transform and two-hot targets for regression. "using symlog⁡\operatorname{symlog} two-hot encoding"
  • Teacher forcing: Training by feeding the ground-truth previous output rather than the model’s prediction. "one-step teacher-forced flow MSE"
  • Tokenizer: A module that encodes images into latent tokens and decodes them back to pixels. "Our video tokenizer is instantiated as a symmetric encoder-decoder Transformer architecture."
  • Zero-shot: Applying a pretrained model to new tasks without additional training. "Pretraining transfers, to some extent, zero-shot"
View Paper Prompt  View All Prompts 

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 7 tweets with 50 likes about this paper.

Sign Up for Free