Aligning Agentic World Models via Knowledgeable Experience Learning

Abstract: Current LLMs exhibit a critical modal disconnect: they possess vast semantic knowledge but lack the procedural grounding to respect the immutable laws of the physical world. Consequently, while these agents implicitly function as world models, their simulations often suffer from physical hallucinations-generating plans that are logically sound but physically unexecutable. Existing alignment strategies predominantly rely on resource-intensive training or fine-tuning, which attempt to compress dynamic environmental rules into static model parameters. However, such parametric encapsulation is inherently rigid, struggling to adapt to the open-ended variability of physical dynamics without continuous, costly retraining. To bridge this gap, we introduce WorldMind, a framework that autonomously constructs a symbolic World Knowledge Repository by synthesizing environmental feedback. Specifically, it unifies Process Experience to enforce physical feasibility via prediction errors and Goal Experience to guide task optimality through successful trajectories. Experiments on EB-ALFRED and EB-Habitat demonstrate that WorldMind achieves superior performance compared to baselines with remarkable cross-model and cross-environment transferability.

• The paper presents the WorldMind framework, which integrates Process Experience and Goal Experience to align agentic world models and mitigate physical hallucinations.
• It employs an explicit World Knowledge Repository that updates internal models based on error-driven corrections and successful trajectory heuristics.
• Empirical evaluations on standard benchmarks show significant improvements in success rates and cross-model transfer of experiential learning.

Experiential Alignment for Agentic World Models: The WorldMind Framework

Motivation and Problem Statement

Despite the increasing versatility of LLMs in embodied AI, a fundamental modal disconnect persists between their high-level semantic knowledge and low-level procedural grounding. As a result, LLM-based agents often generate plans that are logically coherent yet physically infeasible, manifesting as "physical hallucinations"—actions that violate environmental constraints or object affordances. Existing alignment approaches primarily rely on resource-intensive parametric training, which fails to flexibly accommodate open-ended environmental variability or to systematically correct for plan-execution mismatches.

The paper "Aligning Agentic World Models via Knowledgeable Experience Learning" (2601.13247) introduces the WorldMind framework—a paradigm that augments embodied agents with an explicit, experience-driven World Knowledge Repository (WKR). This framework enables agents to dynamically align and ground their internal world models through the dual synthesis of Process Experience (derived from prediction errors) and Goal Experience (distilled from successful task executions).

Figure 1: Conceptual illustration of experiential alignment: the agent aligns its internal world model via Process Experience and Goal Experience.

Framework Architecture

WorldMind: Explicit World Knowledge Augmentation

WorldMind shifts agent world modeling from static parametric storage toward an explicit, symbolic scaffold external to model weights. The agent's decision process is formalized as a World Knowledge-Augmented Markov Decision Process (WK-MDP), which extends the traditional POMDP tuple with a dynamic world knowledge component: $\mathcal{W} = \{\mathcal{W}_p, \mathcal{W}_g\}$, representing process-level and goal-level experiential abstractions, respectively.

• Process Experience ($\mathcal{W}_p$): Verbalized causal rules synthesized in response to prediction errors, enforcing physical feasibility and filtering hallucinated/plausibility-breaking states.
• Goal Experience ($\mathcal{W}_g$): Procedural heuristics extracted from successful trajectories, guiding the agent along efficient, contextually valid task-solving paths.

The agent's policy jointly produces both actions and predicted future states, with a learning objective favoring both task success and minimal semantic divergence between predicted and actual environment states.

Figure 2: Overview of the WorldMind framework; the agent constructs a World Knowledge Repository by unifying Process Experience (from prediction errors) and Goal Experience (from successful trajectories) to guide grounded simulation.

Operational Dynamics: Predict-Act-Verify and Experience Synthesis

The agent executes in a loop of prediction, execution, and outcome verification. On execution failures or mismatches, state abstraction, judgment, and self-reflection modules collaboratively synthesize process-level rules, which are incorporated into $\mathcal{W}_p$ to gate further simulation and ensure adherence to physical laws. Concurrently, successful trajectories are mined for context-robust heuristics, which populate $\mathcal{W}_g$ for future reference.

WorldMind’s inference pipeline leverages semantic similarity-based retrieval from the WKR to condition both state simulation and action selection, explicitly gating predictions to avoid updating the internal model on ungrounded or visually ambiguous states. This approach minimizes both physical and semantic hallucinations, suppresses redundant simulation, and improves inference efficiency relative to sampling-heavy or parametric fine-tuning paradigms.

Empirical Evaluation

Benchmarking Environments and Experimental Protocol

WorldMind is benchmarked on EB-ALFRED and EB-Habitat—standard embodied agent evaluation suites—across diverse fine-grained capability subsets including spatial reasoning, visual appearance sensitivity, and complex instruction-following. Core metrics include Success Rate (SR; strict full-goal completion) and Goal Condition (GC; partial credit for correct subgoal completion). Primary baselines include ReAct, SimuRA, ReasoningBank, and AWM across LLM backbones GPT-3.5-turbo and GPT-4.1-mini.

Core Results and Numerical Highlights

WorldMind demonstrates strong and consistent improvements across strict and process-level evaluation criteria:

• On EB-Habitat (GPT-4.1-mini), SR reaches 50.8%, +9.2% over ReAct; on EB-ALFRED (GPT-3.5-turbo), SR increases from 44.4% to 48.0%.
• GC is similarly elevated: On EB-ALFRED (GPT-3.5-turbo), GC jumps from 50.4% (ReAct) to 63.0% (WorldMind), and on EB-Habitat achieves 57.2%, significantly exceeding baselines.
• Robustness is evidenced across challenging subsets, with SR of 86% in EB-Habitat's Base setting for both major backbones.

WorldMind also generalizes to low-level atomic control (EB-Navigation), reliably improving average SR and confirming cross-granularity policy grounding.

Cross-Model and Cross-Environment Transfer

Experience stored in the World Knowledge Repository is substantially transferable across heterogeneous backbone architectures. Exchange of WKR between GPT-3.5-turbo and GPT-4.1-mini results in marked SR and GC gains over model-specific learning, empirically validating the framework's model-agnostic universality.

Figure 3: Performance of cross-model experience transfer; symbolic, externalized world knowledge transfers successfully between different LLM-based agent backbones.

WorldMind further scales to hybrid environments (e.g., Embodied Web Agent setups requiring both web and physical task interleaving), maintaining significant gains in completion rates and reducing error modes associated with physical hallucinations and execution dead-ends.

Error and Ablation Analysis

Disaggregated error types reveal that Process Experience sharply reduces invalid actions (105→67 for GPT-3.5-turbo on EB-Habitat), while Goal Experience dampens logical misalignments and wrong terminations. Ablation studies confirm the necessity of both components for optimal agent robustness; isolated removal of either results in degraded SR/GC tradeoffs.

Figure 4: Habitat (GPT-3.5-turbo). Error distribution profile highlights decreased incidence of invalid actions and wrong terminations when using WorldMind.

Theoretical and Practical Implications

WorldMind operationalizes the predictive coding paradigm for LLM-based embodied agents, demonstrating that training-free, experience-driven alignment of internal world models is both effective and scalable. By externalizing knowledge and focusing on error-driven symbolic rule construction, the framework sidesteps the rigidity and inefficiency of static parametric adherence to physical laws.

The capacity for cross-model experience sharing opens avenues for collective agent learning, federated symbolic world modeling, and bootstrapping generalist agentics. Moreover, the interpretability and modularity of the world knowledge repository facilititate robust tool use, context injection, and explainable action planning.

There remain open challenges, such as scaling up to multi-agent shared world models with consistent symbolic consensus and mapping the precise latent effect of symbolic WKR updates on model state and transition dynamics.

Conclusion

WorldMind introduces a robust, generalizable, and explicitly interpretable paradigm for aligning agentic world models through knowledgeable experiential learning. By fusing process-level error correction and goal-level heuristic distillation in a symbolic external memory, the approach delivers significant reductions in physical and logical failure, enhances transfer across architectures and environments, and establishes a scalable blueprint for future development of generalist, robust agentic AI systems (2601.13247).