Geometry Aware World Model Architecture

• Geometry Aware World Model Architecture is a framework where agents' internal representations are explicitly structured using mathematical geometry and group actions.
• The approach leverages explicit geometric priors, such as Euclidean and projective transformations, to improve active inference, prediction, and spatial generalization.
• It features a modular design that decomposes tasks into world modeling, belief updating, and policy selection, driving robust 3D scene understanding and autonomous control.

A geometry aware world model architecture is a computational framework in which the agent’s internal representation of the environment is explicitly structured by mathematical geometry, typically via group actions or explicit 3D fields. This embedding of geometry into the model’s latent space fundamentally alters inference, action policy, prediction, and the ability to generalize spatial relationships. Such architectures span theoretical frameworks defined by group actions on state spaces to practical systems leveraging neural rendering, explicit volumetric fields, or learned attention patterns parameterized by scene geometry. Their utility extends to embodied agents, vision-language systems, model-based control, 3D scene understanding, and generative prediction.

1. Geometric Structure in Internal Representation

A core innovation in geometry aware world models is the explicit definition of the internal state space $W$ as a manifold endowed with a group action, typically representing possible coordinate or perspective transformations. In the canonical study by Sergeant-Perthuis et al.:

• The agent’s world model $W$ is a topological space on which a group $G$ acts as a set of invertible transformations.
  • Euclidean model: $W = \mathbb{R}^3$, with group $G = E(3)$ (affine rotations and translations).
  • Projective model: $W = \mathbb{P}_3(\mathbb{R})$, with $G = \mathrm{PGL}(4, \mathbb{R})$ (projective transformations).
• The action of $g \in G$ on $x \in W$ (written $g \cdot x$) structurally determines how the world model encodes spatial relationships and how alternative perspectives or imagination are constructed.
• In the projective case, transformations include nonlinear magnification aligned to image-plane distortions and depth-dependent effects, whereas Euclidean actions preserve metric properties and uniform uncertainty volumes.

This geometric formalization aligns the internal machinery of world models with fundamental principles of spatial cognition, enabling explicit “perspective-taking” and high-fidelity simulation of environment dynamics (Sergeant-Perthuis et al., 2023).

2. Geometry-Aware Active Inference and Epistemic Value

The geometry imposed on $W$ directly impacts how the agent calculates its epistemic (curiosity-driven) exploration policy. In geometry aware architectures:

• The epistemic value of a candidate policy $m$ is evaluated as

  $$C(m) = \mathbb{E}_{y \sim P_Y} [ \mathrm{KL}[\, P_{X|y} \,\|\, Q \,] ] = I(X; Y)$$

  which quantifies expected information gain about the latent state $X$ from a new observation $Y$ after executing $m$.

• Simulating action $m$ translates to pushing the current belief $Q$ forward by the group action $\psi_m$: $Q \rightarrow \psi_{m*} Q$.
• The transformation of epistemic value under group action:
  • Euclidean: $C(\psi_{g*}Q) = C(Q)$ for any $g \in E(n)$, i.e. no action increases expected information—leading to idleness.
  • Projective: $C(\psi_{m*}Q)$ is depth-dependent, with the Jacobian determinant introducing a magnification factor; epistemic value increases as the agent approaches an object, driving curiosity-based approach behaviors.
• This leads to two qualitatively distinct behaviors: Euclidean models exhibit no directional exploration bias, while projective models induce strong gradients in epistemic value aligned with approach to objects of interest (Sergeant-Perthuis et al., 2023).

3. Architectural Module Decomposition in Geometry-Aware World Models

Geometry aware architectures instantiate modular decomposition where geometry is tightly embedded in each principal component:

Module	Functionality	Where Geometry Appears
World-Model	Internal state ($W$), group $G$ actions, sensory likelihood	Definition of $W$, group $G$
Belief Updater	Bayesian update, group push-forward	Push-forward under $G$, $Q_X$
Policy Selector	Computes $C(\psi_{m*}Q_X)$, selects $m^*=\arg\max C$	All $m$ as group elements

In practice, the belief $Q_X$ is represented as a Gaussian in $W$, group actions are applied to beliefs during “imagination”, and Bayesian updates are performed given new observations after action execution. Policy selection operates in the transformed space and directly leverages the geometry—policies are “perspective-dependent” and intrinsically geometric (Sergeant-Perthuis et al., 2023).

In neural instantiations, analogous modules exist: encoders extract spatial features with geometric priors (e.g., via epipolar lines (Tobin et al., 2019), 3D occupancy (Xie et al., 25 Jun 2025)), world models operate on group-structured tokens, and decision modules use explicit 3D reasoning (Guo et al., 14 Nov 2025).

4. Extensions Across 3D World Modeling Paradigms

Recent developments show that geometry aware world models underpin progress from 2D perception to complete 3D cognition:

• 3D representation backbones: Voxel grids, implicit fields (e.g., NeRF, SDF), Gaussian Splatting, point clouds, and meshes serve as the structural substrate for geometry (Xie et al., 25 Jun 2025).
• Geometry-aware attention: Techniques such as Epipolar Cross Attention route feature aggregation along epipolar lines between views, producing efficient, perspective-consistent latent representations that outperform unstructured pooling (Tobin et al., 2019).
• Physical scene generation and reasoning: Integration of differentiable physics, occupancy constraints, and spatial priors enables simulation of plausible 3D scene dynamics, physical stability, and manipulation (Xie et al., 25 Jun 2025).
• Policy and control: Geometry-aware modules allow agents to perform spatial interaction and decision-making grounded in calibrated 3D world models (e.g., for active exploration, object manipulation, trajectory control) (Sergeant-Perthuis et al., 2023, Chen et al., 28 May 2025).

FantasyWorld and related frameworks introduce bidirectional cross-branch supervision between video and geometry branches, allowing mutual refinement and superior coherence across modalities (Dai et al., 25 Sep 2025).

5. Empirical Properties and Domain-Specific Implementations

Geometry aware architectures demonstrate consistent empirical benefits:

• Curiosity-driven exploration: Projective geometry endows agents with a spatially biased information gradient, leading to approach and exploration behavior—demonstrated to exceed Euclidean baselines by an order of magnitude in epistemic value and approach behavior (Sergeant-Perthuis et al., 2023).
• Vision-language systems: Injection of explicit geometry (epipolar, relative pose, group structure) into transformer-based VLMs and LLMs unlocks strong 3D spatial reasoning, with rationale modules and regression heads enabling state-of-the-art numerical spatial inference (Guo et al., 14 Nov 2025, Cao et al., 1 Dec 2025).
• 3D-aware generative models: Surrogate models for physical fields (e.g., CFD, heat transfer) benefit from local geometric masks and global PDE parameter conditioning, yielding high-fidelity, controllable generative simulations (Doganay et al., 29 Jan 2026).
• 3D scene reconstruction and manipulation: Explicit geometric structures such as dual-state segmentation-aware Gaussians (Hu et al., 5 Jun 2025) support novel view synthesis, object-level editing, and cross-state alignment with precise mathematical constraints.
• Autonomous driving and robotics: Trajectory-conditioned geometry extraction, metric-accurate rendering, and diffusion-based prediction yield high action fidelity and spatial coherence in world models designed for control and planning (Chen et al., 28 May 2025).

6. Theoretical and Practical Significance

Geometry aware world models unify perception, imagination, and action under rigorous mathematical frameworks:

• They provide a systematic means to structure internal representations according to task-relevant geometric principles, which is crucial for spatial awareness, perspective transformation, and physical consistency.
• They enable robust information integration, aligning prediction, active exploration, and planning via group-theoretic operations on beliefs and memories.
• Their modular design (world-model, update, policy) allows straightforward adaptation to new group structures, sensor models, and downstream tasks.
• Empirically, geometry aware models demonstrate superior data efficiency, zero-shot generalization, persistent object permanence, and real-time control across diverse domains from embodied AI to high-fidelity simulation.

The explicit imposition of a geometric inductive bias thus constitutes a key innovation, bridging fundamental cognitive mechanisms and modern deep learning architectures for spatially grounded world modeling (Sergeant-Perthuis et al., 2023, Tobin et al., 2019, Xie et al., 25 Jun 2025, Guo et al., 14 Nov 2025, Dai et al., 25 Sep 2025).