gWorld (8B, 32B): Renderable Code Generation

• gWorld (8B, 32B) is a model that uses large-scale vision-language architectures to generate executable code for visual world modeling, enhancing interpretability and structure.
• It achieves high precision by converting image inputs into simulation code, leading to up to 79.6% instruction accuracy and render fail rates below 1% in complex GUI tasks.
• Its modular design integrates standard programming pipelines with rendering engines, enabling efficient procedural simulation across diverse domains such as urban planning and GUI forecasting.

Visual world modeling via renderable code generation refers to a set of methodologies and systems wherein high-level representations of physical or virtual environments are expressed as executable code, which, when rendered, synthesizes pixel-level images or interactive scenes. Rather than directly generating raw pixel data (as in conventional generative models), systems based on renderable code generation predict source code (e.g., Python, HTML/CSS, shader code) that defines the generative logic of the scene. These mechanisms have found application across domains such as physical system modeling, graphical user interface (GUI) state forecasting, and automatic virtual scene synthesis. This modeling approach yields advantages in structural fidelity, interpretability, and modularity, and often provides superior precision in text/numeric content relative to pixel synthesis alone.

1. Core Principles and Definitions

Renderable code generation in visual world modeling is rooted in the insight that many aspects of the visible world—physical phenomena, engineered artifacts, or UI layouts—are more effectively represented through symbolic generative processes than as undifferentiated pixel arrays. The paradigm consists of three fundamental steps:

1. Analysis: The system (generally a Vision LLM or VLM) inputs an image (or state/action pair) and infers a generative description—sometimes in natural language, but crucially as executable code.
2. Synthesis: The generated code simulates or reconstructs the underlying process or scene, defining the visual output.
3. Rendering: Executing the code within the appropriate environment produces a synthetic image or scene whose properties can be evaluated against the original.

The paradigm enables both interpretation (discovering the underlying structure) and generation (producing new, diverse instances consistent with that structure) of complex visual environments, relying on standard programming and rendering pipelines for execution.

2. Methodologies and Architectures

Vision-LLM–Driven Approaches

Recent advances employ large-scale multimodal transformers trained on triplets of images, textual descriptions, and code snippets (Eppel, 8 Jan 2026, Koh et al., 2 Feb 2026). The standard pipeline is:

• Encoding: The image (or image+action pair) is embedded via a vision backbone (e.g., CLIP-style CNN, patch-based ViT).
• Cross-Modal Decoding: Code is generated by a language decoder (e.g., GPT-5, Gemini-2.5, Qwen3-VL), conditioned on the image embedding, maximizing

$$L(\theta) = -\mathbb{E}_{(I, C)}\left[\sum_{t=1}^{T} \log P_{\theta}(c_t \mid c_{<t}, f_{enc}(I)) \right]$$

or with code-specific extensions for GUI states:

$$\mathcal{L} = -\sum_{t=1}^{T-1}\sum_{k=1}^{M_t+L_t} \log P_\theta(y_{t,k}\mid y_{t,<k}, S_t, A_t)$$

• Rendering: Output code is run in suitable sandboxes (e.g., Jupyter, headless Chromium, 3D engines) to yield pixel output.

Visual Programming Frontends

Alternative systems leverage visual programming languages (VPLs), in which users construct procedural “flowcharts” representing scene generation logic (Lucanin, 2012). These VPLs compile user-assembled graphs to structured code (e.g., Python), which is subsequently executed in real-time renderers for interactive scene creation.

Approach	Input	Output Code Domain	Rendering Target
VLM-based	Image/(Image,Action)	Python, HTML/CSS, Blender API	Matplotlib, Browser, Blender
VPL (e.g., vIDE)	Flowchart diagram	Python (procedural API)	Ogre3D

3. Domains of Application

The renderable-code paradigm is applicable across a broad spectrum:

• Complex Emergent Physical Systems: Simulating water caustics, waves, Chladni plates, flame evolution, sand dunes (Eppel, 8 Jan 2026).
• Growth and Pattern Systems: Modeling vegetation (e.g., L-systems), reaction-diffusion textures.
• Urban and Structural Environments: Procedural city and building layouts with randomness-driven variety (Lucanin, 2012).
• Graphical User Interfaces: Predicting next-state GUI renderings as executable HTML/CSS, supporting mobile GUI world-modeling for agent training and evaluation (Koh et al., 2 Feb 2026).
• Diagrammatic/Symbolic Content: Programmatic rendering of text, handwritten/printed symbols, or floor mosaics.

In all cases, renderable code allows for both concise parameterization and structural decomposition, resulting in interpretability and domain-adaptive flexibility.

4. Evaluation Metrics and Empirical Findings

Evaluation of renderable code-based visual modeling relies on both quantitative and qualitative benchmarks:

• Quantitative Image Similarity:
  • $L_p$ norms: $$\|I - \hat{Y}\|_p = \left(\sum_x |I(x) - \hat{Y}(x)|^p \right)^{1/p}$$
  • Structural Similarity Index (SSIM): Captures perceptual similarity between original and rendered images.
  • DINO-based cosine similarity for GUI screenshots (Koh et al., 2 Feb 2026).
• Task-Specific Metrics:
  • Instruction Accuracy (IAcc.): Fraction of model-generated GUI code whose renderings are judged action-consistent by VLM ensembles.
  • Render Fail Rate: Fraction of outputs failing to render valid HTML.
• User/Model Matching Benchmarks: For Im2Sim (Eppel, 8 Jan 2026), VLMs achieved 50–80% accuracy in matching synthetic to real images across a range of domains (chance: 10%). GPT-5 (Color) scored 80% accuracy, outperforming humans (70%).

Empirical findings show that leading models such as gWorld (8B, 32B) set a new Pareto frontier in GUI world-modeling, with gWorld 32B achieving up to 79.6% IAcc., exceeding models over 50× larger in parameter count (Koh et al., 2 Feb 2026). In urban scene modeling, the combination of procedural code and rendering engines yields immediate, physically plausible virtual environments (Lucanin, 2012).

5. Advantages and Limitations

Strengths

• Structural and Textual Precision: Code-based generation provides exact reproduction of layout primitives and text/numeric content (e.g., timestamps, labels) (Koh et al., 2 Feb 2026).
• Interpretability and Compositionality: Generated code exposes underlying mechanisms (e.g., Snell’s law, L-systems), enabling system decomposition and modular simulation (Eppel, 8 Jan 2026).
• Scalability: Synthetic code-based training data can be generated at scale with predictable scaling laws, facilitating model improvements with data volume (Koh et al., 2 Feb 2026).
• Speed and Simplicity: The renderable code pipeline (model → code → renderer) is efficient, with render times around 0.3 seconds per sample for GUI scenes, in contrast to diffusion-based pipelines.

Limitations

• Fine Detail Reproduction: VLMs exhibit limited fidelity on low-level textures, spatial arrangements, and parameter tuning; precise visual correspondence is rarely matched (Eppel, 8 Jan 2026).
• Rendering Code Quality: Earlier baselines suffered high render fail rates (up to 40% for code-generation baselines), whereas state-of-the-art systems such as gWorld keep these rates below 1%.
• Asymmetry in Abstraction: VLMs demonstrate high-level, mechanistic comprehension, but are less successful at faithfully replicating detailed patterns (e.g., micro-structure of caustics).

This suggests that while renderable code models advance mechanistic and compositional understanding, pixel-level fidelity remains challenging at the extremes of detail.

6. Comparison to Prior Work

Prior approaches to image-to-simulation or code-based scene synthesis include:

• pix2code: Mapping GUI screenshots to HTML/CSS code (limited to GUI domains).
• InverseCSG, DeepCAD: 3D model recovery from images, focusing on geometric primitives.
• CAD-Coder, Geocode: Diagram-to-shape program translation.

The Im2Sim and gWorld frameworks deliver several novel advances:

• Direct modeling of natural images encompassing complex, emergent systems, rather than clean, symbolic diagrams.
• Generation of fully runnable simulation or GUI code spanning multiple domains, not merely fitting parametric surfaces or shapes.
• Unified VLM-based interpretation and code authoring, rather than separate, pipeline-based systems.
• Evaluation of performance via the closed image-to-simulation-to-image loop, not merely via code correctness.

By integrating procedural code generation, symbolic reasoning, and fast rendering, these methods set a new standard for interoperability, interpretability, and downstream agent utility.

7. System Design: Visual Programming for Scene Modeling

Lučanin et al. (Lucanin, 2012) outline a VPL (vIDE) for procedural urban scene generation:

• User Interface: Flowchart-based diagrams with block (action), branch (test), and start/end nodes; constraint-driven editing enforces valid control flow.
• Compilation Pipeline: Diagrams are transformed to GOTO AST, then to WHILE-tree AST, and finally to indented Python code.
• Procedural API: High-level constructs (e.g., ManhattanLayout, ProceduralBuildingGenerator) provide methods for layout, structure, details, and stochastic variation.
• Integration: Code is executed in C++/Python hybrid environments, creating scene graphs (e.g., via Ogre3D) immediately renderable in real-time graphics engines.

A plausible implication is that such visual programming tools may further democratize procedural content creation, translating conceptual diagrams directly into efficient, renderable simulations without requiring expertise in low-level graphics coding.

---

References:

• "Coding the Visual World: From Image to Simulation Using Vision LLMs" (Eppel, 8 Jan 2026)
• "Generative Visual Code Mobile World Models" (Koh et al., 2 Feb 2026)
• "Visual definition of procedures for automatic virtual scene generation" (Lucanin, 2012)