VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation

Abstract: Code has emerged as a precise and executable medium for reasoning and action in the agent era. Yet, progress has largely focused on language-centric tasks such as program synthesis and debugging, leaving visual-centric coding underexplored. Inspired by how humans reason over sketches, we advocate SVG code as a compact, interpretable, and executable visual representation. We introduce VCode, a benchmark that reframes multimodal understanding as code generation: given an image, a model must produce SVG that preserves symbolic meaning for downstream reasoning. VCode covers three domains - general commonsense (MM-Vet), professional disciplines (MMMU), and visual-centric perception (CV-Bench). To assess symbolic fidelity, we propose CodeVQA, a novel evaluation protocol in which a policy model answers questions over rendered SVGs; correct answers indicate faithful symbolic preservation. Empirically, frontier VLMs struggle to generate faithful SVGs, revealing a persistent gap between language-centric and visual-centric coding. To close this gap, we introduce VCoder, an agentic framework that augments VLMs along two axes: (i) Thinking with Revision, which iteratively analyzes discrepancies and refines SVG code; and (ii) Acting with Visual Tools, where detectors and parsers supply structured cues such as objects, shapes, and text beyond the model's intrinsic capacity. Across benchmarks, frontier VLMs with strong reasoning capabilities score well overall yet remain limited in professional knowledge and 3D reasoning. VCoder delivers a 12.3-point overall gain over the top-performing Claude-4-Opus. Human studies show that both humans and VLMs perform worse on rendered SVGs, their consistency reveals the promise of symbolic visual representation. The benchmark and code are available at https://github.com/CSU-JPG/VCode.

• The paper introduces VCode, reframing image comprehension as SVG generation to capture symbolic meaning.
• It employs iterative revision and external vision tools to refine and improve the semantic fidelity of generated SVGs.
• Empirical results show that advanced models yield significant gains in symbolic coding, highlighting persistent gaps in current approaches.

VCode: Benchmarking Multimodal Coding with SVG as Symbolic Visual Representation

Introduction and Motivation

VCode introduces a paradigm shift in multimodal understanding by reframing image comprehension as a code generation task, specifically targeting the translation of natural images into symbolic SVG representations. Unlike conventional pixel-based approaches, SVG code offers a compact, interpretable, and executable abstraction, aligning with human reasoning over sketches and structural outlines. The benchmark is constructed by repurposing established multimodal datasets—MM-Vet (commonsense), MMMU (professional disciplines), and CV-Bench (vision-centric perception)—and tasks models with generating SVG code that preserves the symbolic meaning of the input image for downstream reasoning.

Figure 1: VCode pipeline: RGB image (left) is translated into symbolic SVG code (middle) and rendered back into an image (right), aiming to preserve symbolic meaning and provide a compact, interpretable, and executable representation.

Benchmark Design and Evaluation Protocols

VCode comprises 464 image-question pairs spanning three domains, with a balanced taxonomy distribution (Figure 2). The evaluation protocol, CodeVQA, assesses symbolic fidelity by requiring a policy model to answer questions over the rendered SVG; correct answers indicate successful preservation of semantic content. The benchmark also employs SigLIP embedding similarity and code token length as auxiliary metrics, emphasizing both semantic preservation and representational efficiency.

Figure 2: Distribution of VCode tasks across general, professional, and vision-centric categories.

The evaluation objective is formalized as minimizing the discrepancy between symbolic information in the original and rendered images:

$$\mathcal{L} = \min \big| I(\mathcal{V}) - I(\widetilde{\mathcal{V}}) \big|$$

where $I(\cdot)$ denotes symbolic information. Embedding-based similarity (SigLIP) and CodeVQA accuracy are used to quantify representational fidelity.

VCoder: Agentic Augmentation for Visual-Centric Coding

Direct image-to-SVG generation remains challenging due to long-context code requirements, cross-modal output blindness, and weak visual fineness. VCoder addresses these with two complementary test-time augmentations:

1. Thinking with Revision: The model iteratively analyzes discrepancies between the original and rendered images, revising SVG code to improve fidelity. This is implemented as a loop where the model comments on differences and updates the code accordingly.
2. Acting with Visual Tools: External modules (object detectors, segmenters, OCR) provide structured cues—object categories, locations, shapes, and text—which are translated into SVG code signals. This enables the model to overcome limitations in low-level perception and encode fine-grained details.

   Figure 3: Left: Thinking with Revision—iterative discrepancy analysis and SVG refinement. Right: Acting with Vision Tools—external modules supply structured cues for more faithful renderings.

Experimental Results and Analysis

Main Results

Empirical evaluation across proprietary and open-source VLMs reveals a persistent gap between language-centric and visual-centric coding. Closed-source models (GPT-5, Gemini-2.5-Pro) outperform open-source baselines, with VCoder (Claude-4-Opus backbone) achieving a +12.3-point overall gain over the top-performing baseline. However, even the best SVG-based results trail the raw-image upper bound, indicating substantial headroom for improvement.

Key findings include:

• Reasoning ability correlates with coding fidelity: Models with stronger reasoning produce more faithful symbolic renderings.
• SVG specialists underperform: Models trained on synthetic SVG corpora (OmniSVG, StarVector) struggle with natural images due to long-context and semantic abstraction requirements.
• Professional and vision-centric tasks are challenging: Scores on MMMU and CV-Bench remain modest, especially for 3D spatial reasoning and domain-specific knowledge.

Ablation Studies

Ablations demonstrate the incremental benefits of vision tools and revision strategies:

• Vision tools: Adding location, category, shape, and text cues yields steady gains, with full tool integration providing the largest improvement (Figure 4).
• Revision rounds: Iterative refinement improves performance, especially for models with strong difference analysis capabilities (Figure 5).
• Input modality: Translating images into linguistic captions before coding (Img2Text2SVG) outperforms direct image-to-SVG generation, highlighting the utility of language as an intermediate representation.

  Figure 4: Effects of vision tool modules on CodeVQA performance; full tool integration yields the largest overall improvement.

  

  Figure 5: Impact of revision rounds; performance improves with iterative refinement, especially for models with strong reasoning.

Human Studies and Policy Evaluation

Human studies reveal that both humans and VLMs perform worse on rendered SVGs compared to raw images, but their consistency suggests the promise of symbolic visual representation for advancing human-like multimodal intelligence. Policy evaluation across different models and human annotators shows alignment in reasoning from high-level representations, despite inevitable information loss.

Implementation Considerations

The VCoder framework is implemented in PyTorch, leveraging an RTX 4090 GPU with a maximum output length of 16,384 tokens. Vision tools are integrated via Florence-2 (object detection), SAM-2 (segmentation), and OpenOCR (text extraction). The revision loop is formalized in Algorithm 1, enabling iterative code refinement based on visual difference analysis.

for t in range(T): delta = model.compare(original_image, rendered_image) revised_code = model.revise(svg_code, delta) rendered_image = render_svg(revised_code)

Scaling considerations include long-context handling, efficient tokenization, and modular integration of external vision tools. Limitations arise from the fidelity of external detectors and the inherent abstraction loss in SVG representations.

Qualitative Examples

Qualitative results demonstrate progressive improvements in rendering fidelity through initial decoding, visual-tool assistance, and revision. VCoder produces SVGs that are more faithful and interpretable for downstream reasoning, especially in knowledge-intensive and vision-centric tasks.

Figure 6: Original image example from VCode benchmark.

Implications and Future Directions

VCode establishes a rigorous benchmark for visual-centric coding, exposing the limitations of current VLMs in symbolic abstraction and reasoning. The agentic augmentation strategies in VCoder—revision and vision-tool integration—provide a pathway for more faithful and efficient visual code generation. The findings suggest that future research should focus on end-to-end vision–language coders with scalable training data, improved long-context reasoning, and autonomous tool invocation.

Theoretical implications include the need for models that bridge perception and symbolic reasoning, moving beyond pixel-level similarity to semantic preservation. Practically, VCode enables more interpretable, executable, and compact representations for downstream tasks in agentic intelligence, GUI automation, and visual question answering.

Conclusion

VCode reframes multimodal understanding as a visual-centric coding task, leveraging SVG as a symbolic representation and introducing CodeVQA for principled evaluation. The benchmark reveals persistent gaps in current VLMs, with agentic augmentation via VCoder yielding substantial improvements. Human studies and ablation analyses underscore the promise of symbolic visual coding for advancing multimodal intelligence. Future work should pursue scalable, end-to-end coders capable of faithful symbolic abstraction and autonomous tool integration.