GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning

Abstract: We present GLM-4.1V-Thinking, GLM-4.5V, and GLM-4.6V, a family of vision-LLMs (VLMs) designed to advance general-purpose multimodal understanding and reasoning. In this report, we share our key findings in the development of the reasoning-centric training framework. We first develop a capable vision foundation model with significant potential through large-scale pre-training, which arguably sets the upper bound for the final performance. We then propose Reinforcement Learning with Curriculum Sampling (RLCS) to unlock the full potential of the model, leading to comprehensive capability enhancement across a diverse range of tasks, including STEM problem solving, video understanding, content recognition, coding, grounding, GUI-based agents, and long document interpretation. In a comprehensive evaluation across 42 public benchmarks, GLM-4.5V achieves state-of-the-art performance on nearly all tasks among open-source models of similar size, and demonstrates competitive or even superior results compared to closed-source models such as Gemini-2.5-Flash on challenging tasks including Coding and GUI Agents. Meanwhile, the smaller GLM-4.1V-9B-Thinking remains highly competitive-achieving superior results to the much larger Qwen2.5-VL-72B on 29 benchmarks. We open-source both GLM-4.1V-9B-Thinking and GLM-4.5V. We further introduce the GLM-4.6V series, open-source multimodal models with native tool use and a 128K context window. A brief overview is available at https://z.ai/blog/glm-4.6v. Code, models and more information are released at https://github.com/zai-org/GLM-V.

• The paper introduces a scalable reinforcement learning framework that enables dual 'thinking' and 'non-thinking' inference in unified vision–language models.
• It employs a transformer-based architecture with ViT encoding, MLP visual projection, and chain-of-thought capabilities to support multimodal reasoning over images and videos.
• The models achieve state-of-the-art results on 42 benchmarks, demonstrating superior cross-domain generalization and parameter efficiency.

Versatile Multimodal Reasoning via Scalable Reinforcement Learning: The GLM-4.5V and GLM-4.1V-Thinking Models

Model Design and Multimodal Architecture

The GLM-V series (GLM-4.1V-Thinking, GLM-4.5V, GLM-4.6V) embodies a highly unified transformer-based vision-language architecture optimized for scalable multimodal reasoning and generalization. The architectural core comprises a ViT encoder (AIMv2-Huge initialization), an MLP-based visual projector, and a LLM which acts as a unified multimodal decoder. Video input support is natively integrated through 3D convolutions and time index token encoding, allowing the model to retain temporal coherence and operate on videos at native resolutions and aspect ratios. 2D-RoPE and 3D-RoPE modifications enable spatial and temporal reasoning over variable aspect ratios and high-resolution images, while bicubic interpolation adapts absolute position embeddings for arbitrary input shapes.

Figure 1: Shared GLM-V architecture: ViT encoder, MLP projector, and large language decoder provide unified image/video/text processing and multimodal reasoning, with explicit temporal encoding for videos.

The GLM-4.5V (106B-A12B MoE model) and GLM-4.1V-Thinking (9B dense model) instantiate these capabilities at two different computational scales, with GLM-4.5V supporting dual “thinking” and “non-thinking” output modes via prompt flags, facilitating both efficient and chain-of-thought (CoT) inference.

Data Curation for Broad Reasoning and Robust Perception

Substantial effort is devoted to data curation across multiple axes of vision-language capability. Large-scale, high-quality image-caption data is filtered using heuristics, CLIP-based relevance, concept-balanced resampling, and factual recaptioning (see Figure 2), producing a corpus with minimal hallucination and enhanced density of factual content.

Figure 2: Recaptioning pipeline outputs more factual and less noisy image-text pairs, mitigating the effect of web-scale hallucinations.

Interleaved image-text document alignment leverages both web and academic corpora with custom pipelines targeting high-information-density content (scientific illustrations, diagrams, GUI screenshots, etc.). OCR data is synthesized and mined from synthetic renderings, natural scenes with Paddle-OCR, and structured academic documents. Visual grounding is supported by 40M LAION-GLIPv2 natural image annotations and 140M GUI-specific question-answer pairs, and video data is agglomerated, filtered, and de-duplicated with a human-in-the-loop approach for compositionality and caption faithfulness.

Domain-specific instruction and long-chain-of-thought formats are meticulously enforced for RL cold start, using special tags for answer boxing and reasoning, improving parsing and reward precision, and enabling verifiable reasoning evaluations. The corpus covers general visual reasoning, STEM, document analysis, GUI agent trajectories, and more, maximizing cross-domain interaction during RL.

Supervised Fine-Tuning and Dual-Mode Inference

Supervised fine-tuning (SFT) bridges foundation pretraining and RL, strategically switching from knowledge-injection to reasoning-style alignment. For verifiable and open-ended tasks, SFT datasets are strictly formatted for explicit reasoning/answer demarcation, and mixture-of-experts architectures are used with rigorous parallelism and balance optimization. Both multimodal and text-only long-form reasoning data are used to maintain language and cross-modal consistency.

Distinctively, GLM-4.5V/4.6V support explicit “thinking” and “non-thinking” inference by prompt flag, allowing resource-constrained, high-throughput decoding as well as CoT-style traceable reasoning.

Reinforcement Learning with Curriculum Sampling (RLCS): Data, Reward, and Stability

The paper’s central technical contribution is a reinforcement learning recipe that scales across multimodal domains without catastrophic forgetting or reward hacking. RLCS combines curriculum learning with adaptive, online and offline sample difficulty ranking and dynamic expansion ratios (using EMA tracking), filtering out low or over-hard samples during rollouts and maximizing meaningful gradient throughput.

A comprehensive domain-specific reward infrastructure is constructed (see Figure 3), using extraction policies (rule-based/LLM-extraction), content- and style-aware validators, and task-specific checking (numeric, textual, action, reasoning structure, etc). The reward pipeline is engineered for robustness: flaws in a single domain’s verifier can poison or collapse RL training, as empirically demonstrated.

Figure 3: Reward hacking in multimodal RL: noisy or insecure verifiers cause reward inflation and performance collapse despite progress in other sub-domains, necessitating resistant verifiers for stable RL.

RL training employs GRPO optimization with the KL and entropy penalties disabled, and applies large batches with rollouts dynamically packed and sequenced to maximize compute utilization. Per-sample loss yields higher stability than per-token, and top-p sampling with $p=1$ is empirically optimal for RL consistency.

Cross-Domain Generalization and Mutual Capability Transfer

Empirical results show extensive cross-domain transfer and mutual benefit during RL, as visualized in Figure 4. Training in one domain, such as STEM, results in measurable improvement in disjoint tasks such as visual grounding and general VQA; joint (mix-all) training yields further increases except in domains with extreme specificity (grounding, GUI-agent), where targeted multi-domain schedules remain necessary.

Figure 4: RL-induced cross-domain generalization: single-domain RL yields positive performance transfer, while mixed RL maximizes collective advancement, visualized as performance deltas per domain.

Further, GLM-4.5V’s use of scalable RL, robust reward design, and comprehensive domain scheduling achieves strong mutual reinforcement and capability alignment, a prerequisite for deploying universal multimodal reasoning agents.

Benchmarking and Numerical Results

GLM-4.5V and GLM-4.1V-Thinking are comprehensively benchmarked on 42 tasks across VQA, STEM (MMMU, MathVista, LogicVista), OCR/chart/document, long-document understanding, grounding, spatial reasoning, GUI agents, coding (Design2Code, Flame-React-Eval), and video understanding. On nearly all open benchmarks, GLM-4.5V outperforms state-of-the-art open-source models such as Qwen2.5-VL-72B, Step-3-321B, InternVL3, and Kimi-VL—even matching or exceeding closed-source Gemini-2.5-Flash on 22 benchmarks (see Figure 5).

Figure 5: GLM-4.5V achieves near-linear scaling from 9B to 106B, outperforming prior open models and matching closed-source Gemini-2.5-Flash on challenging benchmarks.

Notably, GLM-4.1V-9B-Thinking, despite only 9B parameters, outperforms Qwen2.5-VL-72B (72B parameters) on 29 out of 42 tasks, demonstrating the efficacy of the architecture and training regime for parameter efficiency. Case studies in the appendix show detailed qualitative reasoning traces on challenging tasks: GUI recognition and planning (Figure 6), front-end code generation from UI (Figure 7), video scene and action interpretation (Figures 8–10), chart QA (Figure 8), spatial and document reasoning (Figures 17–18), and code debugging (Figure 9).

Limitations and Perspective

Despite the robust capabilities, GLM-4.5V inherits persistent issues of current VLMs: (1) reward models for RL only assess outcomes, not the reasoning path, thus permitting “correct answers” with hallucinated justification; (2) RL-induced instabilities remain, though mitigated via reward and data pipeline improvements; (3) perceptual bottlenecks, especially in cluttered or ambiguous inputs, can still undermine downstream reasoning. Addressing intermediate-chain reward modeling and adversarial reward hacking remains an open challenge.

From a broader perspective, the demonstrated positive transfer in RL across modalities suggests new directions for joint training and “capability bootstrapping”—eyeing hybrid workflows where, e.g., visual coding tasks could improve text-only coding generalization. As model performance saturates mainstream benchmarks, diagnostic datasets explicitly targeting reasoning chain hallucination, reward shortcutting, and robust, grounded inference are critical future milestones.

Conclusion

GLM-4.5V and GLM-4.1V-Thinking, as presented, expand the frontier of open vision-LLMs under a unified reasoning-centric RL regime. By integrating robust architectural innovations, principled data curation, comprehensive reward engineering, and curriculum-informed RL, these models deliver superior parameter efficiency, strong cross-domain generalization, and state-of-the-art results on a broad spectrum of multimodal benchmarks. These findings motivate future research on reward design for reasoning-path supervision, adversarial robustness in RL, and continuously scalable, alignment-aware multimodal models.

---

Reference: "GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning" (2507.01006)