Qwen-Image Baseline – Diffusion-Based Synthesis

• Qwen-Image Baseline is a diffusion-based image generation and editing model that combines a frozen multimodal LLM, hybrid VAE, and a diffusion transformer for high-fidelity outputs.
• It employs a three-part architecture with around 27B parameters that integrates semantic encoding, image tokenization, and cross-attention to balance global understanding with fine-grained visual detail.
• The model supports multi-task functions—including text-to-image, image-to-image, and multi-image composition—with advanced training strategies and reinforcement learning to enhance editing quality.

Qwen-Image Baseline

Qwen-Image is a diffusion-based image generation and editing foundation model in the Qwen series, designed for high-fidelity image synthesis, complex text rendering, and robust editing capabilities. It achieves state-of-the-art performance across generation, editing, and multilingual text rendering benchmarks, and serves as the backbone for advanced multi-image composition systems such as Qwen-MICo. The architecture combines a frozen multimodal LLM (Qwen2.5-VL), a hybrid variational autoencoder (VAE), and a Multimodal Diffusion Transformer (MMDiT) to conditionally generate images from text, image, or mixed inputs.

1. Model Architecture

Qwen-Image employs a three-part architecture with approximately 27 billion parameters:

• Qwen2.5-VL (VLM): Acts as the text/image encoder, providing high-level semantic representations. The ViT submodule has 32 layers (16/16 heads, 80 head size), while the LLM submodule features 28 layers (28/4 heads, 128 head size), totaling ~7B parameters.
• Hybrid VAE: Functions as the image tokenizer, with an encoder (11 layers, 16 channels, 54M params) and decoder (15 layers, 16 channels, 73M params).
• Multimodal Diffusion Transformer (MMDiT): The core generative backbone (60 layers, 24/24 heads, 128 head size, hidden dim 12288, ~20B params).

The VLM and VAE are frozen during inference, with active parameterization dominated by the MMDiT transformer. All input representations—semantic (from Qwen2.5-VL) and reconstructive (from VAE)—are concatenated and input to the MMDiT cross-attention structure, balancing global semantic comprehension and fine-grained visual fidelity (Wu et al., 4 Aug 2025).

2. Data Pipeline and Training Strategy

The training pipeline emphasizes both massive scale (billions of pairs) and refined balance across natural scenes, human figures, complex text, and synthetic images. It incorporates a rigorous 7-stage filtering regime to maximize quality and domain balance, followed by hierarchical resampling for rare-content enrichment.

A progressive curriculum training strategy ramps up image resolutions (256×256 → 640×640 → 1328×1328) and gradually integrates more complex text rendering tasks, moving from simple non-text synthesis to dense, multi-paragraph, multilingual text layout. Synthetic data augmentation and iterative filtering increase data diversity and text coverage (Wu et al., 4 Aug 2025).

Annotation uses Qwen2.5-VL to yield both free-text captions and structured JSON metadata (scene types, styles, artifacts).

3. Multi-Task Learning and Objective Design

Qwen-Image is natively multi-task, supporting:

• Text-to-Image (T2I): Text-only conditioning.
• Text + Image-to-Image (TI2I): Joint image and instruction-based editing.
• Image-to-Image (I2I)/Inpainting: Input image refinement or completion.

All tasks are trained with a flow-matching loss: $$\mathcal{L}_{\mathrm{FM}} = \mathbb{E}_{(x_0, h), x_1, t}\; \big\|\, v_\theta(x_t, t, h) - v_t \big\|_2^2$$ where $x_0$ is the clean image latent, $h$ is the VLM guidance latent, and $x_t$ is an interpolated latent between $x_0$ and Gaussian noise $x_1$.

For advanced instruction adherence and output quality, Qwen-Image further incorporates supervised fine-tuning on human-annotated image splits and reinforcement learning objectives:

• Direct Preference Optimization (DPO): Uses human preference pairs for reward shaping.
• Group Relative Policy Optimization (GRPO): On-policy group-based advantage objectives, including KL-divergence regularization (Wu et al., 4 Aug 2025).

Distributed optimization utilizes AdamW, mixed precision, and head-wise tensor parallelism at component level (Megatron-LM framework).

4. Qwen-Image-Edit and Qwen-MICo: Multi-Image Composition Baselines

Qwen-Image-Edit utilizes the above MMDiT backbone for single-reference (text+image) editing, preserving identity and context through dual-encoding.

Qwen-MICo is a fine-tuned variant for arbitrary multi-image composition, foundational to the MICo-150K benchmark (Wei et al., 8 Dec 2025):

• No architectural change: The MMDiT transformer is unmodified; only the conditioning stream (visual tokens) is extended to handle $N$ reference images, concatenated in the input.
• Training setup: The visual encoder and VAE are frozen. Fine-tuning is performed solely on the MMDiT weights with AdamW (β₁=0.9, β₂=0.999, weight decay=0.01, learning rate $2 \times 10^{-6}$, batch 64, 120,000 steps, 5,000-step linear warmup). Standard DDPM denoising loss is used:

$$\mathcal{L}_{\mathrm{diffusion}} = \mathbb{E}_{x_0, \epsilon, t} \left\| \epsilon - \epsilon_\theta(z_t; \{I_i\}, p) \right\|^2$$

where $\{I_i\}$ is the set of $N$ source images and $p$ the prompt.

Qwen-MICo is trained on all 150K pairs in MICo-150K, spanning 7 composite tasks—including an 11K “Decompose-and-Recompose” set—and ablated variants excluding or isolating real/synthetic De&Re data (Wei et al., 8 Dec 2025).

5. Evaluation Protocols and Metrics

Evaluation uses MICo-Bench, consisting of 1,000 cases stratified across the 7 major MICo tasks and De&Re recompositions.

The core metric is the Weighted-Ref-VIEScore, combining identity, fidelity, and artifact penalizations:

• Presence weight $W$: Fraction of sources present in the generated image.
• Prompt-Following ($PF$), Subject-Resemblance ($SR$), Naturalness ($N$), and Artifacts ($A$): Provided by a GPT-4o VQA system.
• Overall score:

$$\text{Weighted-Ref-VIEScore} = W \times \sqrt{PF \times SR} \times \sqrt{N \times (10 - A)}$$

Presence is assessed by QwenVL2.5-72B (non-faces) or ArcFace (faces, thresholds at 0.50 or 0.45). Single high-quality references are used for reliability in similarity and prompt adherence scoring. The structure penalizes overfitting to copy–paste and enforces compositional fidelity (Wei et al., 8 Dec 2025).

6. Benchmark Performance and Comparative Analysis

The table below summarizes Qwen-MICo’s performance (Weighted-Ref-VIEScore) versus Qwen-Image-2509 on MICo-Bench for 3-reference tasks.

Task	Qwen-MICo	Qwen-Image-2509
2O + 1S	56.12	56.00
3O	59.56	45.32
2M + 1W	59.04	42.63
2W + 1M	58.96	52.46
3M	50.11	48.40
3W	56.19	50.78
2P + 1S	60.97	49.70
1P + 2O	54.92	50.64
2P + 1O	52.16	54.65
1P + 2C	55.82	51.77
1P + 1C + 1O	54.26	47.91

Qwen-MICo outperforms Qwen-Image-2509 on 10 out of 11 3-source tasks, with gains up to +14.24 ("3O"), despite using two orders of magnitude less data for fine-tuning. Qwen-MICo also removes the hard cap of $N=3$ reference images present in previous baselines, extending to arbitrary $N$ (Wei et al., 8 Dec 2025).

7. Strengths, Limitations, and Research Directions

Strengths

• Identity preservation: High ArcFace similarity (≥0.50) across single/multi-person and multi-source tasks.
• Robust object placement: Correct spatial arrangement in high-$N$ scenarios.
• Aesthetic consistency: Coherent lighting, shading, and style blending derived from references.

Limitations

• Overcrowding: Mild misalignment arises with 6+ objects or tightly packed scenes.
• Depth ambiguity: Occasional "floating" artifacts when underlying scene geometry is ambiguous.
• Fine detail loss: Degradation on intricate, small source objects.

Qwen-MICo yields even prompt following and compositional fidelity exceeding Qwen-Image-2509, except in specialized human–object-interaction (HOI) tasks (notably “2P+1O”), likely due to extensive object-interaction pretraining in the latter (Wei et al., 8 Dec 2025).

Ongoing challenges include explicit 2D layout conditioning, scaling to highly cluttered (8+ source) scenes, and extension to video/multi-frame, 3D, and interactive editing settings. The combined releases of MICo-150K, MICo-Bench, and Qwen-MICo establish a rigorous and accessible foundation for open research in multi-image composition.