Fashion130K: An E-commerce Fashion Dataset for Outfit Generation with Unified Multi-modal Condition

Abstract: Recent research work on fashion outfit generation focuses on promoting visual consistency of garments by leveraging key information from reference image and text prompt. However, the potential of outfit generation remains underexplored, requiring comprehensive e-commercial dataset and elaborative utilization of multi-modal condition. In this paper, we propose a brand-new e-commerce dataset, named Fashion130k, with various occasions, models, and garment types. For the consistent generation of garment, we design a framework with Unified Multi-modal Condition (UMC) to align and integrate the text and visual prompts into generation model. Specifically, we explore an embedding refiner to extract the unified embeddings of multi-modal prompts, within which a Fusion Transformer is proposed to align the multi-modal embeddings by adjusting the modality gap between text and image. Based on unified embeddings, the attention in generation model is redesigned to emphasis the correlations between prompts and noise image, inducing that the noise image can select the pivotal tokens of prompts for consistent outfit generation. Our dataset and proposed framework offer a general and nuanced exploration of multi-modal prompts for generation models. Extensive experiments on real-world applications and benchmark demonstrate the effectiveness of UMC in visual consistency, achieving promising result than that of SoTA methods.

• The paper introduces a comprehensive Fashion130K dataset of 130K image pairs and structured prompts to benchmark multi-modal outfit generation.
• The paper describes the Unified Multi-modal Condition framework with a Fusion Transformer and selective, masked attention to improve visual-text alignment.
• The paper demonstrates that UMC outperforms baselines in LPIPS, DINO, and FFA metrics, proving its efficacy in preserving garment fidelity and scalability.

Fashion130K: A Comprehensive Resource and Framework for Multi-modal E-commerce Outfit Generation

Introduction

Fashion outfit generation under multi-modal conditions presents complex challenges due to the intricate interplay between visual references and structured textual descriptions. The paper “Fashion130K: An E-commerce Fashion Dataset for Outfit Generation with Unified Multi-modal Condition” (2605.10127) addresses two critical problems: the lack of a large-scale, diverse benchmark tailored to real-world e-commerce scenarios, and the ineffective integration of visual and linguistic conditioning in generative models. The authors introduce Fashion130K, a public dataset curated from extensive e-commerce records, and design a Unified Multi-modal Condition (UMC) framework leveraging advanced architectural modules to refine and align multi-modal embeddings, specifically targeting garment consistency and fidelity in the generative process.

Fashion130K Dataset

Fashion130K provides 130,386 model-garment image pairs and structured prompts, establishing it as the most comprehensive open-source dataset for controllable outfit generation to date. Distinctive attributes of Fashion130K include:

• Rich diversity: Nine first-level clothing categories (e.g., women’s wear, men’s wear, accessories, underwear, kids’ wear) and 203 fine-grained second-level categories. The dataset covers a broad spectrum of real-world backgrounds and multiple aspect ratios.
• Multi-modal prompt structure: Each data entry contains a garment image, a corresponding model image (where the model is wearing the referenced garment), and a structured, garment-agnostic textual prompt describing the occasion, background, and model attributes. Garment details are deliberately omitted from the text, enforcing appearance control through the visual prompt.
• Curation methodology: High-quality samples are selected through a pipeline involving de-duplication via image features, removal of low-quality images with OCR and sticker detection, and manual annotation for garment-model correspondence.
• Multi-resolution support: Images are available at a variety of aspect ratios and resolutions, supporting robust training and evaluation of generative models under realistic e-commerce constraints.

This dataset enables rigorous benchmarking of garment-text-driven image synthesis and facilitates research on multi-modal conditioning strategies as well as post-training for multi-resolution diffusion models.

Unified Multi-modal Condition (UMC) Framework

The UMC framework addresses key deficiencies in prior art by explicitly aligning and integrating text and image embeddings for generative diffusion models. Its primary innovations are:

Embedding Refiner with Fusion Transformer

Standard diffusion-based methods inadequately fuse multi-modal conditions, frequently concatenating modal embeddings and introducing significant modality gaps that degrade fidelity. The UMC’s Embedding Refiner employs a divide-and-merge strategy. The central Fusion Transformer:

• Preprocesses text and image embeddings through independent branches to retain modality-specific semantics.
• Unifies these representations with masked self-attention and MLP modules, maintaining essential intra-modality properties while resolving spatial misalignment.
• Applies masking strategies to prevent text-induced contamination of garment appearance, a crucial factor for faithful subject transfer.

Ablation studies confirm that the Fusion Transformer variant achieves the lowest validation loss and highest visual-semantic consistency scores relative to simple MLP, joint, and parallel transformer alternatives.

Attention-Enhanced Correlation: Selective and Masked Attention

The UMC introduces Selective Attention mechanisms to optimize token-level alignment between noised latent representations and multi-modal condition tokens.

• Selective Attention: Noised image tokens (queries) attend to multi-modal condition tokens (keys/values), with top-k selection enforcing focus on the most relevant condition attributes. Empirically, $k=8$ yields optimal trade-off between information capacity and noise suppression.
• Masked Attention: To further prevent cross-contamination (particularly from ambiguous textual guidance), attention is masked between modalities at both the refiner and generator stages.
• These modules synergistically preserve critical garment attributes and suppress irrelevant or redundant prompts, as validated by both loss metrics and perceptual similarity analyses.

Experimental Results

UMC is fine-tuned from the FLUX.1 pretrained model using LoRA adaptation, with experiments conducted on the test splits of Fashion130K and the VITON-HD benchmark.

Quantitative Metrics

• Visual fidelity: UMC achieves the strongest performance in LPIPS, DINO, and FFA on Fashion130K—improving over leading baselines such as Qwen Image Edit 48.
• Consistency and alignment: DINO and FFA scores reflect superior preservation of garment structure and detail. Text-image alignment is competitive but does not subdue appearance supervision from the visual prompt.
• Robust generalization: UMC maintains leading performance even when transferred to the VITON-HD dataset, indicating strong scalability and dataset transferability.

Qualitative Assessment

UMC consistently reproduces reference garment attributes, such as skirt design or accurate accessory placement, where other methods (e.g., Qwen Image Edit, FLUX.1 Kontext, OmniGen2, DreamO) demonstrate mode collapse or attribute drift under complex multi-modal prompts. The increased fidelity is attributable to the refined embedding alignment and selective attention operations.

Module Ablation

The impact of individual modules (Embedding Refiner, Selective Attention, Masked Attention) is systematically validated. The combination of all proposed components yields the highest consistency and fidelity, outperforming models omitting any single innovation.

Implications and Future Directions

Fashion130K establishes a new foundation for e-commerce fashion generation tasks, particularly those requiring nuanced garment-text conditioning under realistic and diverse scenarios (field backgrounds, pose variety, accessories). The structured prompt design minimizes text ambiguity and maximizes visual control, positioning the dataset as an indispensable benchmark for the next generation of virtual try-on, visual search, or personalized content generation engines.

The UMC framework advances architectural paradigms for multi-modal conditioning, emphasizing the necessity of both careful representation alignment and token-level selection strategies for complex synthesis tasks. In broader terms, methodologies from UMC—particularly modular embedding refiners and tailored attention modules—are extensible to other structured conditioning settings beyond fashion, including medical imaging, fine-grained image editing, or scenario-aware generative agents.

Future developments may include scaling both the refiner and selective attention modules for deeper architectures, expanding conditioned attribute diversity, integrating user interaction or feedback as dynamic prompts, and exploring emergent abilities with even more structured or loosely supervised modalities. The continued expansion of multi-modal datasets and refinement of cross-modal representation learning will be central to progress in AI-driven content generation.

Conclusion

The “Fashion130K” dataset and the UMC architecture represent significant advances in e-commerce fashion generation, providing both a robust multi-modal benchmark and a technically rigorous framework for addressing modality alignment and controllability in diffusion models. Empirical results validate the superiority of UMC in garment consistency and visual fidelity. Collectively, these contributions will catalyze further research in structured multi-modal generation and practical deployment of controllable content synthesis for fashion and beyond (2605.10127).