Learning from Acquisition: Metadata-driven Multimodal Pre-training for Cardiac MRI

Abstract: Cardiac magnetic resonance imaging (CMR) routinely records structured acquisition metadata, yet most CMR foundation models rely primarily on image-only pre-training and leave this naturally available source of weak semantic supervision largely underexplored. We propose MetaCLIP-CMR, a metadata-driven framework based on Contrastive Language--Image Pre-training (CLIP), which converts imaging modality, anatomical view, scanner vendor, field strength, and scanner model into textual supervision for CMR representation learning. The pretrained image encoder is evaluated on imaging modality classification, cine view classification, and cardiac segmentation. MetaCLIP-CMR achieves 86.8% modality accuracy and 86.5% cine view accuracy, clearly outperforming ImageNet and masked reconstruction initialisations. For downstream cardiac segmentation, MetaCLIP-CMR consistently obtains the highest Dice score across the evaluated ACDC and M&Ms cine short-axis (SAX) settings under both full-data and 20% fine-tuning regimes. Compared with recent image-focused large-scale CMR pre-training models, MetaCLIP-CMR achieves comparable ACDC segmentation performance, while requiring less than 1% of their pre-training image scale. These results suggest that metadata learning offers a natural and easy-to-use strategy for transforming routinely recorded acquisition information into effective supervision for foundation-level CMR representation learning, highlighting the promise of metadata-driven multimodal pre-training.

• The paper introduces a novel metadata-driven approach that converts acquisition data into textual supervision for context-aware image and text alignment.
• It employs a soft-label contrastive loss to improve classification accuracy and segmentation performance, outperforming traditional ImageNet and masked pre-training methods.
• The method demonstrates strong transfer learning capabilities in low-data regimes, significantly enhancing cine anatomical view classification and myocardial segmentation tasks.

Metadata-driven Multimodal Pre-training for Cardiac MRI: An Expert Essay on MetaCLIP-CMR

Introduction

MetaCLIP-CMR presents a technically rigorous approach to multimodal foundation model pre-training for cardiac MRI (CMR) that addresses longstanding limitations of image-only self-supervised learning. By leveraging structured, routinely recorded acquisition metadata—encompassing imaging modality, anatomical view, scanner vendor, field strength, and scanner model—MetaCLIP-CMR generates textual supervision to drive alignment in a high-dimensional joint embedding space. This framework circumvents the necessity for labor-intensive manual clinical text annotations or paired reports, enabling scalable and semantically enriched pre-training.

Methodological Framework

MetaCLIP-CMR builds on the CLIP paradigm, integrating a single-channel ResNet-50 image encoder and a DistilBERT text encoder, each projecting representations into a shared 256-dimensional space. Acquisition metadata is automatically parsed and formulated into natural language sentences (e.g., “cine cardiac MRI in short-axis view acquired on a 3.0T scanner”) via deterministic templates. During pre-training, image slices and metadata-derived text representations are aligned using a novel metadata-aware soft-label contrastive loss, which accounts not only for exact image-text pairs but also assigns partial positive alignment for samples sharing modality or anatomical view attributes.

Figure 1: Overview of MetaCLIP-CMR. Image sequences and metadata-derived text are projected into a shared 256-dimensional space and aligned using the proposed soft-label contrastive loss.

By replacing CLIP's traditional one-hot positive/negative pairing with soft positive targets reflecting acquisition similarities, the supervision appropriately models the semantic structure present in CMR datasets. This contrasts with image-only masked reconstruction pre-training or conventional ImageNet initialization which lack such domain-informed alignment.

Metadata conversion is systematic and fully automated:

Figure 2: Metadata-derived text generation from CMR modality, anatomical view, scanner vendor, field strength, and scanner model.

Evaluation Protocol and Downstream Transfer

The experimental protocol comprises two core settings: probe (linear evaluation on frozen encoder) and Probe+FT (full end-to-end fine-tuning). The model is evaluated on three axes:

• Imaging modality classification (10-way)
• Cine anatomical view classification
• Cine short-axis (SAX) segmentation

Pre-training utilizes a subset of MMCMR-427K with balanced diversity across modalities, vendors, and field strengths. Downstream segmentation is benchmarked on ACDC and M&Ms datasets, both in full-data and low-data (20%) regimes, with mean 3D Dice as the primary metric.

Empirical Results and Comparative Analysis

Classification Transfer

MetaCLIP-CMR achieves modality and view classification accuracy of 86.8% and 86.5%, respectively, under probe+fine-tune, significantly outperforming ImageNet and masked reconstruction initialization (relative improvements $>$8\%). Notably, the already high probe (linear) performance indicates that acquisition-aware semantics are directly encoded within the embedding space, demonstrating explicit transfer of metadata-driven supervision.

Segmentation Transfer

For cine SAX segmentation, MetaCLIP-CMR attains the highest Dice across all train/test settings, including the challenging few-shot regime:

• ACDC 100%: 0.9021
• ACDC 20%: 0.8795
• M&Ms 100%: 0.8368
• M&Ms 20%: 0.8008

Comparative analysis underscores that, despite being pre-trained on just $<$1% as many images as large-scale DINO-based models (“Towards a cardiovascular magnetic resonance foundation model for multi-task cardiac image analysis” (Wang et al., 25 Dec 2025)) or CineMA [Fu2026], MetaCLIP-CMR approaches their benchmark segmentation accuracy. This efficiency is attributed directly to the semantic supervision provided by acquisition metadata.

Representative visualisations reinforce the quantitative findings:

Figure 3: Representative cine SAX segmentation results on ACDC and M&Ms under full-data and 20% settings. RV, myocardium, and LV are shown in red, green, and blue, respectively.

Accurate myocardial boundary detection—especially under label-scarce conditions—shows that the MetaCLIP-CMR pre-trained encoder provides robust, transferable features for pixel-level anatomical delineation, even though segmentation itself is not explicitly encoded in the metadata.

Implications, Limitations, and Future Directions

MetaCLIP-CMR demonstrates that acquisition metadata—long overlooked by most pre-training pipelines—can be operationalized as an effective, scalable, and clinically meaningful supervision signal. The architecture efficiently encodes modality- and view-awareness into the visual backbone. Practical implications include:

• Enhanced performance for CMR applications, particularly in low-label or cross-vendor deployment contexts.
• Drastic reduction in the requirement for curated clinical textual annotations or vast image-only pretraining, increasing feasibility for deployment in clinical research settings with heterogeneous device setups.
• Potential for robust transfer to other downstream tasks such as disease classification, quality assurance, and cross-modal retrieval.

Theoretically, the positive outcomes of soft-label contrastive learning suggest that structured, weak semantic signals can meaningfully regularize visual representation learning, with possible relevance for other medical imaging domains. Limitations include a reliance on the completeness and accuracy of metadata, and potential model sensitivity to missing or inconsistent acquisition information. Future work should explore richer protocol metadata (e.g., patient demographics, study indications), multi-institutional temporal modeling, and extension to additional diagnostic and cross-modal tasks.

Conclusion

MetaCLIP-CMR rigorously establishes metadata-driven multimodal pre-training as a credible and efficient alternative to image-only self-supervised or weakly supervised approaches in cardiac MRI. By efficiently leveraging structured acquisition metadata via automated text generation and a novel soft-label contrastive loss, the framework delivers strong linear and fine-tuned transfer across both acquisition-aware classification and anatomy-specific segmentation. The study provides compelling evidence that metadata constitutes a potent, underutilized source of supervision for foundation model pre-training in medical imaging, warranting further exploration and generalization across imaging modalities.