MedImageInsight: An Open-Source Embedding Model for General Domain Medical Imaging

Abstract: In this work, we present MedImageInsight, an open-source medical imaging embedding model. MedImageInsight is trained on medical images with associated text and labels across a diverse collection of domains, including X-Ray, CT, MRI, dermoscopy, OCT, fundus photography, ultrasound, histopathology, and mammography. Rigorous evaluations demonstrate MedImageInsight's ability to achieve state-of-the-art (SOTA) or human expert level performance across classification, image-image search, and fine-tuning tasks. Specifically, on public datasets, MedImageInsight achieves SOTA in CT 3D medical image retrieval, as well as SOTA in disease classification and search for chest X-ray, dermatology, and OCT imaging. Furthermore, MedImageInsight achieves human expert performance in bone age estimation (on both public and partner data), as well as AUC above 0.9 in most other domains. When paired with a text decoder, MedImageInsight achieves near SOTA level single image report findings generation with less than 10\% the parameters of other models. Compared to fine-tuning GPT-4o with only MIMIC-CXR data for the same task, MedImageInsight outperforms in clinical metrics, but underperforms on lexical metrics where GPT-4o sets a new SOTA. Importantly for regulatory purposes, MedImageInsight can generate ROC curves, adjust sensitivity and specificity based on clinical need, and provide evidence-based decision support through image-image search (which can also enable retrieval augmented generation). In an independent clinical evaluation of image-image search in chest X-ray, MedImageInsight outperformed every other publicly available foundation model evaluated by large margins (over 6 points AUC), and significantly outperformed other models in terms of AI fairness (across age and gender). We hope releasing MedImageInsight will help enhance collective progress in medical imaging AI research and development.

• The paper introduces an open-source embedding model that captures essential features from diverse medical images.
• It leverages deep learning to standardize image representation and support cross-domain analysis.
• The methodology improves diagnostic integration and provides scalable insights for medical research.

Overview of "Formatting Instructions for ICLR 2023 Conference Submissions"

The paper "Formatting Instructions for ICLR 2023 Conference Submissions" provides a comprehensive guide on the formatting requirements and submission process for papers intended for the International Conference on Learning Representations (ICLR) 2023. Authored by Antiquus S. Hippocampus, Natalia Cerebro, Amelie P. Amygdale, Ji Q. Ren, and Yevgeny LeNet, the document focuses on ensuring the uniformity and presentation quality of submissions.

Submission Process

ICLR mandates electronic submissions through the platform OpenReview, emphasizing the necessity for authors to refer to the ICLR website for further instructions. After acceptance, specific formatting changes must be applied to adhere to the camera-ready requirements.

Formatting Specifications

The document describes that submissions must utilize a variant of the NeurIPS format. Key specifications include:

• Text Layout: Confined within a 5.5-inch by 9-inch rectangle, with appropriate margins, font type, and size.
• Document Elements: Includes detailed guidance on paragraphs, headings, citations, footnotes, figures, and tables.

Authors are instructed to utilize specific style files obtainable from the ICLR website, ensuring alignment with standardized formatting.

Notation and Symbols

The authors encourage standardized mathematical notation, referencing the deep learning textbook by Goodfellow et al. While the adoption of this notation style is not compulsory, it aids in the consistency of mathematical expressions across submissions.

Illustrative Examples

The paper provides explicit examples for implementing figures and tables, highlighting the importance of visual coherence. The use of graphic resources is advised to maintain clarity across different print formats.

Concluding Directions

The paper concludes by advising against any alterations to the established formatting parameters. This includes prohibitions on modifying text box dimensions and font sizes, ensuring all submissions maintain the specified structure. Additionally, guidance is offered for generating compatible PostScript or PDF files, alongside addressing potential margin issues.

Implications and Future Directions

Uniform formatting rules, such as those outlined in this document, are critical to maintaining the professional and cohesive presentation of academic work. As conferences advance, the potential integration of automated formatting tools or real-time submission feedback systems could further streamline adherence to these detailed guidelines. Additionally, as digital submissions become more sophisticated, considerations for accessibility and diverse formatting outputs might also evolve, ensuring broader inclusiveness in academic dissemination.

In summary, the paper provides a detailed roadmap for aligning submissions with ICLR’s formatting criteria, essential for ensuring a consistent conference presentation standard. This meticulous attention to formatting not only facilitates peer review but also enhances the professional visibility of research contributions within the academic community.