Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPT

Abstract: In this paper, we aimed to provide a review and tutorial for researchers in the field of medical imaging using LLMs to improve their tasks at hand. We began by providing an overview of the history and concepts of LLMs, with a special focus on LLMs. We then reviewed the current literature on how LLMs are being used to improve medical imaging, emphasizing different applications such as image captioning, report generation, report classification, finding extraction, visual question answering, interpretable diagnosis, and more for various modalities and organs. The ChatGPT was specially highlighted for researchers to explore more potential applications. We covered the potential benefits of accurate and efficient LLMs for medical imaging analysis, including improving clinical workflow efficiency, reducing diagnostic errors, and assisting healthcare professionals in providing timely and accurate diagnoses. Overall, our goal was to bridge the gap between LLMs and medical imaging and inspire new ideas and innovations in this exciting area of research. We hope that this review paper will serve as a useful resource for researchers in this field and encourage further exploration of the possibilities of LLMs in medical imaging.