Emergency Department Decision Support using Clinical Pseudo-notes
Abstract: In this work, we introduce the Multiple Embedding Model for EHR (MEME), an approach that serializes multimodal EHR tabular data into text using pseudo-notes, mimicking clinical text generation. This conversion not only preserves better representations of categorical data and learns contexts but also enables the effective employment of pretrained foundation models for rich feature representation. To address potential issues with context length, our framework encodes embeddings for each EHR modality separately. We demonstrate the effectiveness of MEME by applying it to several decision support tasks within the Emergency Department across multiple hospital systems. Our findings indicate that MEME outperforms traditional machine learning, EHR-specific foundation models, and general LLMs, highlighting its potential as a general and extendible EHR representation strategy.
- Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis. IEEE journal of biomedical and health informatics, 22(5):1589–1604, September 2018.
- A Survey of Large Language Models, September 2023. arXiv:2303.18223 [cs].
- Tabllm: Few-shot classification of tabular data with large language models. In International Conference on Artificial Intelligence and Statistics, pages 5549–5581. PMLR, 2023.
- Mug: A multimodal classification benchmark on game data with tabular, textual, and visual fields. In Findings of the Association for Computational Linguistics: EMNLP 2023, Dec. 2023.
- AMMU: A survey of transformer-based biomedical pretrained language models. Journal of Biomedical Informatics, 126:103982, February 2022.
- The shaky foundations of large language models and foundation models for electronic health records. npj Digital Medicine, 6(1):135, July 2023.
- BEHRT: Transformer for Electronic Health Records. Scientific Reports, 10(1):7155, December 2020.
- Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health records, June 2021. arXiv:2106.11360 [cs].
- Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. npj Digital Medicine, 4(1):86, December 2021.
- CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks. page 22.
- OMOP Common Data Model.
- ExBEHRT: Extended Transformer for Electronic Health Records to Predict Disease Subtypes & Progressions, April 2023. arXiv:2303.12364 [cs].
- Incorporating medical knowledge in BERT for clinical relation extraction. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5357–5366, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.
- Generating contextual embeddings for emergency department chief complaints. JAMIA Open, pages 160–166, 2020.
- MIMIC-III, a freely accessible critical care database. Scientific Data, 3(1):160035, December 2016.
- Multimodal clinical benchmark for emergency care (mc-bec): A comprehensive benchmark for evaluating foundation models in emergency medicine, 2023.
- Medical transformer for multimodal survival prediction in intensive care: integration of imaging and non-imaging data. Scientific Reports, 13:10666, 2023.
- A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics. Nature Biomedical Engineering, 7:743–755, 2023.
- MIMIC-IV.
- Medbert: A pre-trained language model for biomedical named entity recognition. In 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pages 1482–1488, 2022.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
