Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data

Abstract: Knee osteoarthritis (OA) is the most common musculoskeletal disease without a cure, and current treatment options are limited to symptomatic relief. Prediction of OA progression is a very challenging and timely issue, and it could, if resolved, accelerate the disease modifying drug development and ultimately help to prevent millions of total joint replacement surgeries performed annually. Here, we present a multi-modal machine learning-based OA progression prediction model that utilizes raw radiographic data, clinical examination results and previous medical history of the patient. We validated this approach on an independent test set of 3,918 knee images from 2,129 subjects. Our method yielded area under the ROC curve (AUC) of 0.79 (0.78-0.81) and Average Precision (AP) of 0.68 (0.66-0.70). In contrast, a reference approach, based on logistic regression, yielded AUC of 0.75 (0.74-0.77) and AP of 0.62 (0.60-0.64). The proposed method could significantly improve the subject selection process for OA drug-development trials and help the development of personalized therapeutic plans.

• The paper proposes a multimodal machine learning method using CNNs and GBM to predict knee OA progression from radiographs and clinical data, achieving an AUC of 0.80.
• The multimodal model significantly outperforms traditional logistic regression in predicting OA progression and provides insights into relevant image features.
• This multimodal machine learning approach enables automatic clinical prediction and can accelerate drug development by improving patient selection.

Multimodal Machine Learning-Based Prediction of Knee Osteoarthritis Progression

The presented study explores the application of multimodal machine learning for predicting knee osteoarthritis (OA) progression, combining radiographic imaging with clinical data. This approach aims to address the significant challenge of forecasting OA progression, which has implications for drug development and surgical interventions.

Methodology

The authors propose a novel method that leverages convolutional neural networks (CNNs) combined with Gradient Boosting Machines (GBM) for predicting OA progression. The model integrates raw knee radiographs with clinical data, including Kellgren-Lawrence (KL) grades, symptomatic assessments, and demographic information. The dataset utilized, derived from the Osteoarthritis Initiative (OAI) and Multicenter Osteoarthritis Study (MOST), consists of 4,928 knees for training and 3,918 knees for testing the model. Notably, the research excludes knees with terminal OA (KL-4) and total knee replacements at baseline.

Results

The model successfully predicts OA progression, substantially outperforming traditional methodologies like logistic regression. The CNN-based approach yielded an area under the ROC curve (AUC) of 0.79 and an average precision (AP) of 0.68 on the test set, which marks a significant improvement over a reference logistic regression model that achieved an AUC of 0.75 and AP of 0.62. Further enhancement was observed when integrating CNN outputs with clinical data through a GBM-based fusion, which elevated the overall AUC to 0.80 and AP to 0.70. Attention maps generated using GradCAM suggested that the model captures clinically interpretable features beyond KL grading, such as tibial spine characteristics, indicating potentially valuable visual cues associated with disease progression.

Implications

The implications of this research are multifaceted:

1. Clinical Implementation: The approach facilitates automatic, radiologist-independent progression predictions, potentially extending to primary healthcare settings where resources are limited. This opens the opportunity for early interventions through low-cost predictive tools.
2. Drug Development: By enhancing subject selection through precise progression predictions, the model can accelerate disease-modifying drug trials, thereby refining therapeutic strategies.
3. Technological Advancements: The research underscores the capacity of deep learning models to process multimodal data, advancing predictive analytics in medical imaging. The model's ability to handle missing data without imputation marks a significant technical achievement.
4. Interdisciplinary Research: As machine learning models become less transparent, tackling this challenge through interpretable models (e.g., GradCAM) becomes increasingly critical, suggesting a pathway for further investigation.

Future Directions

Although the model demonstrates robust predictive capabilities, further validation across diverse populations and imaging protocols is essential. Moreover, incorporating symptomatic data into progression metrics could refine model outputs, addressing limitations in current progression definitions. Prospective studies should also consider diverse imaging techniques and settings to broaden applicability, leveraging the models' ability to generalize across varying data conditions.

In conclusion, the study makes significant strides in OA predictive modeling by integrating multimodal data and state-of-the-art machine learning techniques. By addressing existing challenges in prognostic evaluations, this research forms the basis for enhanced clinical applications, contributing to personalized medicine in musculoskeletal disorders.