Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models

Abstract: MRI and CT are most widely used medical imaging modalities. It is often necessary to acquire multi-modality images for diagnosis and treatment such as radiotherapy planning. However, multi-modality imaging is not only costly but also introduces misalignment between MRI and CT images. To address this challenge, computational conversion is a viable approach between MRI and CT images, especially from MRI to CT images. In this paper, we propose to use an emerging deep learning framework called diffusion and score-matching models in this context. Specifically, we adapt denoising diffusion probabilistic and score-matching models, use four different sampling strategies, and compare their performance metrics with that using a convolutional neural network and a generative adversarial network model. Our results show that the diffusion and score-matching models generate better synthetic CT images than the CNN and GAN models. Furthermore, we investigate the uncertainties associated with the diffusion and score-matching networks using the Monte-Carlo method, and improve the results by averaging their Monte-Carlo outputs. Our study suggests that diffusion and score-matching models are powerful to generate high quality images conditioned on an image obtained using a complementary imaging modality, analytically rigorous with clear explainability, and highly competitive with CNNs and GANs for image synthesis.

• The paper demonstrates that diffusion and score-matching models generate high-fidelity synthetic CT images from MRI data using advanced denoising and gradient-based techniques.
• The methodology employs iterative forward-reverse diffusion and SDE-based score matching to achieve superior SSIM and PSNR metrics compared to CNN and GAN baselines.
• The study quantifies uncertainty via Monte Carlo sampling, revealing trade-offs between image synthesis quality and computational speed across various sampling strategies.

Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models

Abstract and Contextual Relevance

Medical imaging modalities such as MRI and CT are indispensable for comprehensive diagnosis and treatment planning due to their complementary imaging capabilities. MRI is preferred for soft tissue visualization, whereas CT is crucial for imaging hard tissues and facilitating radiotherapy planning owing to its electron density mapping features. The logistical and technical intricacies involved in acquiring simultaneous multi-modality imaging – due to hardware costs and alignment issues – drive the need for computational image synthesis methods. This study leverages diffusion and score-matching models for synthetic CT image generation from MRI data, validating the algorithmic superiority of these models compared to traditional CNN and GAN approaches.

Methodological Advancements and Evaluation Metrics

Diffusion and Score-Matching Models

Diffusion Models such as DDPM employ an iterative forward and reverse process, wherein the forward process incrementally corrupts the original image using Gaussian noise, and the reverse process seeks to reconstruct the image using learned denoising dynamics. The reverse diffusion journey is tracked from Gaussian noise initialization to realistic image formation (Figure 1).

Figure 1: Reverse diffusion results using different sampling methods. The first row shows a T2w MR image as the condition and a CT image as the ground truth. The second to fifth rows present intermediate results in the DDPM, ODE, EM, and PC sampling processes respectively.

Score-Matching Models employ a stochastic differential equation (SDE) framework capturing data gradients across continuous time scales for noise and score prediction. Three advanced sampling techniques— Euler-Maruyama (EM), Predictor-Corrector (PC), and ODE methods— are adapted in this study, showcasing inherent differences in sampling efficiency and result fidelity.

Dataset Utilization

Utilizing the Gold Atlas male pelvis dataset comprising co-registered T2w MRI and CT pairs, the exploration involved training on 17 patients and testing on data from two distinct patients. All imaging data were scaled uniformly to facilitate robust learning and evaluation.

Comparative Evaluation

Uncertainty Quantification and Monte Carlo Sampling

The proposed models leverage Monte Carlo sampling to quantify associated uncertainties, aiming to enhance predictive robustness (Figure 2). The synthesis procedures demonstrated low standard deviations with the EM and PC methods showing advantageous consistency over DDPM in pixel-level evaluations.

Figure 2: Comparison of Monte Carlo sampling results using different reverse methods. The first five rows show the results conditioned on the same MR image using four different sampling strategies respectively. The bottom row presents results averaged over all ten MC sampling results and their pixel-wise standard deviation maps.

Resultant Image Quality Metrics

Assessments utilizing SSIM and PSNR metrics indicate that diffusion and score-matching models excel over CNN and GAN implementations, primarily concerning the avoidance of over-smoothing and artifact introduction (Figure 3). Diffusion models produce markedly realistic and structurally coherent synthetic CT images.

Figure 3: Comparison of two image synthesis results using different methods. For each example, the first row shows the whole image, and the second and third rows present the zoomed regions bounded by red and green boxes, respectively.

Statistical and Temporal Analysis

Evaluations (Figure 4) reveal the trade-offs between model fidelity and execution times across sampling strategies, endorsing EM as a preferred compromise between synthesis quality and computational efficiency.

Figure 4: Statistical comparison. (a) The average SSIM and PSNR scores of CT images synthesized using different methods, where the error bars show standard deviations; (b) the average model uncertainties of different sampling methods; and (c) the average times for sampling a slice using different sampling methods respectively.

Conclusion

Diffusion and score-matching models prove to be analytically rigorous and versatile for generating high-fidelity synthetic CT images from MRI datasets. Their principled mechanism allows for a competitive edge over traditional CNN/GAN models in predictive accuracy albeit slower sampling speeds. Future developments should focus on optimizing these models’ inference capabilities, potentially adopting alternative sampling acceleration methodologies to bridge existing computational transitions. The study underscores a pivotal advance in computational medical imaging, marrying diffusion principles with high-quality generative potential, and paves the way for more accessible multi-modal imaging solutions.