Papers
Topics
Authors
Recent
Search
2000 character limit reached

CT to PET Translation: A Large-scale Dataset and Domain-Knowledge-Guided Diffusion Approach

Published 29 Oct 2024 in eess.IV and cs.CV | (2410.21932v1)

Abstract: Positron Emission Tomography (PET) and Computed Tomography (CT) are essential for diagnosing, staging, and monitoring various diseases, particularly cancer. Despite their importance, the use of PET/CT systems is limited by the necessity for radioactive materials, the scarcity of PET scanners, and the high cost associated with PET imaging. In contrast, CT scanners are more widely available and significantly less expensive. In response to these challenges, our study addresses the issue of generating PET images from CT images, aiming to reduce both the medical examination cost and the associated health risks for patients. Our contributions are twofold: First, we introduce a conditional diffusion model named CPDM, which, to our knowledge, is one of the initial attempts to employ a diffusion model for translating from CT to PET images. Second, we provide the largest CT-PET dataset to date, comprising 2,028,628 paired CT-PET images, which facilitates the training and evaluation of CT-to-PET translation models. For the CPDM model, we incorporate domain knowledge to develop two conditional maps: the Attention map and the Attenuation map. The former helps the diffusion process focus on areas of interest, while the latter improves PET data correction and ensures accurate diagnostic information. Experimental evaluations across various benchmarks demonstrate that CPDM surpasses existing methods in generating high-quality PET images in terms of multiple metrics. The source code and data samples are available at https://github.com/thanhhff/CPDM.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Accuracy of CT-based attenuation correction in PET/CT bone imaging. Physics in Medicine & Biology, 57(9):2477, 2012.
  2. Radiobiological risks in terms of effective dose and organ dose from 18f-fdg whole-body pet/ct procedures. Saudi Journal of Biological Sciences, 28:5947–5951, 2021.
  3. Medgan: Medical image translation using GANs. Computerized Medical Imaging and Graphics, 79:101684, 2020.
  4. Virtual PET Images from CT Data Using Deep Convolutional Networks: Initial Results, page 49–57. Springer International Publishing, 2017.
  5. Cross-modality synthesis from ct to pet using fcn and gan networks for improved automated lesion detection, 2018.
  6. Total-body PET: maximizing sensitivity to create new opportunities for clinical research and patient care. Journal of Nuclear Medicine, 59(1):3–12, 2018.
  7. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  8. Diffusion models beat GANs on image synthesis. Advances in Neural Information Processing Systems, 34:8780–8794, 2021.
  9. George H Dunteman. Principal components analysis, volume 69. Sage, 1989.
  10. Taming transformers for high-resolution image synthesis. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12873–12883, 2021.
  11. A whole-body fdg-pet/ct dataset with manually annotated tumor lesions (fdg-pet-ct-lesions), 2022. Dataset.
  12. Generative adversarial nets. Advances in Neural Information Processing Systems, 27, 2014.
  13. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  14. Image-to-image translation with conditional adversarial networks. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, pages 1125–1134, 2017.
  15. Adam: A method for stochastic optimization. Computing Research Repository arXiv Preprints arXiv:1412.6980, 2014.
  16. Breaking the dilemma of medical image-to-image translation. Advances in Neural Information Processing Systems, 34:1964–1978, 2021.
  17. Drit++: Diverse image-to-image translation via disentangled representations, 2019.
  18. BBDM: Image-to-image translation with brownian bridge diffusion models. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1952–1961, 2023.
  19. A large-scale ct and pet/ct dataset for lung cancer diagnosis (lung-pet-ct-dx), 2020. Data set.
  20. Leveraging GAN-based CBCT-to-CT translation models for enhanced image quality and accurate photon and proton dose calculation in adaptive radiotherapy. Journal of Radiation Research and Applied Sciences, 17(1):100809, 2024.
  21. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073, 2021.
  22. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In Proceedings of the 4th International Conference on 3D Vision, pages 565–571, 2016.
  23. Data from rider lung pet-ct, 2015. Dataset.
  24. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.
  25. U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 234–241, 2015.
  26. Palette: Image-to-image diffusion models. In Proceedings of the 2022 ACM Special Interest Group on Computer Graphics and Interactive Techniques Conference, pages 1–10, 2022.
  27. Denoising diffusion implicit models. Computing Research Repository arXiv Preprints arXiv:2010.02502, 2020.
  28. Dual diffusion implicit bridges for image-to-image translation. Computing Research Repository arXiv Preprints arXiv:2203.08382, 2022.
  29. Multi-channel attention selection GAN with cascaded semantic guidance for cross-view image translation. In Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2417–2426, 2019.
  30. Uvcgan: Unet vision transformer cycle-consistent GAN for unpaired image-to-image translation. In Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision, pages 702–712, 2023.
  31. Uncertainty-guided progressive GANs for medical image translation. In Proceedings of the 24th Internation Conference on Medical Image Computing and Computer Assisted Intervention, pages 614–624, 2021.
  32. Data from head-neck-pet-ct, 2017. Dataset.
  33. Changing role of PET/CT in cancer care with a focus on radiotherapy. Cureus, 14(12), 2022.
  34. Spatial-intensity transforms for medical image-to-image translation. IEEE Transactions on Medical Imaging, 2023.
  35. High-resolution image synthesis and semantic manipulation with conditional gans, 2018.
  36. Artificial intelligence for response evaluation with PET/CT. In Seminars in Nuclear Medicine, volume 51, pages 157–169, 2021.
  37. Mri cross-modality image-to-image translation. Scientific reports, 10(1):3753, 2020.
  38. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the 2017 IEEE International Conference on Computer Vision, pages 2223–2232, 2017.
  39. Cross-domain medical image translation by shared latent gaussian mixture model. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part II 23, pages 379–389. Springer, 2020.
  40. Unsupervised medical image translation with adversarial diffusion models, 2023.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up