Papers
Topics
Authors
Recent
Search
2000 character limit reached

TAI-GAN: A Temporally and Anatomically Informed Generative Adversarial Network for early-to-late frame conversion in dynamic cardiac PET inter-frame motion correction

Published 14 Feb 2024 in eess.IV and cs.CV | (2402.09567v1)

Abstract: Inter-frame motion in dynamic cardiac positron emission tomography (PET) using rubidium-82 (82-Rb) myocardial perfusion imaging impacts myocardial blood flow (MBF) quantification and the diagnosis accuracy of coronary artery diseases. However, the high cross-frame distribution variation due to rapid tracer kinetics poses a considerable challenge for inter-frame motion correction, especially for early frames where intensity-based image registration techniques often fail. To address this issue, we propose a novel method called Temporally and Anatomically Informed Generative Adversarial Network (TAI-GAN) that utilizes an all-to-one mapping to convert early frames into those with tracer distribution similar to the last reference frame. The TAI-GAN consists of a feature-wise linear modulation layer that encodes channel-wise parameters generated from temporal information and rough cardiac segmentation masks with local shifts that serve as anatomical information. Our proposed method was evaluated on a clinical 82-Rb PET dataset, and the results show that our TAI-GAN can produce converted early frames with high image quality, comparable to the real reference frames. After TAI-GAN conversion, the motion estimation accuracy and subsequent myocardial blood flow (MBF) quantification with both conventional and deep learning-based motion correction methods were improved compared to using the original frames.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Paired-unpaired unsupervised attention guided gan with transfer learning for bidirectional brain mr-ct synthesis. Computers in Biology and Medicine 136, 104763.
  2. Semantically consistent hierarchical text to fashion image synthesis with an enhanced-attentional generative adversarial network, in: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), IEEE. pp. 3121–3124.
  3. Semantically consistent text to fashion image synthesis with an enhanced attentional generative adversarial network. Pattern Recognition Letters 135, 22–29.
  4. Cardiac positron emission tomography: Overview of myocardial perfusion, myocardial blood flow and coronary flow reserve imaging. Mol. Imag .
  5. Region-adaptive deformable registration of ct/mri pelvic images via learning-based image synthesis. IEEE Transactions on Image Processing 27, 3500–3512.
  6. Cross contrast multi-channel image registration using image synthesis for mr brain images. Medical image analysis 36, 2–14.
  7. Neural ordinary differential equations. Advances in neural information processing systems 31.
  8. Direct and indirect strategies of deep-learning-based attenuation correction for general purpose and dedicated cardiac spect. European Journal of Nuclear Medicine and Molecular Imaging 49, 3046–3060.
  9. 3d u-net: learning dense volumetric segmentation from sparse annotation, in: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19, Springer. pp. 424–432.
  10. The permeability of capillaries in various organs as determined by use of the ‘indicator diffusion’method. Acta physiologica scandinavica 58, 292–305.
  11. Image synthesis in multi-contrast mri with conditional generative adversarial networks. IEEE transactions on medical imaging 38, 2375–2388.
  12. Generative adversarial registration for improved conditional deformable templates, in: Proceedings of the IEEE/CVF international conference on computer vision, pp. 3929–3941.
  13. Identifying autism from resting-state fmri using long short-term memory networks, in: International Workshop on Machine Learning in Medical Imaging, Springer. pp. 362–370.
  14. Self-gating: an adaptive center-of-mass approach for respiratory gating in pet. IEEE transactions on medical imaging 37, 1140–1148.
  15. Corridor4dm: the michigan method for quantitative nuclear cardiology. Journal of nuclear cardiology 14, 455–465.
  16. Quantification of myocardial blood flow with 82 rb: Validation with 15 o-water using time-of-flight and point-spread-function modeling. EJNMMI research 6, 1–12.
  17. Generative adversarial networks. Communications of the ACM 63, 139–144.
  18. Seam-stress: A weakly supervised framework for interstitial lung disease segmentation in chest ct, in: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), IEEE. pp. 1–4.
  19. Tai-gan: Temporally and anatomically informed gan for early-to-late frame conversion in dynamic cardiac pet motion correction.
  20. Characterization of early stage parkinson’s disease from resting-state fmri data using a long short-term memory network. Frontiers in Neuroimaging 1, 952084.
  21. Inter-pass motion correction for whole-body dynamic pet and parametric imaging. IEEE Transactions on Radiation and Plasma Medical Sciences 7, 344–353.
  22. Mcp-net: Introducing patlak loss optimization to whole-body dynamic pet inter-frame motion correction. IEEE Transactions on Medical Imaging .
  23. Mcp-net: Inter-frame motion correction with patlak regularization for whole-body dynamic pet, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part IV, Springer. pp. 163–172.
  24. Unsupervised inter-frame motion correction for whole-body dynamic pet using convolutional long short-term memory in a convolutional neural network. Medical Image Analysis 80, 102524. doi:10.1016/j.media.2022.102524.
  25. Deep learning based respiratory pattern classification and applications in pet/ct motion correction, in: 2019 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), IEEE. pp. 1–5.
  26. Denoising diffusion probabilistic models. Advances in neural information processing systems 33, 6840–6851.
  27. Long short-term memory. Neural computation 9, 1735–1780.
  28. Patient motion effects on the quantification of regional myocardial blood flow with dynamic pet imaging. Medical physics 43, 1829–1840.
  29. Is synthesizing mri contrast useful for inter-modality analysis?, in: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2013: 16th International Conference, Nagoya, Japan, September 22-26, 2013, Proceedings, Part I 16, Springer. pp. 631–638.
  30. Image-to-image translation with conditional adversarial networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1125–1134.
  31. Spatio-temporal pharmacokinetic model based registration of 4d pet neuroimaging data. Neuroimage 84, 225–235.
  32. Unified framework for development, deployment and robust testing of neuroimaging algorithms. Neuroinformatics 9, 69–84.
  33. Three-dimensional self-attention conditional gan with spectral normalization for multimodal neuroimaging synthesis. Magnetic resonance in medicine 86, 1718–1733.
  34. Image-based motion correction of the blood pool phase of dynamic pet data using blood pool isolation.
  35. Automated dynamic motion correction using normalized gradient fields for 82 rubidium pet myocardial blood flow quantification. Journal of Nuclear Cardiology 27, 1982–1998.
  36. Mri-only based synthetic ct generation using dense cycle consistent generative adversarial networks. Medical physics 46, 3565–3581.
  37. Small animal pet to ct image synthesis based on conditional generation network, in: 2021 14th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), IEEE. pp. 1–6.
  38. Image synthesis-based multi-modal image registration framework by using deep fully convolutional networks. Medical & Biological Engineering & Computing 57, 1037–1048.
  39. Respiratory motion compensation for pet/ct with motion information derived from matched attenuation-corrected gated pet data. Journal of nuclear Medicine 59, 1480–1486.
  40. Data-driven voluntary body motion detection and non-rigid event-by-event correction for static and dynamic pet. Physics in Medicine & Biology 64, 065002.
  41. Patient motion correction for dynamic cardiac pet: Current status and challenges. Journal of Nuclear Cardiology 27, 1999–2002.
  42. Data-driven motion detection and event-by-event correction for brain pet: Comparison with vicra. Journal of Nuclear Medicine 61, 1397–1403.
  43. Bilinear representation for language-based image editing using conditional generative adversarial networks, in: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE. pp. 2047–2051.
  44. X-ray synthesis based on triangular mesh models using gpu-accelerated ray tracing for multi-modal breast image registration, in: Simulation and Synthesis in Medical Imaging: 6th International Workshop, SASHIMI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings 6, Springer. pp. 87–96.
  45. Generation of whole-body fdg parametric ki images from static pet images using deep learning. IEEE Transactions on Radiation and Plasma Medical Sciences .
  46. Off-line motion correction methods for multi-frame pet data. European journal of nuclear medicine and molecular imaging 36, 2002–2013.
  47. Repurposing the microsoft kinect for windows v2 for external head motion tracking for brain pet. Physics in Medicine & Biology 60, 8753.
  48. Film: Visual reasoning with a general conditioning layer, in: Proceedings of the AAAI Conference on Artificial Intelligence.
  49. Quantification of myocardial blood flow with 82 rb positron emission tomography: clinical validation with 15 o-water. European journal of nuclear medicine and molecular imaging 39, 1037–1047.
  50. Predicting the evolution of white matter hyperintensities in brain mri using generative adversarial networks and irregularity map, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part III, Springer. pp. 146–154.
  51. Data-driven event-by-event respiratory motion correction using tof pet list-mode centroid of distribution. Physics in Medicine & Biology 62, 4741.
  52. Effects of blood flow on diffusion kinetics in isolated, perfused hindlegs of cats: a double circulation hypothesis. American Journal of Physiology-Legacy Content 183, 125–136.
  53. Transport of potassium-42 from blood to tissue in isolated mammalian skeletal muscles. American Journal of Physiology-Legacy Content 197, 1205–1210.
  54. Mr to ct registration of brains using image synthesis, in: Medical Imaging 2014: Image Processing, SPIE. pp. 307–314.
  55. Enhancing cardiac pet by motion correction techniques. Current cardiology reports 19, 1–8.
  56. Eanm procedural guidelines for pet/ct quantitative myocardial perfusion imaging. European Journal of Nuclear Medicine and Molecular Imaging 48, 1040–1069.
  57. Whole image synthesis using a deep encoder-decoder network, in: Simulation and Synthesis in Medical Imaging: First International Workshop, SASHIMI 2016, Held in Conjunction with MICCAI 2016, Athens, Greece, October 21, 2016, Proceedings 1, Springer. pp. 127–137.
  58. Automatic inter-frame patient motion correction for dynamic cardiac pet using deep learning. IEEE Transactions on Medical Imaging .
  59. Direct list mode parametric reconstruction for dynamic cardiac spect. IEEE transactions on medical imaging 39, 119–128.
  60. Deep learning-based attenuation map generation for myocardial perfusion spect. European Journal of Nuclear Medicine and Molecular Imaging 47, 2383–2395.
  61. A novel loss function incorporating imaging acquisition physics for pet attenuation map generation using deep learning, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part IV 22, Springer. pp. 723–731.
  62. Deep learning-based attenuation map generation with simultaneously reconstructed pet activity and attenuation and low-dose application. Physics in Medicine and Biology .
  63. An iterative image-based inter-frame motion compensation method for dynamic brain pet imaging. Physics in Medicine & Biology 67, 035012.
  64. Conditional generative adversarial networks aided motion correction of dynamic 18f-fdg pet brain studies. Journal of Nuclear Medicine 62, 871–879.
  65. Data-driven motion compensation using cgan for total-body [18f] fdg-pet imaging.
  66. Weakly-supervised deep learning of interstitial lung disease types on ct images, in: Medical Imaging 2019: Computer-Aided Diagnosis, SPIE. pp. 373–379.
  67. Generation of synthetic pet images of synaptic density and amyloid from 18f-fdg images using deep learning. Medical physics 48, 5115–5129.
  68. Automatic 3d registration of dynamic stress and rest 82rb and flurpiridaz f 18 myocardial perfusion pet data for patient motion detection and correction. Medical physics 38, 6313–6326.
  69. A review on 3d deformable image registration and its application in dose warping. Radiation Medicine and Protection 1, 171–178.
  70. 3d cgan based cross-modality mr image synthesis for brain tumor segmentation, in: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), IEEE. pp. 626–630.
  71. Mdpet: A unified motion correction and denoising adversarial network for low-dose gated pet. IEEE Transactions on Medical Imaging .
  72. Fast-mc-pet: A novel deep learning-aided motion correction and reconstruction framework for accelerated pet. arXiv preprint arXiv:2302.07135 .
  73. Hi-net: hybrid-fusion network for multi-modal mr image synthesis. IEEE transactions on medical imaging 39, 2772–2781.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up