To deform or not: treatment-aware longitudinal registration for breast DCE-MRI during neoadjuvant chemotherapy via unsupervised keypoints detection
Abstract: Clinicians compare breast DCE-MRI after neoadjuvant chemotherapy (NAC) with pre-treatment scans to evaluate the response to NAC. Clinical evidence supports that accurate longitudinal deformable registration without deforming treated tumor regions is key to quantifying tumor changes. We propose a conditional pyramid registration network based on unsupervised keypoint detection and selective volume-preserving to quantify changes over time. In this approach, we extract the structural and the abnormal keypoints from DCE-MRI, apply the structural keypoints for the registration algorithm to restrict large deformation, and employ volume-preserving loss based on abnormal keypoints to keep the volume of the tumor unchanged after registration. We use a clinical dataset with 1630 MRI scans from 314 patients treated with NAC. The results demonstrate that our method registers with better performance and better volume preservation of the tumors. Furthermore, a local-global-combining biomarker based on the proposed method achieves high accuracy in pathological complete response (pCR) prediction, indicating that predictive information exists outside tumor regions. The biomarkers could potentially be used to avoid unnecessary surgeries for certain patients. It may be valuable for clinicians and/or computer systems to conduct follow-up tumor segmentation and response prediction on images registered by our method. Our code is available on \url{https://github.com/fiy2W/Treatment-aware-Longitudinal-Registration}.
- Cost function masking during normalization of brains with focal lesions: still a necessity? Neuroimage 53, 78–84.
- Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Medical image analysis 12, 26–41.
- Voxelmorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging 38, 1788–1800.
- Primary chemotherapy in operable breast cancer: eight-year experience at the milan cancer institute. Journal of Clinical Oncology 16, 93–100.
- Spatial normalization of brain images with focal lesions using cost function masking. Neuroimage 14, 486–500.
- Deformable image registration using a cue-aware deep regression network. IEEE Transactions on Biomedical Engineering 65, 1900–1911.
- Oncologic safety of breast-conserving surgery compared to mastectomy in patients receiving neoadjuvant chemotherapy for locally advanced breast cancer. Journal of surgical oncology 108, 531–536.
- De-escalating and escalating treatments for early-stage breast cancer: the st. gallen international expert consensus conference on the primary therapy of early breast cancer 2017. Annals of Oncology 28, 1700–1712.
- A deep learning framework for unsupervised affine and deformable image registration. Medical image analysis 52, 128–143.
- Neoadjuvant trials in early breast cancer: pathological response at surgery and correlation to longer term outcomes–what does it all mean? BMC medicine 13, 1–11.
- A prm approach for early prediction of breast cancer response to chemotherapy based on registered mr images. International journal of computer assisted radiology and surgery 13, 1233–1243.
- Pulmonary ct registration through supervised learning with convolutional neural networks. IEEE transactions on medical imaging 38, 1097–1105.
- Deep learning-based concurrent brain registration and tumor segmentation. Frontiers in computational neuroscience 14, 17.
- Motion correction in dual gated cardiac pet using mass-preserving image registration. IEEE transactions on medical imaging 31, 698–712.
- Deformable registration of glioma images using em algorithm and diffusion reaction modeling. IEEE transactions on medical imaging 30, 375–390.
- Glistr: glioma image segmentation and registration. IEEE transactions on medical imaging 31, 1941–1954.
- Multi-modal image registration with unsupervised deep learning. Ph.D. thesis. Massachusetts Institute of Technology.
- Deformable registration of brain mr images via a hybrid loss, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 141–146.
- Mrf-based deformable registration and ventilation estimation of lung ct. IEEE transactions on medical imaging 32, 1239–1248.
- Hypermorph: amortized hyperparameter learning for image registration, in: International Conference on Information Processing in Medical Imaging, Springer. pp. 3–17.
- Weakly-supervised convolutional neural networks for multimodal image registration. Medical image analysis 49, 1–13.
- Unsupervised learning of object landmarks through conditional image generation, in: Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 4020–4031.
- Elastix: a toolbox for intensity-based medical image registration. IEEE transactions on medical imaging 29, 196–205.
- Unsupervised learning of object keypoints for perception and control. arXiv preprint arXiv:1906.11883 .
- A nonrigid registration algorithm for longitudinal breast mr images and the analysis of breast tumor response. Magnetic resonance imaging 27, 1258–1270.
- Low-rank to the rescue–atlas-based analyses in the presence of pathologies, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 97–104.
- Low-rank atlas image analyses in the presence of pathologies. IEEE transactions on medical imaging 34, 2583–2591.
- Deformable registration for longitudinal breast mri screening. Journal of digital imaging 31, 718–726.
- Fast free-form deformation using graphics processing units. Computer methods and programs in biomedicine 98, 278–284.
- Conditional deformable image registration with convolutional neural network, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 35–45.
- A threshold selection method from gray-level histograms. IEEE transactions on systems, man, and cybernetics 9, 62–66.
- Dramms: Deformable registration via attribute matching and mutual-saliency weighting. Medical image analysis 15, 622–639.
- Deformable registration for quantifying longitudinal tumor changes during neoadjuvant chemotherapy. Magnetic resonance in medicine 73, 2343–2356.
- Volume-preserving nonrigid registration of mr breast images using free-form deformation with an incompressibility constraint. IEEE transactions on medical imaging 22, 730–741.
- Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: a cancer journal for clinicians 71, 209–249.
- Groupwise registration of mr brain images with tumors. Physics in Medicine & Biology 62, 6853.
- A new multi-atlas registration framework for multimodal pathological images using conventional monomodal normal atlases. IEEE Transactions on Image Processing 28, 2293–2304.
- Impact of deformable registration methods for prediction of recurrence free survival response to neoadjuvant chemotherapy in breast cancer: Results from the ispy 1/acrin 6657 trial. Translational Oncology 20, 101411.
- Diffeomorphic demons: Efficient non-parametric image registration. NeuroImage 45, S61–S72.
- Semi-supervised deep learning-based image registration method with volume penalty for real-time breast tumor bed localization. Sensors 21, 4085.
- Self-supervised learning of pixel-wise anatomical embeddings in radiological images. arXiv preprint arXiv:2012.02383 .
- A new criterion for automatic multilevel thresholding. IEEE Transactions on Image Processing 4, 370–378.
- Unsupervised discovery of object landmarks as structural representations, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2694–2703.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.