Papers
Topics
Authors
Recent
Search
2000 character limit reached

Deep learning phase recovery: data-driven, physics-driven, or combining both?

Published 1 Apr 2024 in eess.IV, cs.LG, and physics.optics | (2404.01360v2)

Abstract: Phase recovery, calculating the phase of a light wave from its intensity measurements, is essential for various applications, such as coherent diffraction imaging, adaptive optics, and biomedical imaging. It enables the reconstruction of an object's refractive index distribution or topography as well as the correction of imaging system aberrations. In recent years, deep learning has been proven to be highly effective in addressing phase recovery problems. Two most direct deep learning phase recovery strategies are data-driven (DD) with supervised learning mode and physics-driven (PD) with self-supervised learning mode. DD and PD achieve the same goal in different ways and lack the necessary study to reveal similarities and differences. Therefore, in this paper, we comprehensively compare these two deep learning phase recovery strategies in terms of time consumption, accuracy, generalization ability, ill-posedness adaptability, and prior capacity. What's more, we propose a co-driven (CD) strategy of combining datasets and physics for the balance of high- and low-frequency information. The codes for DD, PD, and CD are publicly available at https://github.com/kqwang/DLPR.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. K. Wang, L. Song, C. Wang, et al., “On the use of deep learning for phase recovery,” Light: Science & Applications 13(1), 4 (2024). [doi:10.1038/s41377-023-01340-x].
  2. Y. Park, C. Depeursinge, and G. Popescu, “Quantitative phase imaging in biomedicine,” Nature Photonics 12(10), 578–589 (2018). [doi:10.1038/s41566-018-0253-x].
  3. J. Miao, P. Charalambous, J. Kirz, et al., “Extending the methodology of X-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens,” Nature 400(6742), 342–344 (1999). [doi:10.1038/22498].
  4. M. V. Klibanov, P. E. Sacks, and A. V. Tikhonravov, “The phase retrieval problem,” Inverse Problems 11(1), 1–28 (1995). [doi:10.1088/0266-5611/11/1/001].
  5. D. Gabor, “A New Microscopic Principle,” Nature 161, 777–778 (1948). [doi:10.1038/161777a0].
  6. J. Hartmann, “Bermerkungen über den bau und die justierung von spektrographen,” Zeitschrift für Instrumentenkunde 20, 47–58 (1900).
  7. R. V. Shack and B. C. Platt, “Production and use of a lenticular Hartmann screen,” Journal of the Optical Society of America 61, 656–661 (1971).
  8. M. R. Teague, “Deterministic phase retrieval: A Green’s function solution,” Journal of the Optical Society of America 73(11), 1434–1441 (1983). [doi:10.1364/JOSA.73.001434].
  9. C. Zuo, J. Li, J. Sun, et al., “Transport of intensity equation: A tutorial,” Optics and Lasers in Engineering 135, 106187 (2020). [doi:10.1016/j.optlaseng.2020.106187].
  10. G. Barbastathis, A. Ozcan, and G. Situ, “On the use of deep learning for computational imaging,” Optica 6(8), 921–943 (2019). [doi:10.1364/OPTICA.6.000921].
  11. A. Sinha, J. Lee, S. Li, et al., “Lensless computational imaging through deep learning,” Optica 4(9), 1117–1125 (2017). [doi:10.1364/OPTICA.4.001117].
  12. H. Wang, M. Lyu, and G. H. Situ, “eHoloNet: A learning-based end-to-end approach for in-line digital holographic reconstruction,” Optics Express 26(18), 22603–22614 (2018). [doi:10.1364/OE.26.022603].
  13. M. J. Cherukara, Y. S. G. Nashed, and R. J. Harder, “Real-time coherent diffraction inversion using deep generative networks,” Scientific Reports 8(1), 16520 (2018). [doi:10.1038/s41598-018-34525-1].
  14. T. Nguyen, Y. Xue, Y. Li, et al., “Deep learning approach for Fourier ptychography microscopy,” Optics Express 26(20), 26470–26484 (2018). [doi:10.1364/OE.26.026470].
  15. Z. Ren, Z. M. Xu, and E. Y. Lam, “End-to-end deep learning framework for digital holographic reconstruction,” Advanced Photonics 1(01), 016004 (2019). [doi:10.1117/1.AP.1.1.016004].
  16. L. Hu, S. Hu, W. Gong, et al., “Deep learning assisted Shack–Hartmann wavefront sensor for direct wavefront detection,” Optics Letters 45(13), 3741–3744 (2020). [doi:10.1364/OL.395579].
  17. K. Wang, J. Di, Y. Li, et al., “Transport of intensity equation from a single intensity image via deep learning,” Optics and Lasers in Engineering 134, 106233 (2020). [doi:10.1016/j.optlaseng.2020.106233].
  18. D. Pirone, D. Sirico, L. Miccio, et al., “Speeding up reconstruction of 3D tomograms in holographic flow cytometry via deep learning,” Lab on a Chip 22(4), 793–804 (2022). [doi:10.1039/D1LC01087E].
  19. D. J. Chang, C. M. O’Leary, C. Su, et al., “Deep-Learning Electron Diffractive Imaging,” Physical Review Letters 130(1), 016101 (2023). [doi:10.1103/PhysRevLett.130.016101].
  20. Y. Xue, S. Cheng, Y. Li, et al., “Reliable deep-learning-based phase imaging with uncertainty quantification,” Optica 6(5), 618–629 (2019). [doi:10.1364/OPTICA.6.000618].
  21. X. Li, H. Qi, S. Jiang, et al., “Quantitative phase imaging via a cGAN network with dual intensity images captured under centrosymmetric illumination,” Optics Letters 44(11), 2879–2882 (2019). [doi:10.1364/OL.44.002879].
  22. K. Wang, J. Dou, Q. Kemao, et al., “Y-Net: A one-to-two deep learning framework for digital holographic reconstruction,” Optics Letters 44(19), 4765–4768 (2019). [doi:10.1364/OL.44.004765].
  23. K. Wang, Q. Kemao, J. Di, et al., “Y4-Net: A deep learning solution to one-shot dual-wavelength digital holographic reconstruction,” Optics Letters 45(15), 4220–4223 (2020). [doi:10.1364/OL.395445].
  24. T. Zeng, H. K. H. So, and E. Y. Lam, “RedCap: Residual encoder-decoder capsule network for holographic image reconstruction,” Optics Express 28(4), 4876–4887 (2020). [doi:10.1364/OE.383350].
  25. L. Huang, T. Liu, X. Yang, et al., “Holographic Image Reconstruction with Phase Recovery and Autofocusing Using Recurrent Neural Networks,” ACS Photonics 8(6), 1763–1774 (2021). [doi:10.1021/acsphotonics.1c00337].
  26. H. Chen, L. Huang, T. Liu, et al., “Fourier Imager Network (FIN): A deep neural network for hologram reconstruction with superior external generalization,” Light: Science & Applications 11(1), 254 (2022). [doi:10.1038/s41377-022-00949-8].
  27. H. Chen, L. Huang, T. Liu, et al., “eFIN: Enhanced Fourier Imager Network for Generalizable Autofocusing and Pixel Super-Resolution in Holographic Imaging,” IEEE Journal of Selected Topics in Quantum Electronics 29, 6800810 (2023). [doi:10.1109/JSTQE.2023.3248684].
  28. X. Shu, M. Niu, Y. Zhang, et al., “NAS-PRNet: Neural Architecture Search generated Phase Retrieval Net for Off-axis Quantitative Phase Imaging,” arXiv preprint arXiv:2210.14231 (2022). [doi:10.48550/arXiv.2210.14231].
  29. Z. Ren, Z. M. Xu, and E. Y. Lam, “Learning-based nonparametric autofocusing for digital holography,” Optica 5(4), 337–344 (2018). [doi:10.1364/OPTICA.5.000337].
  30. Z. Ren, H. K. H. So, and E. Y. Lam, “Fringe Pattern Improvement and Super-Resolution Using Deep Learning in Digital Holography,” IEEE Transactions on Industrial Informatics 15(11), 6179–6186 (2019). [doi:10.1109/TII.2019.2913853].
  31. K. Wang, Y. Li, Q. Kemao, et al., “One-step robust deep learning phase unwrapping,” Optics Express 27(10), 15100–15115 (2019). [doi:10.1364/OE.27.015100].
  32. Y. Zhu, C. H. Yeung, and E. Y. Lam, “Digital holographic imaging and classification of microplastics using deep transfer learning,” Applied Optics 60(4), A38 (2021). [doi:10.1364/AO.403366].
  33. Y. Zhu, C. H. Yeung, and E. Y. Lam, “Microplastic pollution monitoring with holographic classification and deep learning,” Journal of Physics: Photonics 3(2), 024013 (2021). [doi:110.1088/2515-7647/abf250].
  34. L. Boominathan, M. Maniparambil, H. Gupta, et al., “Phase retrieval for Fourier Ptychography under varying amount of measurements,” arXiv preprint arXiv:1805.03593 (2018). [doi:10.48550/arXiv.1805.03593].
  35. F. Wang, Y. Bian, H. Wang, et al., “Phase imaging with an untrained neural network,” Light: Science & Applications 9(1), 77 (2020). [doi:10.1038/s41377-020-0302-3].
  36. X. Zhang, F. Wang, and G. H. Situ, “BlindNet: An untrained learning approach toward computational imaging with model uncertainty,” Journal of Physics D: Applied Physics 55(3), 034001 (2022). [doi:10.1088/1361-6463/ac2ad4].
  37. C. Bai, T. Peng, J. Min, et al., “Dual-wavelength in-line digital holography with untrained deep neural networks,” Photonics Research 9(12), 2501 (2021). [doi:10.1364/PRJ.441054].
  38. D. Yang, J. Zhang, Y. Tao, et al., “Dynamic coherent diffractive imaging with a physics-driven untrained learning method,” Optics Express 29(20), 31426–31442 (2021). [doi:10.1364/OE.433507].
  39. D. Yang, J. Zhang, Y. Tao, et al., “Coherent modulation imaging using a physics-driven neural network,” Optics Express 30(20), 35647–35662 (2022). [doi:10.1364/OE.472083].
  40. L. Bouchama, B. Dorizzi, J. Klossa, et al., “A Physics-Inspired Deep Learning Framework for an Efficient Fourier Ptychographic Microscopy Reconstruction under Low Overlap Conditions,” Sensors 23(15), 6829 (2023). [doi:10.3390/s23156829].
  41. L. Huang, H. Chen, T. Liu, et al., “Self-supervised learning of hologram reconstruction using physics consistency,” Nature Machine Intelligence 5(8), 895–907 (2023). [doi:10.1038/s42256-023-00704-7].
  42. O. Hoidn, A. A. Mishra, and A. Mehta, “Physics constrained unsupervised deep learning for rapid, high resolution scanning coherent diffraction reconstruction,” Scientific Reports 13(1), 22789 (2023). [doi:10.1038/s41598-023-48351-7].
  43. Y. Yao, H. Chan, S. Sankaranarayanan, et al., “AutoPhaseNN: Unsupervised physics-aware deep learning of 3D nanoscale Bragg coherent diffraction imaging,” npj Computational Materials 8(1), 124 (2022). [doi:10.1038/s41524-022-00803-w].
  44. R. Li, G. Pedrini, Z. Huang, et al., “Physics-enhanced neural network for phase retrieval from two diffraction patterns,” Optics Express 30(18), 32680–32692 (2022). [doi:10.1364/OE.469080].
  45. K. Wang, Q. Kemao, J. Di, et al., “Deep learning spatial phase unwrapping: A comparative review,” Advanced Photonics Nexus 1(1), 014001 (2022). [doi:10.1117/1.APN.1.1.014001].
  46. Z.-Q. J. Xu, Y. Zhang, T. Luo, et al., “Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks,” Communications in Computational Physics 28(5), 1746–1767 (2020). [doi:10.4208/cicp.OA-2020-0085].
  47. Y. Gao and L. Cao, “Iterative projection meets sparsity regularization: Towards practical single-shot quantitative phase imaging with in-line holography,” Light: Advanced Manufacturing 4(1), 1 (2023). [doi:10.37188/lam.2023.006].
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (2)

  1. Kaiqiang Wang 
  2. Edmund Y. Lam 

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

GitHub

  1. GitHub - kqwang/DLPR (41 stars)  

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free