Atmospheric Turbulence Removal with Video Sequence Deep Visual Priors
Abstract: Atmospheric turbulence poses a challenge for the interpretation and visual perception of visual imagery due to its distortion effects. Model-based approaches have been used to address this, but such methods often suffer from artefacts associated with moving content. Conversely, deep learning based methods are dependent on large and diverse datasets that may not effectively represent any specific content. In this paper, we address these problems with a self-supervised learning method that does not require ground truth. The proposed method is not dependent on any dataset outside of the single data sequence being processed but is also able to improve the quality of any input raw sequences or pre-processed sequences. Specifically, our method is based on an accelerated Deep Image Prior (DIP), but integrates temporal information using pixel shuffling and a temporal sliding window. This efficiently learns spatio-temporal priors leading to a system that effectively mitigates atmospheric turbulence distortions. The experiments show that our method improves visual quality results qualitatively and quantitatively.
- “Online blind deconvolution for astronomical imaging,” in 2009 IEEE International Conference on Computational Photography (ICCP). IEEE, 2009, pp. 1–7.
- “Mitigating the effects of atmospheric distortion using dt-cwt fusion,” in 2012 19th IEEE International Conference on Image Processing. IEEE, 2012, pp. 3033–3036.
- “Single frame atmospheric turbulence mitigation: A benchmark study and a new physics-inspired transformer model,” in European Conference on Computer Vision. Springer, 2022, pp. 430–446.
- “Real-time dense field phase-to-space simulation of imaging through atmospheric turbulence,” IEEE Transactions on Computational Imaging, vol. 8, pp. 1159–1169, 2022.
- “Accelerating atmospheric turbulence simulation via learned phase-to-space transform,” in ICCV, 2021.
- “Atmospheric turbulence mitigation for sequences with moving objects using recursive image fusion,” in 2018 25th IEEE international conference on image processing (ICIP). IEEE, 2018, pp. 2895–2899.
- “Learning to generate atmospheric turbulent images,” in Computer Vision, Pattern Recognition, Image Processing, and Graphics. NCVPRIPG 2019, 2020.
- “Atmospheric turbulence removal using convolutional neural network,” in arXiv:1912.11350, 2019.
- Nantheera Anantrasirichai, “Atmospheric turbulence removal with complex-valued convolutional neural network,” Pattern Recognition Letters, vol. 171, pp. 69–75, 2023.
- “Analysis of deep learning architectures for turbulence mitigation in long-range imagery,” in Artificial Intelligence and Machine Learning in Defense Applications II. SPIE, 2020, vol. 11543, p. 1154303.
- “Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising,” IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142–3155, 2017.
- “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. 2015, pp. 234–241, Springer.
- “Deep image prior,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 9446–9454.
- “Deep random projector: Accelerated deep image prior,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 18176–18185.
- “Early stopping for deep image prior,” arXiv preprint arXiv:2112.06074, 2021.
- “Deep image prior,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017.
- “Self-validation: Early stopping for single-instance deep generative priors,” arXiv preprint arXiv:2110.12271, 2021.
- “Analyzing and improving the image quality of stylegan,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 8110–8119.
- “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1874–1883.
- “A two-step framework for constructing blind image quality indices,” Signal Processing Letters, IEEE, vol. 17, no. 5, pp. 513 –516, May 2010.
- “No-reference quality assessment using natural scene statistics: Jpeg2000,” IEEE Transactions on image processing, vol. 14, no. 11, pp. 1918–1927, 2005.
- “Blind image quality assessment through anisotropy,” J. Opt. Soc. Am, vol. 24, no. 12, pp. B42–B51, 2007.
- “Atmospheric turbulence mitigation using complex wavelet-based fusion,” IEEE Transactions on Image Processing, vol. 22, no. 6, pp. 2398–2408, 2013.
- “Restoration of atmospheric turbulence-degraded short-exposure image based on convolution neural network,” Photonics, vol. 10, no. 6, 2023.
- “At-ddpm: Restoring faces degraded by atmospheric turbulence using denoising diffusion probabilistic models,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 3434–3443.
- “Neutralizing the impact of atmospheric turbulence on complex scene imaging via deep learning,” Nature Machine Intelligence, vol. 3, pp. 876 – 884, 2021.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.