Deep-learning-driven end-to-end metalens imaging

Abstract: Recent advances in metasurface lenses (metalenses) have shown great potential for opening a new era in compact imaging, photography, light detection and ranging (LiDAR), and virtual reality/augmented reality (VR/AR) applications. However, the fundamental trade-off between broadband focusing efficiency and operating bandwidth limits the performance of broadband metalenses, resulting in chromatic aberration, angular aberration, and a relatively low efficiency. In this study, a deep-learning-based image restoration framework is proposed to overcome these limitations and realize end-to-end metalens imaging, thereby achieving aberration-free full-color imaging for mass-produced metalenses with 10-mm diameter. Neural-network-assisted metalens imaging achieved a high resolution comparable to that of the ground truth image.

• The paper introduces a deep-learning framework that corrects chromatic and angular aberrations in mass-produced metalens systems.
• It integrates advanced fabrication techniques with DNN-based methods using positional embedding and Fourier domain adversarial learning.
• Experimental results show significant improvements in metrics such as PSNR, SSIM, and object detection accuracy for practical optical applications.

Deep-learning-driven End-to-End Metalens Imaging

Introduction

The pursuit of compact and high-performance optical imaging systems has seen significant advances in recent years, driven particularly by the development of metasurface lenses, known as metalenses. These devices aim to overcome inherent limitations in traditional lenses, including chromatic and spherical aberrations, bulkiness, shadowing effects, and high manufacturing costs. Metalenses, characterized by their ultrathin films and subwavelength, meta-atom structures, promise a revolutionary alternative with enhanced capabilities for a variety of applications, such as LiDAR, VR/AR, and smartphone cameras.

Despite these advantages, metalenses face significant challenges related to chromatic and angular aberrations and focusing efficiency, which impede their commercialization. The study proposes an innovative, deep-learning-based image restoration framework to address these limitations, supporting aberration-free, full-color imaging for mass-produced metalenses. This approach not only rectifies the aberration and inefficiency issues but also leverages the strength of deep learning, thus heralding a new era in optical imaging systems.

Figure 1: Schematic of our metalens imaging.

Metalens Imaging System

The proposed metallens imaging system integrates advanced fabrication techniques, including nanoimprint lithography and atomic layer deposition, to produce consistent, low-cost metalenses at scale. Despite achieving diffraction-limited focusing, these lenses suffer from chromatic and angular aberrations, necessitating a tailor-made solution for image restoration. The system makes use of photonic structures optimized to address common lens drawbacks like dispersion-induced aberrations.

The system was evaluated based on the physical and optical properties of mass-produced metalenses, as depicted in various figures, showcasing different aspects from focal length dynamics across wavelengths to point spread function analyses. By integrating DNN-based image restoration with the metalens framework, the system promises to enhance the image quality by reducing chromatic and angular aberrations, paving the way for high-resolution imaging applications.

Figure 2: (a) Detailed structure and analysis of mass-produced metalenses and (b-g) their optical properties.

Image Restoration Network

DNN-based image restoration techniques represent significant advancements over classical methods, particularly in handling complex degradations in images. Conventional techniques, such as the Wiener filter, fall short in addressing metalens-related issues due to their sensitivity to position-dependent aberrations. This study introduces a deep-learning framework utilizing adversarial learning and strong regularization techniques, particularly in the Fourier domain, to successfully restore images captured by metalenses.

Figure 3: Proposed image restoration framework with adversarial learning and position embedding innovations.

The network architecture of this framework incorporates patch cropping, positional embedding, and adversarial learning in Fourier space to recover lost spatial-frequency information—critical for detailed image restoration. By embedding positional information, the system efficiently deals with angular aberrations, ensuring superior image quality of restored metalens images.

Results

The results of employing the proposed framework demonstrate substantial improvements in image quality over traditional metalens outputs. Metrics like PSNR, SSIM, and LPIPS unveiled significant advancements, underscoring the effectiveness of integrating deep learning with optical system designs. The framework outperforms established image restoration models, asserting its applicability for the metalens image restoration task.

Figure 4: Comparative restoration results showcase the enhanced resolution and reduced aberrations achieved by the proposed framework.

Furthermore, quality improvements are validated through statistical tests and visual assessments. The advances in spatial-frequency restoration indicate promising applications, enabling commercial viability for metalenses by liberating them from their inherent aberration constraints.

Object Detection Performance

In practical applications such as object detection, the restored images exhibited remarkable improvements. Utilizing pre-trained object detection models like SSD, the restored images demonstrated enhanced accuracy in object localization, indicating readiness for real-world applications such as autonomous navigation and surveillance systems.

Figure 5: Comparative statistical analysis illustrating the framework's superior performance across multiple image quality metrics.

The framework's ability to enhance detection precision and recall underscores its value in critical areas, ensuring reliable performance in demanding environments that rely on optical imaging.

Figure 6: Demonstration of restored USAF test chart images showcasing improved clarity and color fidelity.

Conclusion

The introduction of a deep-learning-driven image restoration system marks a significant advancement in the field of optical imaging, particularly within the field of mass-produced metalenses. By resolving chromatic and angular aberrations and integrating a deep-learning framework, the study proposes a viable pathway to transform imaging systems into compact, efficient entities suitable for diverse applications. The potential impact spans industrial and consumer fields, promising to accelerate innovations in optical design and applications.