Single Pixel Imaging

• Single pixel imaging is an indirect imaging method that uses sequential modulation and a single-bucket detector to reconstruct images computationally.
• It employs diverse pattern bases like Hadamard, Fourier, and random subspaces, enabling applications from hyperspectral sensing to electron microscopy.
• Advances in reconstruction algorithms and deep learning enhance SPI performance, although speed, sampling limitations, and robustness challenges persist.

Single pixel imaging (SPI) is an indirect imaging paradigm in which a scene is sampled by sequentially projecting or detecting a series of known spatial, spectral, or spatiotemporal patterns, with a single-element “bucket” detector collecting only integrated intensity for each modulation. SPI leverages spatial light modulators (SLMs, DMDs, or physical masks) to encode optical or even non-optical signals, substituting for focal-plane arrays, and then computationally reconstructs high-dimensional images via inversion techniques and, increasingly, data-driven methods. Given its unique hardware–algorithm separation and ability to operate outside conventional sensor regimes, SPI has catalyzed advances from hyperspectral remote sensing to high-resolution microscopy, phase/amplitude reconstruction, photon-limited detection, deep learning–accelerated pipelines, and ultrafast electron modalities.

1. Fundamental Measurement Principles and Mathematical Models

SPI operates by modulating the illumination or detection with a sequence of spatial patterns (binary or grayscale) $a_m(x, y)$, collecting a scalar response from each with a bucket detector. The forward model for an N-pixel image $x \in \mathbb{R}^N$ is

$y_m = \sum_{j=1}^N a_{m, j} x_j + \eta_m$

with measurement vector $y \in \mathbb{R}^M$, sensing matrix $A \in \mathbb{R}^{M \times N}$, and noise $\eta$. SPI encompasses both structured illumination (DMD projects $a_m$; scene transmits/refects) and structured detection (scene imaged onto an SLM; mask modulates detection).

In photonic SPI, spatial or spectral patterns are projected by a DMD or SLM. In neutron SPI (He et al., 2020), fabricated physical amplitude masks modulate neutron fluence. In electron SPI (Konečná et al., 2022), optically shaped wavefronts are imposed on ultrafast electron pulses via PINEM, probing dynamic samples at the atomic scale. In single-pixel coherent diffraction imaging (SPI-CDI) (Li et al., 2020), binary patterns illuminate the object and only the DC component of the far-field diffraction is integrated—enabling recovery of both amplitude and phase.

2. Pattern Basis Design and Sampling Strategies

SPI performance is dictated by the choice of measurement basis and pattern sequencing:

• Orthonormal bases: Hadamard and Walsh functions provide maximal mutual incoherence and direct linear inversion, but require full sampling for optimal results.
• Fourier basis: Sinusoidal patterns sample spatial frequency space directly; conjugate symmetry and energy sparsity in natural images allow importance sampling (Bian et al., 2015Zhang et al., 2016). Complementary Fourier SPI (CFSI) uses DMD-complementary encoding to halve pattern counts per spectral coefficient while maintaining four-step differential noise robustness (Zhou et al., 2021).
• Random subspaces: Morlet wavelet–correlated patterns cover joint spatial-frequency features more efficiently than purely random or noiselet patterns, enabling high-fidelity reconstructions at compression ratios down to a few percent (Czajkowski et al., 2017).
• Hierarchical deterministic construction: Origami pattern design generates ordered orthogonal bases where early patterns contribute most to reconstruction, reducing required sampling ratios for real-time compressive video (Yu et al., 2019).
• Region-aware/differential binary maps: Full-resolution DMD SPI employs partitioned, lookup-table–encoded binary maps to separate active from empty image regions, supporting 1024×768 reconstruction at 0.4% sampling in 0.3 s (Stojek et al., 2022Pastuszczak et al., 1 Sep 2025).
• Physical amplitude masks: Direct fabrication in neutron (Gd₂O₃-on-Si) or THz (metal disk perforation) regimes (He et al., 2020Vallés et al., 2020) enables SPI where dynamic/pixelated SLMs are unavailable.

Compressed sensing further reduces required measurements provided the image is sparse or compressible in an appropriate transform domain (e.g., wavelets, DCT, TV).

3. Reconstruction Algorithms and Advances

SPI inversion techniques span a spectrum:

• Direct linear inversion: For orthonormal or circulant bases, simple pseudo-inverse or FFT-based inversion enables rapid reconstruction at full sampling (Bian et al., 2015Fedorov, 10 Apr 2025Zhang et al., 2016).
• Total variation (TV) and ℓ₁ minimization: Enforces piecewise smoothness or sparsity; solved by convex optimization (FISTA, ADMM, TVAL3) (Czajkowski et al., 2017Yu et al., 2019).
• Differential Fourier-domain regularized inversion (D-FDRI): Incorporates smoothness priors and efficient pre-computation to allow real-time high-resolution inverse (Stojek et al., 2022). In region-aware SPI, a two-stage pipeline leverages coarse region means for sparsity enhancement either iteratively or with deep neural nets (Pastuszczak et al., 1 Sep 2025).
• Phase retrieval: SPI-CDI uses a non-convex alternating projection algorithm to recover 2D amplitude and phase maps from intensity-only bucket measurements, exploiting the drastically reduced dynamic range of DC-only collection (Li et al., 2020).
• Compressed spectral inversion: In spectral-domain SPI, Fourier probe masks are encoded in the wavelength domain; coefficients are recovered either by direct differential subtraction or via convex sparse optimization (Ryczkowski et al., 2020).
• Instant ghost imaging (IGI): Real-time FPGA-based difference-and-accumulate algorithm eliminates post-acquisition delay by integrating image reconstruction into the measurement phase (Yang et al., 2020).
• Deep learning: CNNs (U-Net, autoencoders, GANs) learn non-linear mappings from measurement vectors to high-dimensional images, supporting video-rate, photon-limited, color, super-resolved, and advanced sensing tasks (classification, segmentation, encryption) (Song et al., 2023Zhao et al., 2021Pastuszczak et al., 1 Sep 2025). GAN-based SPI jointly optimizes binary measurement masks and neural inversion for maximum fidelity at low sampling rates.

Computational resource needs and latency depend on both pattern complexity and inversion algorithm; real-time performance is obtainable for sparse/deterministic sampling and deep learning approaches on modern hardware.

4. Spectral, Polarimetric, and Multimodal Extensions

SPI generalizes to non-visible and multidimensional modalities by adapting pattern basis and detection hardware:

• Spectral SPI: Broadband sources (SLED, DFG) are modulated with programmable optical filters or mask wheels, probing transmission/reflection spectra by encoding orthogonal or compressive Fourier masks in wavelength (Ryczkowski et al., 2020Vallés et al., 2020).
• Photon-counting and hyperspectral: Time-correlated single-photon counting (TCSPC) with time-division multiplexing enables simultaneous R/G/B imaging and extension to hyperspectral detection under ultralow light (Zhao et al., 2021).
• Polarimetry: Combining polarization state generators and analyzers with calibrated Mueller matrices allows SPI to recover full spatially-resolved Mueller maps (polarimetric features) through scattering media, exceeding conventional imaging depth limits (Seow et al., 2020).
• Holography and phase imaging: Inline Gabor holography with cyclic binary or amplitude-phase masks, coupled with FFT reconstruction, delivers compact, broadband, phase-preserving microscopy without scientific cameras (Fedorov, 10 Apr 2025).
• Neutron and electron SPI: SPI frameworks ported to neutron (He et al., 2020) and electron (Konečná et al., 2022) instrumentation enable energy-resolved, low-dose, and sub-nanometer imaging in regimes where focal-plane arrays are impractical.

5. Experimental Implementations and Performance Benchmarks

Recent SPI systems demonstrate rapid, high-fidelity acquisition and scalability:

Domain	Pixels	Sampling Ratio (%)	Frame Rate (Hz)	SNR/PSNR (dB)	Reconstruction Time (ms–s)
Visible	1024×768	0.4–0.5	6.8–7	30–34 (PSNR)	80–300 (iterative/NN)
THz	1200×1200	<10	–	>30 (dynamic range)	<210 min total (correlation)
Neutron	32×32	100	–	SNR>20	– (CNN refinement)
Holography	101×103	100	–	~25 (contrast)	O(N log N) (FFT)
Photon-count	64×64	10–20	<1 (full RGB)	≳20 (PSNR)	<1 s (single-step)

Imaging through scattering media achieves contrast enhancement without resolution loss via Fourier spatial filtering (Jauregui-Sánchez et al., 2018). Real-time single-pixel video and ultra-low-light photon-counting modalities are demonstrated with compressed sampling and instant ghost imaging pipelines (Yang et al., 2020Zhao et al., 2021).

6. Impact, Limitations, and Future Directions

SPI provides performance and flexibility advantages where wavelength constraints, hardware limitations, cost, or environment restrict multi-pixel sensor use. For dynamic, sparse, or structured scenes, modern sampling and inversion strategies yield resolutions meeting or surpassing camera standards, with millisecond–real-time latency.

Identified limitations include fundamental speed trade-offs (number of patterns vs. resolution), sampling ratio degradation for non-sparse images, SNR ceilings under high compression, and generalization bounds for deep networks when scene statistics stray from training distributions. Increasing robustness and adaptability will require further integration of physics-based priors, deep optics co-design, on-device learning, and hardware acceleration (FPGA/GPU/ASIC).

Future directions include quantum-enhanced modalities, gigapixel imaging, adaptive sampling pipelines, phase and 3D extension, mid-IR/THz/electron/hyperspectral SPI, and seamless embedding of advanced inference tasks (classification, detection, segmentation) directly on SPI measurement vectors—suggesting SPI will remain pivotal for indirect, computational imaging at the frontiers of experimental science.