Fourier Enhancement Module

• Fourier Enhancement Modules are components that modify the amplitude and phase spectra of images, audio, or feature representations via Fourier transforms.
• They employ techniques like channel-wise and quaternion transforms, spectral selection, and learned amplitude/phase mapping to robustly tackle tasks such as super-resolution and low-light enhancement.
• Empirical results show that these modules improve metrics like PSNR while reducing computational costs and preserving essential structural and perceptual features.

A Fourier Enhancement Module is a neural or algorithmic component that explicitly manipulates the amplitude and/or phase spectra of image, audio, or feature representations in the Fourier domain to improve signal quality, restore lost content, or control artifacts. Such modules now underpin state-of-the-art systems in image super-resolution, low-light image enhancement, image demosaicking, color image contrast, binaural speech enhancement, partial differential equation (PDE) solvers, and other domains. Core technical elements include channel-wise or quaternion Fourier transforms, explicit spectral selection or enhancement, learned amplitude/phase mapping, and loss functions that directly enforce frequency-domain fidelity.

1. Mathematical Basis: Fourier Transformation and Spectral Manipulation

At the heart of all Fourier Enhancement Modules is a (real or complex) 2D or 1D Discrete Fourier Transform (DFT), generally channel-wise for image or audio data: $$F(u,v) = \sum_{x=0}^{H-1}\sum_{y=0}^{W-1} I(x,y) \, e^{-j2\pi\left(\frac{ux}{H} + \frac{vy}{W}\right)}$$ with inverse: $$I(x,y) = \frac{1}{HW} \sum_{u=0}^{H-1}\sum_{v=0}^{W-1} F(u,v) \, e^{+j2\pi\left(\frac{ux}{H} + \frac{vy}{W}\right)}$$

Channel-wise decomposition yields:

• Amplitude (or magnitude): $A(u,v) = |F(u,v)|$
• Phase: $P(u,v) = \angle F(u,v)$

Many modern variants further embed color as a pure quaternion signal and operate via a Quaternion Discrete Fourier Transform (QDFT), enabling true inter-channel spectral manipulation (Grigoryan et al., 2018, Grigoryan et al., 2017).

Key spectral operations include:

• Element-wise pointwise nonlinearities on amplitude (e.g., α-rooting: $|F(u,v)|^\alpha$), phase, or both.
• Learnable convolutional post-processing of spectral components.
• Fourier feature positional encoding for frequency-aware spatial upsampling.
• Selective masking or attention over specific frequency bands.

2. Architectural Strategies: Module Positioning and Integration

Fourier Enhancement Modules are integrated at various points of the system, depending on application:

• Super-resolution: As an upsampling unit, replacing sub-pixel convolution; main example is Frequency-Guided Attention (FGA), which combines Fourier-feature MLP embeddings and cross-resolution correlation attention, plus explicit frequency-domain L₁ supervision (Choi et al., 14 Aug 2025).
• Low-light enhancement: As bottleneck or stagewise blocks for amplitude–phase disentanglement and restoration, with lightweight spatial/frequency fusion (Li et al., 2023, Wang et al., 2023, Tan et al., 2024, She et al., 1 Aug 2025, Zhuang et al., 2022, Zhang et al., 2024, Zhang et al., 6 Aug 2025).
• Image demosaicking: Dual-path frequency enhancement with learnable Fourier domain selectors to (i) spatially generate missing details from selected sub-spectra and (ii) suppress false frequencies using CFA guidance (Liu et al., 20 Mar 2025).
• Speech enhancement: Channel-independent global Fourier modulation for long-term dependency modeling, strictly preserving phase to maintain interaural cues (Lu et al., 17 Sep 2025).
• Color image contrast: Full quaternion-based QDFT and α-rooting for joint frequency-domain manipulation across RGB (Grigoryan et al., 2018, Grigoryan et al., 2017).
• Operator learning for PDEs: Complementary convolution and equivariant attention branches are wrapped around traditional FNOs to overcome low-frequency bias and recover oscillatory structure (Zhao et al., 2023).

Module placement is typically at points where large-scale structure, detail, or global context must be controlled with minimal computational cost and maximal interpretive power in the frequency domain.

3. Core Mechanisms: Frequency Manipulation and Amplitude/Phase Processing

The main innovations involve:

• Separate or joint amplitude/phase refinement: Most modules branch amplitude and phase, either to allow noise suppression in phase while boosting brightness in amplitude (as in UHDFour, (Li et al., 2023)), or to ensure that joint restoration preserves both sharp edges (phase) and luminance (amplitude) (She et al., 1 Aug 2025, Zhang et al., 2024).
• Learnable frequency selection/masking: Binary or continuous masks operate in the 2D/3D Fourier planes to select bands for refinement or suppression (notably SFE in demosaicking, (Liu et al., 20 Mar 2025)).
• Spectral-domain losses: Direct L₁, L₂, or perceptual (VGG) losses on amplitude, phase, or full complex spectra, driving module parameters toward reference frequency distributions (Choi et al., 14 Aug 2025, Wang et al., 2023, Xue et al., 2024).
• Attentive fusion: Channel/spatial/frequency attention blocks merge findings from different bands or processing branches at multiple scales.

Multiple modules further capitalize on domain priors, such as SNR maps for spatial–frequency fusion (Wang et al., 2023), or structural priors derived from auxiliary networks or input modalities (Zhang et al., 6 Aug 2025, Zhang et al., 2024).

4. Comparative Methodology and Module Variants

A range of Fourier Enhancement Module instantiations can be organized as follows:

Application domain	Module type / key innovations	Notable references
Single Image SR	Fourier-feature MLP, correlation attention, spectral loss	FGA (Choi et al., 14 Aug 2025)
LLIE (UHD, standard)	Amplitude–phase separation, blockwise FFT/IFFT, explicit branch fusion	UHDFour (Li et al., 2023), FourLLIE (Wang et al., 2023), DMFourLLIE (Zhang et al., 2024)
Demosaicking	Dual-path SFE, learnable selectors, CFA-guided suppression	DFENet (Liu et al., 20 Mar 2025)
Color enhancement	Quaternion QDFT, α-rooting, genetic search on CEME	(Grigoryan et al., 2018, Grigoryan et al., 2017)
Audio (binaural speech)	Frequency-wise global adaptive modulation, phase preservation	GAFM (Lu et al., 17 Sep 2025)
Physics/PDE	Complementary convolution, equivariant attention wrapped over FNO	(Zhao et al., 2023)

Major distinctive features include:

• Use of QDFT and quaternion representations for unified color treatment (Grigoryan et al., 2018, Grigoryan et al., 2017).
• Explicit spectral supervision with direct loss functions in the Fourier domain, required for stable and sharp restoration (Choi et al., 14 Aug 2025, Wang et al., 2023, Xue et al., 2024).
• Efficient scaling via low-res or subband-only transforms (UHDFour, SPJFNet), with HR adjustment blocks or multi-stage joint frequency–spatial refinement (Li et al., 2023, Zhang et al., 6 Aug 2025).
• Content- and context-adaptive masks or gates, dynamically modulating gain in the frequency domain based on learned priors or external features (Liu et al., 20 Mar 2025, Lu et al., 17 Sep 2025, Zhang et al., 6 Aug 2025).

5. Empirical Impact and Ablative Findings

Fourier Enhancement Modules consistently yield improvements across diverse tasks and benchmarks. Key empirical results include:

• Super-resolution: FGA adds 0.12–0.14 dB PSNR over sub-pixel convolution on standard benchmarks and up to +29% frequency-domain consistency in high-frequency bands (Choi et al., 14 Aug 2025).
• Low-light enhancement: Removal of the module in various contexts drops PSNR by 0.5–1 dB or more; additional spectral loss or SNR-guided fusion often brings another ~1 dB uplift and better perceptual structure (Li et al., 2023, Wang et al., 2023, She et al., 1 Aug 2025, Zhang et al., 2024).
• Demosaicking: CFA-guided suppression drives PSNR up to 32.04 dB vs. baseline 31.33 on LineSet37; dual learnable selectors outperform single/masked variants (Liu et al., 20 Mar 2025).
• UHD low-light: Separate amplitude/phase enhancement at 1/8 scale preserves quality with sub-25ms latency for 4K frames (Li et al., 2023).
• PDE surrogate modeling: Enhanced FNOs via convolution/attention drop normalized MSE from 1.9×10⁻² to 0.78×10⁻² in high-oscillation benchmarks (Zhao et al., 2023).
• Binaural audio: GAFM backbone achieves average ILD error 3.86 dB and IPD error 0.75 rad while maintaining low parameter count and high MBSTOI (Lu et al., 17 Sep 2025).

Ablations routinely demonstrate that spatial-only or channel-by-channel transforms underperform compared to modules exploiting explicit, parametric control in the Fourier domain.

6. Computational Considerations and Scalability

Fourier Enhancement Modules can greatly reduce parameter count and computational overhead:

• Channel-wise FFT/IFFT at reduced resolution (e.g., 1/8 original) yields O(1/N) cost reduction (Li et al., 2023, Zhang et al., 6 Aug 2025).
• Plug-in modules (FGA, FouSpa) add ≤0.3M parameters (super-resolution, LLIE) (Choi et al., 14 Aug 2025, Li et al., 2023).
• SPJFNet reduces cost by 75% compared to U-Net on equivalent low-frequency bands (Zhang et al., 6 Aug 2025).
• Direct frequency selection suppresses irrelevant computation in both demosaicking and PDE surrogate learning (Liu et al., 20 Mar 2025, Zhao et al., 2023).

Some modules (e.g., multi-branch, multi-scale variants) scale linearly or sub-linearly with input dimensions, making them suitable for high-resolution or real-time workflows.

7. Limitations, Failure Modes, and Future Directions

Observed and hypothesized limitations include:

• Computational overhead in modules with multi-scale frequency selection, cross-domain attention, or deep bottleneck stacking (Xue et al., 2024).
• Spectral overconstraint: Excessive spectral loss weighting may harm robustness in extremely noisy or information-degraded regions (Xue et al., 2024).
• Resilience to modality mismatch: Performance can degrade if structural priors or auxiliary branches (e.g., event/infrarad/X modality) are error-prone or unavailable (She et al., 1 Aug 2025, Zhang et al., 6 Aug 2025).
• Complex tuning: Heavy fusion of spatial/frequency/semantic losses with many hyperparameters can complicate joint optimization and reproducibility (Xue et al., 2024).
• Strict phase preservation may not always be optimal for some domains (e.g., strongly denoising applications), yet is essential when structural or spatial cues (ILD/IPD, moiré) must be strictly maintained (Lu et al., 17 Sep 2025).

Emerging trends suggest increasing synergy with transformer-based architectures, wavelet–Fourier hybrids, and modules that explicitly model both magnitude and phase in data-driven or physics-constrained settings.

---

References:

• (Choi et al., 14 Aug 2025): "Fourier-Guided Attention Upsampling for Image Super-Resolution"
• (Li et al., 2023): "Embedding Fourier for Ultra-High-Definition Low-Light Image Enhancement"
• (Wang et al., 2023): "FourLLIE: Boosting Low-Light Image Enhancement by Fourier Frequency Information"
• (Zhang et al., 2024): "DMFourLLIE: Dual-Stage and Multi-Branch Fourier Network for Low-Light Image Enhancement"
• (Liu et al., 20 Mar 2025): "Frequency Enhancement for Image Demosaicking"
• (Xue et al., 2024): "Low-light Image Enhancement via CLIP-Fourier Guided Wavelet Diffusion"
• (She et al., 1 Aug 2025): "Exploring Fourier Prior and Event Collaboration for Low-Light Image Enhancement"
• (Zhuang et al., 2022): "DPFNet: A Dual-branch Dilated Network with Phase-aware Fourier Convolution for Low-light Image Enhancement"
• (Lu et al., 17 Sep 2025): "A Lightweight Fourier-based Network for Binaural Speech Enhancement with Spatial Cue Preservation"
• (Grigoryan et al., 2018): "Alpha-rooting color image enhancement method by two-side 2-D quaternion discrete Fourier transform followed by spatial transformation"
• (Grigoryan et al., 2017): "Modified Alpha-Rooting Color Image Enhancement Method On The Two-Side 2-D Quaternion Discrete Fourier Transform And The 2-D Discrete Fourier Transform"
• (Zhao et al., 2023): "Enhancing Solutions for Complex PDEs: Introducing Complementary Convolution and Equivariant Attention in Fourier Neural Operators"
• (Zhang et al., 6 Aug 2025): "SPJFNet: Self-Mining Prior-Guided Joint Frequency Enhancement for Ultra-Efficient Dark Image Restoration"
• (Tan et al., 2024): "Wavelet-based Mamba with Fourier Adjustment for Low-light Image Enhancement"