Semantic-Aware Exposure Correction Network

• The paper introduces semantic-aware networks that leverage object-level semantic priors to locally adjust exposure, significantly reducing color shift and contrast artifacts.
• It employs a multi-stream feature injection framework with attention-based and non-linear transformation modules, alongside CLIP-guided unsupervised pseudo-ground-truth generation.
• Experimental results demonstrate superior performance with improved PSNR and SSIM metrics, highlighting the method’s effectiveness in preserving natural hues and fine details.

Semantic-aware exposure correction networks are deep learning architectures that integrate object- or region-level semantic information into the process of adjusting image exposure. By leveraging semantic priors—typically generated by semantic segmentation or vision-LLMs—these networks adapt exposure correction strategies at a local, object- or region-specific scale, mitigating artifacts such as color shift and unnatural contrast that often plague conventional, semantics-agnostic correction algorithms. Recent advances also incorporate unsupervised learning paradigms, where pseudo-ground-truths are automatically synthesized, often guided by multimodal models such as CLIP, to avoid manual image relabeling and promote alignment with high-level semantic intent (Wu et al., 27 Jan 2026, Ju et al., 2022).

1. Motivation and Context

Traditional exposure correction approaches—both classical and deep learning-based—tend to optimize for global image statistics or rely on limited local receptive fields. This can lead to suboptimal restoration in the presence of semantically diverse content, as different objects may require distinct correction strategies. Two central challenges are:

• Semantic inconsistency: The disregard for object-wise regional semantics leads to color shift artifacts and inconsistent local corrections.
• Labeling constraints: Real-world exposure correction lacks ground-truth references, as manual relabeling is infeasible at scale.

Semantic-aware architectures address these challenges by infusing semantic masks or context features directly into the correction network and, in unsupervised settings, synthesizing training targets using high-level semantic matching.

2. Architecture of Semantic-Aware Exposure Correction Networks

2.1. Multi-Stream Feature Injection

A representative supervised semantic-aware exposure correction pipeline is SLLEN (Ju et al., 2022), which comprises:

• Main enhancement network (LLEmN): A U-Net variant with three specialized modules:
  • Feature Extraction Module: Extracts low-level features from the encoder, high-level semantic features (HSF) via a hierarchical convolutional module on the segmentation output, and intermediate embedding features (IEF) from the auxiliary segmentation network.
  • Feature Enhancement Module: Combines HSF via attention-based matching and IEF via non-linear transformation branches.
  • Feature Fusion Branch: Learns spatial fusion weights to combine the two enhanced streams.
  • Decoder: Produces the exposure-corrected image.
• Auxiliary semantic segmentation network (SSaN): A U-Net trained for semantic segmentation, with encoder blocks shared with the LLEmN to facilitate joint representation learning.

2.2. Unsupervised Semantic-Aware Exposure Correction

The CLIP-guided framework (Wu et al., 27 Jan 2026) introduces:

• FastSAM segmentation: A frozen pre-trained FastSAM model generates pixel-wise segmentation maps, projected into semantic feature tensors.
• Multi-scale encoder-decoder: Utilizes a four-scale stack of Semantics-Informed Mamba Reconstruction (SIMR) blocks, each containing:
  • Adaptive Semantic-Aware Fusion (ASF): Cross-attention between image and semantic features, followed by parallel spatial and frequency feed-forward branches.
  • Residual Spatial Mamba Group (RSMG): Sequential spatial mamba and spatial-attention submodules for global context propagation, based on efficient state-space mixing.
• CLIP-guided pseudo-ground-truth generation: CLIP is fine-tuned for exposure categories; similarity to textual prompts guides the synthesis of a pseudo-reference via gamma adjustments and further refines the parameters using cosine similarity maximization in CLIP latent space.

3. Semantic Feature Extraction and Fusion

Semantic-aware exposure correction relies on two principal streams of semantic representation:

Semantic Feature Stream	Source	Role in Correction
High-level Semantic Feature (HSF)	Final semantic map or segmentation output	Class-level, global semantics
Intermediate Embedding Feature	Intermediate layers of segmentation encoder	"Randomized" fine-grained details

These streams are fused in the enhancement pipeline via attention-based (HSF) and non-linear transformation (IEF) modules in supervised settings (Ju et al., 2022), or via cross-attention and spatial/frequency branches within ASF in unsupervised networks (Wu et al., 27 Jan 2026). FastSAM in (Wu et al., 27 Jan 2026) supplies fine-grained, object-level semantic masks that enable locally optimal exposure adjustment.

4. Training and Objective Functions

4.1. Alternating and Joint Training Strategies

SLLEN employs an alternating optimization schedule where the semantic segmentation network is iteratively updated using labeled segmentation data, and the exposure correction network is updated using paired low-light exposure data, with shared encoder weights. This interleaved training ensures that representations are shaped by both tasks, facilitating robust semantic-illumination features (Ju et al., 2022).

4.2. Loss Composition

Unsupervised frameworks (Wu et al., 27 Jan 2026) utilize auto-synthesized pseudo-ground-truth images, guided by CLIP, to circumvent manual relabeling. The loss formulation is:

$$L_{\mathrm{total}} = \lambda_1 L_{\mathrm{MSE}} + \lambda_2 L_{\mathrm{COS}} + \lambda_3 L_{\mathrm{SPC}}$$

Where:

• $L_{\mathrm{MSE}}$ is the pixel-wise error to pseudo-GT,
• $L_{\mathrm{COS}}$ is a cosine-based color fidelity loss,
• $L_{\mathrm{SPC}}$ is the semantic-prompt consistency loss combining:
  • Semantic Feature Consistency (SFC): Ensures semantic features extracted from the restored image align with those of the input and reference.
  • Image-Prompt Alignment (IPA): Aligns output with "well-exposed" textual semantics in CLIP space.

SLLEN’s losses (Ju et al., 2022) additionally include segmentation loss, perceptual loss (VGG), knowledge distillation loss for encoder–decoder alignment, total variation for illumination smoothness, and a gradient-based exposure loss.

5. Quantitative and Qualitative Performance

Semantic-aware exposure correction methods consistently outperform non-semantic and prior state-of-the-art approaches on multiple benchmarks.

Method (Dataset)	PSNR (dB)↑	SSIM↑	Notable Gain
SLLEN (LOL-test)	23.8	0.84	+6.2 dB PSNR, +0.24 SSIM over U-Net baseline
CLIP-Guided (MSEC)	19.97	0.8460	Outperforms UEC/PSENet by 1.1 dB/0.01 SSIM
CLIP-Guided (SICE)	18.74	0.6866	+2.0 dB over UEC

Metrics such as LPIPS, BRISQUE, and NIMA also indicate top performance for semantic-aware approaches. Visual analysis shows improved preservation of natural hues, local details, and semantic consistency across objects, where prior methods often exhibit over-brightening or color desaturation (Wu et al., 27 Jan 2026, Ju et al., 2022).

6. Technical Strengths, Limitations, and Extensions

Semantic-aware exposure correction demonstrates the following properties:

• Strengths:
  • Exploitation of both high-level and intermediate semantic context.
  • Locally adaptive correction, reducing color shift and preserving structure.
  • Synergistic multi-tasking through shared networks and joint losses.
  • Unsupervised target synthesis obviates need for manual relabeling.
• Limitations:
  • Increased computational complexity and inference latency versus lightweight baselines.
  • Dependence on semantic segmentation data (for supervised methods) or the quality of pretrained vision-LLMs.
• Extensions (as identified by the authors):
  • Model compression and pruning.
  • Dynamic per-region semantic fusion.
  • Integrating self-supervised or fully unsupervised segmentation.
  • Multi-task pipelines coupling denoising, deblurring, and correction (Ju et al., 2022).

7. Impact and Future Directions

Semantic-aware exposure correction networks have established a new state of the art in both supervised and unsupervised low-light/enhancement tasks, setting benchmarks in PSNR/SSIM and qualitatively outperforming global or region-agnostic baselines (Wu et al., 27 Jan 2026, Ju et al., 2022). The dual use of segmentation-driven priors and multimodal alignment (via CLIP) enables both robust local adaptation and high-level semantic fidelity. Possible future progress may focus on unsupervised learning of semantic priors, real-time inference, and generalization to related enhancement tasks, reflecting a broader trend towards semantics-informed low-level vision.