LCEVC: Low Complexity Enhancement Video Coding

• LCEVC is a layered video coding technique that adds a lightweight enhancement layer to a base codec, ensuring improved rate–distortion performance and backward compatibility.
• It combines traditional block-based residual coding with emerging neural network methods to optimize video quality under constrained bandwidth and latency scenarios.
• Objective and subjective studies confirm that LCEVC achieves significant BD-rate reductions and higher MOS scores, making it ideal for live, interactive, and resource-constrained streaming.

Low Complexity Enhancement Video Coding (LCEVC) is a standardized layered video coding approach (ISO/IEC 23094-2) that provides enhancement capabilities for existing codecs by adding a lightweight secondary “enhancement” layer to a lower-complexity base encoded stream. LCEVC is designed to achieve improved rate–distortion efficiency and reduced computational complexity relative to monolithic next-generation codecs, with wide applicability to live, interactive, and resource-constrained streaming scenarios. The architecture and evaluation of LCEVC are grounded in quantitative and subjective studies and open-source implementations, with recent advances also encompassing hybrid approaches using low-complexity neural networks and compatibility with the latest codecs (Barman et al., 2022, Klopp et al., 2019, Ramzan et al., 15 Jan 2026, Hu et al., 2024).

1. Layered Coding Principles and System Architecture

LCEVC operates on a fundamental two-layer structure. The base layer employs a conventional video codec (e.g., H.264/AVC, H.265/HEVC, VVC) to encode the video at a reduced spatial resolution—typically downsampled by a factor of 2 along each spatial dimension. The enhancement layer captures the residual signal, defined as the difference between the original full-resolution frame and the decoded, upscaled base-layer reconstruction. At the decoder side, the base layer bitstream is decoded and upscaled using a fixed, low-complexity interpolation filter; the enhancement residual, decoded separately, is then added to yield the final high-resolution output (Barman et al., 2022, Ramzan et al., 15 Jan 2026).

Formally, if $X$ denotes the original full-resolution frame, $B$ the down-scaled version, $\hat{C}$ the decoded and upscaled base, and $E = X - \hat{C}$ the residual, the enhancement layer encodes $E$ for later addition. A standard rate–distortion minimization is conducted for the enhancement coder, with the total bit-budget jointly allocated between base and enhancement layers (Barman et al., 2022).

This workflow preserves backward compatibility: decoders that only understand the base layer can still output a lower-quality upscaled video, while LCEVC-aware decoders reconstruct higher-quality outputs by applying the enhancement layer (Barman et al., 2022).

2. Enhancement Layer Realizations: Traditional and Neural Approaches

The LCEVC standard, as implemented in the MPEG-5 Part 2 framework and commercial SDKs, uses a fixed block-based residual coder for the enhancement layer with fast transforms (e.g., DCT), simple quantization, and entropy coding. The block-based enhancement layer is typically lightweight, with minimal memory and computational requirements compared to the full-resolution base codec (Barman et al., 2022, Ramzan et al., 15 Jan 2026).

Recent research extends LCEVC concepts using trainable neural networks as enhancement layers. One approach employs a 5-layer factorized CNN with sub-2 kB parameter count and per-frame complexity of approximately 500 multiply–accumulate operations (MACs) per pixel—a two-order-of-magnitude reduction relative to large pretrained denoisers. The CNN is trained on-the-fly at encoding to predict the enhancement residual based on the decoded base reconstruction, and its quantized parameters are signaled as part of the enhancement bitstream (Klopp et al., 2019). This neural realization closely matches the LCEVC philosophy of codec-agnostic, hardware-friendly, low-complexity enhancement.

A further extension employs jointly optimized neural pre- and post-processors as “switchable neural wrappers” surrounding a standard codec. Here, the encoder selects among multiple downsampling ratios and processing modes via rate–distortion optimized selection, and a residual UNet upsampler (complexity: 516 MACs/pixel, 8.2 k parameters) post-processes the decoded base reconstruction. This approach achieves substantial rate–distortion improvement (9.3% BD-Rate reduction over VVC, 6.4% over AOM CTC A1) while maintaining full standard compliance and extremely low computational overhead (Hu et al., 2024).

3. Rate–Distortion Performance and Complexity Analysis

Objective studies using both classical and perceptual metrics (e.g., VMAF, PSNR) reveal substantial rate–distortion improvements from LCEVC augmentation. For live gaming video at 1080p60, LCEVC-augmented x264 achieves a 42.1% BD-Rate reduction (VMAF) relative to native x264, and LCEVC-augmented x265 yields a 38.9% reduction against standalone x265. The benefits are most pronounced at lower bitrates and higher content complexity, with per-sequence savings ranging from approximately –70% (simple content) to –28% (very low-motion) (Barman et al., 2022).

In the context of UHD coding, layered configurations using LCEVC (with VVC base) result in MOS improvements of 0.4–1.4 points (on an 11-point scale) at low enhancement-layer bitrates (E10 ≈ 10% of total bitrate) compared to upsampled VVC, and approach MOS parity with full ML-VVC profiles at moderate E50 (50%) enhancement bitrates; inter-scheme differences are frequently within statistical confidence intervals (Ramzan et al., 15 Jan 2026).

The complexity overhead for the enhancement layer is modest: LCEVC encoding/decoding at standard settings adds only 10–20% to total throughput, typically outperforming full-resolution base codecs at equivalent CPU presets. Neural enhancement variants maintain <0.5% added per-pixel complexity compared to traditional deep learned post-processors, and are deployable in real time on consumer hardware (Barman et al., 2022, Klopp et al., 2019, Hu et al., 2024).

4. Subjective Quality Assessment

Subjective evaluation corroborates the objective gains observed for LCEVC. Using double-stimulus pairwise comparisons and the extended continuous Absolute Category Rating scale, LCEVC-enhanced streams deliver significantly higher MOS relative to their base codecs, especially for x264 at bitrates below 3000 kbps. Statistical analysis confirms strong effect sizes (e.g., ANOVA p < 0.001, partial η² = 0.96 for x264), and improvements taper for highly efficient base codecs or high bitrate operation (Barman et al., 2022).

Expanded UHD studies using a Degradation Category Rating procedure with 25 naive viewers confirm that LCEVC enhancement on a VVC base surpasses upsampled VVC (MOS improvement +0.4 to +1.4 at E10; ≈7.6–8.3 MOS at E50), with quality nearly indistinguishable from the more complex ML-VVC multilayer coding, especially at higher enhancement bitrates (Ramzan et al., 15 Jan 2026). Both objective and subjective analyses validate that LCEVC’s perceptual gains are most marked at bandwidth-constrained operating points.

5. Standards Compliance, Implementation, and Bitstream Signaling

LCEVC is fully interoperable with existing decoder deployments for the base codec. Enhancement-layer signaling is handled via side-streams or SEI/user-data fields, allowing LCEVC-aware clients to reconstruct enhanced content without breaking backward compatibility. Minimal per-sequence signaling overhead (e.g., two 2-bit flags for neural wrapper modes) renders bitstream expansion negligible in practice (Hu et al., 2024).

The enhancement-layer design—whether traditional block residual coder or neural post-processor—maintains a fixed, shallow inference pathway and small memory/parameter footprint (typically <16 kB for 4K content per-GoP CNN, or ~8 k parameters for neural UNet), enabling efficient hardware and software decoder integration (Klopp et al., 2019, Hu et al., 2024).

6. Limitations, Use Cases, and Future Directions

LCEVC exhibits diminishing returns in operating regimes where the base codec is already highly efficient (e.g., x265 at ≥4000 kbps or VVC at optimal presets), and its additive benefit in very-high-resolution/extreme-HDR scenarios remains to be characterized (Barman et al., 2022, Ramzan et al., 15 Jan 2026). Rate–distortion metrics such as BD-Rate, originally tied to PSNR, require careful interpretation when using perceptual indices (e.g., VMAF).

The principal advantages and recommended use cases lie in applications with strict complexity or latency constraints, such as live or interactive streaming (eSports, cloud gaming) and low-power broadcast/mobile devices. Ideal configurations employ mid-speed base codec presets paired with LCEVC, targeting transmission bitrates where perceptual uplift is most significant (typically <3000 kbps for 1080p60).

Active research extends LCEVC methodologies to multi-resolution adaptive bitrate ladders, 4K/HDR gaming, AV1/VVC base codecs, and further refinement of cross-layer rate–distortion optimization (Barman et al., 2022, Ramzan et al., 15 Jan 2026). The integration of neural enhancement layers deepens the codec-agnostic, low-latency paradigm while opening paths for data-driven residual coding within the standardized LCEVC workflow (Klopp et al., 2019, Hu et al., 2024).

7. Comparative Overview and Broader Implications

The following table summarizes LCEVC’s principal attributes relative to alternative scalable coding approaches and reference codecs:

Feature	MPEG-LCEVC	Neural Residual LCEVC (Klopp et al., 2019)	Switchable Neural Wrapper (Hu et al., 2024)
Compatibility	Any legacy codec	Any legacy codec	Any HEVC/VVC
Enhancement Method	Linear + block residual	On-the-fly 5-layer CNN	UNet pre/post (516 MACs/pix)
Typical BD-Rate Saving	38–42% (VMAF, 1080p60)	7–9% (4K), up to 20% chroma	9.3% (4K, VVC), 6.4% (AOM CTC A1)
Decoding Complexity	<20% increase vs. base	≈500 MACs/pixel	≈0.5 kMACs/pixel
MOS Gain vs. Base (Subjective)	Up to +1.4	Not evaluated	Not reported
Bitstream Overhead	Side bitstream/seamless	≈hundreds of bytes/GoP	4 bits/sequence

The layered, enhancement-oriented design philosophy underlying LCEVC enables persistent rate–distortion gains and complexity savings across diverse coding pipelines. The deployment of such schemes in industry and research contexts suggests a convergence of traditional scalable coding and learned post-processing within regulatory and standards-constrained workflows (Barman et al., 2022, Ramzan et al., 15 Jan 2026, Klopp et al., 2019, Hu et al., 2024).