Tiny AutoEncoder with RRDB Blocks

• Tiny AutoEncoder with RRDB Blocks is a compact deep neural architecture that uses dual autoencoders to separately restore luma and chroma channels.
• It employs specialized pipelines (LumiNet and ChromaNet) with three-layer encoders, RRDB trunks, and decoders to achieve high-fidelity image restoration.
• The design leverages nested residual connections and dense blocks to expand the receptive field and enable scalable model reductions for resource-constrained deployments.

A Tiny AutoEncoder (TinyAE) employing Residual-in-Residual Dense Blocks (RRDB) is a deep neural architecture designed to perform high-fidelity image restoration, with prioritization on efficiency and parameter reduction. Originating in the context of robust JPEG artifact removal regardless of compression quality, such architectures utilize the learning and representational capacity of deep residual and dense connections, organized for memory and computation efficiency. The approach divides restoration into two coupled autoencoders—one for luma (LumiNet) and one for chroma (ChromaNet)—and enables complexity scaling via principled reduction in width, depth, and convolutional kernel sizes (Zini et al., 2019).

1. Dual Autoencoder Architecture

The design consists of two distinct autoencoders each specialized for a color subspace:

• LumiNet restores the Y (luminance) channel using 2D convolutions, accepting inputs of size $H\times W \times 1$ and outputting restored luma.
• ChromaNet processes the $[Y', C_b, C_r]$ channels, leveraging 3D convolutions (in its first layer) for the chroma channels' restoration, using the output of LumiNet as part of its input. ChromaNet maps $H\times W\times 3$ to $H\times W\times 2$.

Each autoencoder contains:

• Encoder: Three sequential convolutional layers with LeakyReLU activations, maintaining the spatial resolution ($\text{stride}=1$, constant $H\times W$), with kernel sizes $3\times3$ and $5\times5$.
• RRDB trunk: A stack of $B_Y = 5$ (LumiNet) or $B_C = 3$ (ChromaNet) RRDBs, each acting on 64-channel feature maps.
• Decoder: Three additional convolutional layers, mirroring the encoder, leading to final outputs with Tanh activation mapping to $[-1, 1]$.

Batch normalization is omitted throughout, as are downsampling/upsampling operations, to reduce overhead and stabilize training. Weight initialization uses He/Kaiming scaled by 0.1.

2. RRDB Block Structure

The RRDB module is a composite block featuring two nested levels of residual connections:

• Dense Block Sequence: Each RRDB comprises $L=5$ dense layers with growth rate $g=32$, where at each layer,

$$x_\ell = \mathrm{LReLU}\left(\mathrm{Conv}^{3\times3}\left([F_0, x_1, ..., x_{\ell-1}]\right)\right)$$

The resulting feature depth grows as $64 + 5 \cdot 32 = 224$, which is reduced back to 64 using a $1 \times 1$ convolution bottleneck.

• Inner and Outer Residuals: The dense block is connected by an inner residual scaled by $\beta = 0.2$,

$F_{\rm out} = F_0 + 0.2 \, F_{\rm DB}$

where $F_{\rm DB}$ is the post-bottleneck representation. RRDBs are then stacked, and the outer skip connects the input to the output of all five dense blocks, again with residual scaling.

• Receptive Field Expansion: Each RRDB increases the receptive field by $11$ pixels, resulting in roughly $60$ pixels for LumiNet after five RRDBs.
• Parameterization: Each RRDB contains approximately $1.03 \times 10^6$ parameters, with precise enumeration available via:

$$\sum_{\ell=1}^5\bigl(9 \cdot (64 + (\ell-1)g )g + g\bigr) + (224 \cdot 64 + 64)$$

3. Parameterization and Computational Footprint

The default configuration results in the following parameter counts:

Component	Encoder	RRDB Trunk	Decoder	Total
LumiNet	≈ 0.3 M	$5\times1.03$ M	≈ 0.3 M	≈ 5.8 M
ChromaNet	≈ 0.4 M	$3\times1.03$ M	≈ 0.3 M	≈ 3.8 M
Combined	—	—	—	≈ 9.6 M

The autoencoders operate fully convolutionally, so memory and computational complexity are dominated by the RRDB blocks.

4. Strategies for Constructing Tiny Variants

Tiny autoencoders with RRDBs are created through dimension, depth, and operation reductions, such as:

• Width Scaling: Scaling all channel widths by a factor $\alpha \in (0,1]$; e.g., $\alpha = 0.5$ reduces base channels 64→32, growth rate 32→16.
• RRDB Count Reduction: Reducing $B_Y$ and $B_C$ (e.g., $B_Y = 3$, $B_C = 2$).
• Dense Layer Pruning: Shortening dense blocks to $L = 3$ layers per block.
• Kernel Replacement: Substituting $5\times5$ convolutions by pairs of $3\times3$ convolutions.

With $\alpha = 0.5$, $B_Y = 3$, and $L = 3$, the total parameter and MAC count can be reduced to under $20\%$ of the full model, while preserving the recovery pipeline structure.

5. Training Procedure and Loss Formulation

Training employs a pixel-wise $\ell_1$ loss across both autoencoders:

$$\mathcal{L} = \frac{1}{N} \sum_{n=1}^{N} \left\| \hat Y_n - Y_n^{\star} \right\|_1 + \lambda_{\rm chroma} \left\| [\hat{C}_{b,n},\hat{C}_{r,n}] - [C_{b,n}^{\star},C_{r,n}^{\star}] \right\|_1$$

with $\lambda_{\rm chroma} = 1$. Optimization uses Adam ($\beta_1=0.9$, $\beta_2=0.999$), an initial learning rate of $2\times 10^{-4}$, and $34,000$ images from DIV2K/Flickr2K, covering JPEG quality factors from 10 to 100. There is no weight decay or dropout.

6. Complexity and Deployment Considerations

Let $P(\alpha,B,L,g)$ denote parameters with base width $c_0 = 64\alpha$ and growth $g = 32\alpha$:

$$P(\alpha,B,L,g) \approx B \Big( L (9 (c_0 + (\ell-1)g)g + g ) + (c_0 + Lg)c_0 \Big)$$

Inference FLOPs per convolution scale as $2HWC_{\rm in}C_{\rm out}k^2$, so total inference cost is proportional to $\alpha^2(B/5)$ times the cost of the full model. The architecture is thus highly amenable to resource-constrained deployments by design.

7. Distinction from Related Approaches

The use of RRDBs as described eschews all BatchNorm layers, leverages residual scaling (0.2), and does not introduce channel attention or weight normalization. The model is remarkable for supporting a quality-independent parameterization, using one set of weights for all JPEG quality factors rather than multiple specialized models. This enables robust operation even on compression qualities not seen during training, which is a significant advancement relative to prior art (Zini et al., 2019).