ResSquVGG16: Compact Residual-Squeeze VGG16 Model

• The paper presents ResSquVGG16, which integrates VGG16’s deep 3×3 convolutions, SqueezeNet’s efficient Fire modules, and ResNet’s residual shortcuts to dramatically reduce model size.
• ResSquVGG16 achieves an 88% reduction in parameters and 23.86% faster training on the MIT Places365 dataset, while maintaining comparable accuracy to standard VGG16.
• The architecture offers practical insights for deploying deep CNNs in resource-constrained environments and can be extended to other backbone networks for improved efficiency.

Residual-Squeeze-VGG16 (ResSquVGG16) is a deep convolutional neural network architecture that integrates VGG16’s 16-layer design with SqueezeNet’s Fire module-based compression and ResNet-style residual shortcuts. The model targets large-scale scene classification while achieving substantial reductions in parameter count, storage requirements, and training time compared to standard VGG16, yet delivers comparable accuracy. The original implementation was empirically evaluated on the MIT Places365-Standard dataset, demonstrating favorable efficiency-accuracy trade-offs by combining architectural innovations from multiple lines of convolutional network research (Qassim et al., 2017).

1. Architectural Design

ResSquVGG16 combines three paradigms: the spatially deep, uniform 3×3 convolutional pattern of VGG16; the parameter-efficient Fire module of SqueezeNet; and the identity-based shortcut connections of ResNet. The network comprises a single 3×3 convolutional layer, twelve Fire modules, three terminal 1×1 convolutions that supplant VGG16’s fully-connected layers, five pooling layers distributed across depth, and four identity-style residual shortcuts.

The architectural staging and channel dimensions are as follows:

Stage	Component	Output Channels
1	Conv1 (3×3, s=2)	64
	Pool1 (3×3, s=2)	64
2	Fire1, Fire2	s1×1=16; e1×1=64; e3×3=64
	Pool2	–
3	Fire3–Fire5	s1×1=32/48; e1×1=128/192; e3×3=128/192
	Pool3	–
4	Fire6–Fire8	s1×1=48/64; e1×1=192/256; e3×3=192/256
	Pool4	–
5	Fire9–Fire12	s1×1=64; e1×1=256; e3×3=256
	Pool5	–
6	Conv6–8 (1×1)	4096, 4096, 365

Each Fire module consists of a squeeze layer with 1×1 convolutions and an expand layer combining parallel 1×1 and 3×3 convolutions, concatenated channel-wise. The squeeze stage restricts the number of input channels to the more expensive expand stage, improving computational efficiency.

Four skip-connections are placed so that each bridges a contiguous block of Fire modules between consecutive pooling layers. Where channel dimensions do not match, the shortcut applies a 1×1 convolutional projection ($W_s$), as formalized by:

$y = F(x;W) + W_s \cdot x$

If channel counts are aligned, $W_s$ reduces to the identity.

2. Parameter Efficiency and Model Compression

ResSquVGG16 achieves remarkable model compression. The standard VGG16 architecture requires approximately 138 million parameters, with a model size of roughly 10.6 GB in Caffe format. This is due largely to the three fully-connected layers at the top of the network. ResSquVGG16, replacing these with three 1×1 convolutions and extensively deploying Fire modules throughout, reduces the parameter count to approximately 15 million, occupying about 1.23 GB on disk—a reduction of 88.4%.

A typical calculation for parameters in the Fire module incorporates squeeze and expand paths:

• $p_{squeeze} = s_{1 \times 1} \cdot C_{in}$
• $p_{expand1} = e_{1 \times 1} \cdot s_{1 \times 1}$
• $$p_{expand3} = e_{3 \times 3} \cdot s_{1 \times 1} \cdot 3 \cdot 3$$

For example, Fire1 comprises 11,264 parameters. The FLOPs per image also drop from ~15.5 GFLOPs (VGG16) to ~1.8 GFLOPs (ResSquVGG16), resulting in estimated 88% computational savings. In terms of parameter memory (FP32), VGG16’s weights consume approximately 552 MB while ResSquVGG16’s require about 60 MB.

3. Training Protocol and Empirical Methodology

The training and evaluation of ResSquVGG16 were conducted on the MIT Places365-Standard dataset, containing 1.8 million training images, 50,000 validation, and 900 test samples across 365 scene categories. Preprocessing included resizing to 256×256, mean-pixel subtraction (per channel), random horizontal flips, and random 227×227 cropping.

Key training settings:

• Optimizer: SGD with momentum 0.9 and weight decay.
• Batch size: 128 (train), 64 (validation).
• Epochs: 50; scheduled learning rate ($LR$) decay by a factor of 5 every 10 epochs and halving the weight decay.
• Xavier initialization ($\sigma=0.01$) for all layers, with final convolutions from $\mathcal{N}(0, 0.01)$.
• Compared to VGG16, which was fine-tuned for 20 epochs, ResSquVGG16 is trained from scratch.

Residual shortcuts are designed to ease gradient flow and mitigate training degradation. Empirically, ResSquVGG16 attains convergence in approximately 2 days and 19 hours, a 23.86% reduction in training time relative to VGG16’s 3 days and 16 hours.

4. Empirical Performance and Comparative Results

Validation accuracy and resource metrics for ResSquVGG16 and VGG16 on the Places365-Standard dataset are:

Network	Top-1 val (%)	Top-5 val (%)	Time	Size
VGG16 (fine-tuned)	54.00	84.30	3d 16h	10.6 GB
ResSquVGG16	51.68	82.04	2d 19h	1.23 GB

ResSquVGG16 matches VGG16 within 2.3% on both Top-1 and Top-5 validation scores, while being 88.4% smaller and training 23.86% faster. Training/validation accuracy curves reveal that ResSquVGG16 converges in approximately 40 epochs compared to VGG16’s 20, suggesting efficiency in gradient propagation despite training from scratch.

5. Design Best Practices, Limitations, and Extensions

The Fire module configuration (s1×1/e1×1/e3×3) may be fine-tuned per architectural stage to trade off parameter count and representational capacity. Residual connections should bridge contiguous convolutional blocks uninhibited by pooling layers. Whenever the shortcut and main paths differ in channel depth, a 1×1 convolutional projection aligns dimensions.

Batch Normalization is not used in the baseline ResSquVGG16 but may further stabilize and accelerate optimization. A minor accuracy drop is observed relative to VGG16, but this is offset by drastic reductions in model size and compute cost, making ResSquVGG16 well-suited for embedded or FPGA/ASIC deployments.

Possible extensions include applying the squeeze+residual paradigm to other backbone architectures (e.g., ResNet, DenseNet) or exploring channel pruning for further dynamic model reduction. These approaches provide a generalized pathway for efficient network scaling and deployment.

6. Implications in Large-Scale Scene Classification

By demonstrating that a convolutional architecture can match VGG16’s classification capacity on a challenging, large-scale dataset while compressing model size and resource requirements by nearly an order of magnitude, ResSquVGG16 provides a practical reference for developing efficient, deep CNNs under hardware or memory constraints. Its design methodology is directly extensible to scenarios where storage, power, or throughput are primary deployment concerns (Qassim et al., 2017).