3DGS$^3$: Joint Super Sampling and Frame Interpolation for Real-Time Large-Scale 3DGS Rendering

Abstract: 3D Gaussian Splatting (3DGS) enables high-quality real-time 3D rendering but faces challenges in efficiently scaling to ultra-dense scenes and high-resolution due to computational bottlenecks that limit its use in latency-sensitive applications. Instead of optimizing the splatting pipeline itself, we propose \textbf{3DGS$^3$}, a unified post-rendering framework that jointly performs super sampling and frame interpolation through differentiable processing of low-resolution outputs to achieve both high-resolution and high-frame-rate rendering. Our \textbf{Gradient- -Aware Super Sampling (GASS)} module leverages the continuous differentiability of 3DGS to extract image gradients that guide a GRU-based refinement network to enable high-fidelity super sampling. Furthermore, a \textbf{Lightweight Temporal Frame Interpolation (LTFI)} module based on a compact U-Net-like backbone fuses temporal and differentiable spatial cues from consecutive frames to synthesize temporally coherent intermediate frames. Experiments on public datasets demonstrate that 3DGS$^3$ achieves superior rendering efficiency and visual quality when compared with state-of-the-art methods and remains compatible with existing 3DGS acceleration techniques. The code will be publicly released upon acceptance.

• The paper introduces a novel post-rendering pipeline that jointly achieves super sampling and frame interpolation on 3D Gaussian Splatting, enhancing real-time efficiency and high-resolution fidelity.
• The proposed GASS and LTFI modules leverage analytic gradients and recurrent networks to improve spatial detail and temporal smoothness through gradient-aware interpolation and feature warping.
• Experimental results demonstrate impressive gains with up to 246 FPS and quality metrics of PSNR 49.22 and SSIM 0.995, validating the framework's effectiveness in large-scale rendering.

Joint Super Sampling and Frame Interpolation for Real-Time Large-Scale 3DGS Rendering

Motivation and Context

3D Gaussian Splatting (3DGS) is a prevailing explicit representation for high-quality real-time radiance field rendering, leveraging anisotropic Gaussians to encode scene information and offering continuous and differentiable outputs. Despite advances in compact Gaussian representations, pruning algorithms, and hardware acceleration, achieving real-time performance, especially for ultra-dense or high-resolution scenarios, remains challenging due to computational bottlenecks. Commercial rendering pipelines such as DLSS, XeSS, and FSR demonstrate that post-process neural upscaling and frame interpolation can efficiently reconstruct high-resolution or high-framerate outputs from low-resolution renderings. 3DGS$^3$ investigates this paradigm within the 3DGS domain, proposing a modular post-rendering pipeline that exploits differentiable cues intrinsic to 3DGS outputs to jointly perform super sampling and frame interpolation (Figure 1).

Figure 1: 3DGS$^3$ employs spatio-temporal post-processing on 3DGS-rendered frames, accelerating 3DGS while preserving high visual quality.

Methodology

Architecture Overview

The 3DGS$^3$ pipeline is constructed around two principal modules: Gradient-Aware Super Sampling (GASS) and Lightweight Temporal Frame Interpolation (LTFI) (Figure 2). The GASS module receives low-resolution 3DGS renderings and exploits image gradients—derived analytically due to 3DGS’s differentiable nature—combined with geometric priors and temporal features, to reconstruct high-resolution frames. Subsequently, LTFI synthesizes intermediate frames by fusing spatial gradients, geometric maps, and temporal recurrent representations via a compact U-Net-like backbone.

Figure 2: The pipeline sequentially applies GASS for super sampling and LTFI for frame interpolation on low-resolution 3DGS outputs.

GASS Module

The GASS module employs Hermite-like gradient-aware interpolation, leveraging both pixel intensities and their partial derivatives, computed from the analytical blending of Gaussian splats. For each neighborhood, a bivariate polynomial interpolation is constructed, yielding high-resolution blocks expressed as linear combinations of local intensities and gradients. This is efficiently implemented using convolutional and transposed convolutional layers. To further enhance temporal consistency, a geometry-temporal recurrent refinement network based on a modified GRU assimilates geometric features (depth, normals) and historical hidden states for robust super sampling.

LTFI Module

LTFI operates by warping features from the preceding super-resolved frame and the subsequent low-resolution frame using camera pose-derived motion vectors. Feature fusion is accomplished through a lightweight U-Net backbone that integrates low-frequency and attention-driven features via pixel shuffle upscaling and spatial attention blocks. The final frame is blended based on learned confidence weights. A temporal recurrent unit ensures inter-frame consistency by accumulating spatio-temporal features across the sequence.

Figure 3: Detailed architecture of GASS (gradient-aware interpolation and recurrent refinement) and LTFI (warping, fusion, and temporal recurrence).

Experimental Results

Dataset and Evaluation Protocol

Experiments are conducted on Mip-NeRF 360, Tanks and Temples, and Deep Blending, with quantitative PSNR, SSIM, and FPS metrics. The pipeline is benchmarked both standalone and integrated with existing acceleration approaches (e.g., Mini-Splatting, Speedy-Splat, AdR-Gaussian).

Comparative Analysis

3DGS$^3$ demonstrates superior rendering efficiency (up to 246 FPS at $2560\times1440$) while maintaining highly competitive visual quality (PSNR 49.22, SSIM 0.995 at $4096\times2160$) compared to bicubic interpolation, learning-based upscaling (NinaSR-B1, RT4KSR, ECBSR), and video frame interpolation methods (FLAVR, InterpAny, GIMM-VFI). The qualitative comparison confirms that GASS yields sharper details and that LTFI achieves smoother temporal transitions with minimal flicker and artifacts.

Figure 4: GASS produces superior spatial detail and LTFI ensures temporally coherent interpolation compared to baselines.

Ablation Studies

Ablations of GASS, gradient-aware interpolation (GAI), geometry-temporal recurrent refinement (GTRR), gradient information (GI), and temporal recurrent unit (TRU) reveal that omitting these components leads to significant degradation in both quantitative metrics and visual fidelity, underscoring their necessity (Figure 5).

Figure 5: Removal of critical GASS, GAI, GI, or TRU elements results in blurred textures and temporal artifacts.

Integration with Acceleration Frameworks

The framework synergistically enhances the FPS of diverse 3DGS accelerators without sacrificing visual fidelity, confirming its modularity. Its pipeline-parallel architecture allows for scalable integration and robust performance in latency-constrained applications.

Theoretical and Practical Implications

3DGS$^3$ establishes a scene-agnostic, modular post-rendering approach that overcomes the scaling bottlenecks inherent to explicit parametric representations. By leveraging differentiable cues unique to 3DGS, it achieves both spatial and temporal super-resolution through lightweight neural modules with analytic gradient guidance. The framework's compatibility with existing acceleration schemes positions it as a versatile solution for VR/AR, real-time graphics, and mobile deployment. Its analytic-gradient foundation could inspire further research in differentiable rendering, neural upscaling, and temporal consistency for novel radiance field representations.

Future Directions

While 3DGS$^3$ primarily targets static 3D scenes, its modular recurrent components suggest extensibility to dynamic and deformable scenarios. Research into compact model variants for mobile hardware and memory-constrained environments is warranted, and integration with dynamic Gaussian splatting or temporal priors could further expand its applicability.

Conclusion

3DGS$^3$ introduces an analytic-gradient driven post-rendering framework for real-time 3DGS acceleration, jointly addressing super sampling and frame interpolation via differentiable spatial and temporal cues. The method achieves bold claims in rendering efficiency and fidelity, demonstrating strong synergy with prior acceleration pipelines and modular extensibility. Its results validate the efficacy of post-rendering approaches for explicit radiance field representations and open new avenues for scalable real-time rendering in interactive applications (2605.11489).