TemporalGS: Training-Free Plug-and-Play Acceleration for 3D Gaussian Splatting Rendering via Temporal Priors

Abstract: 3D Gaussian Splatting (3DGS) has revolutionized novel-view synthesis with its fast and high-fidelity rendering. However, rendering at high FPS and low latency across various scenes remains a challenge, especially when large amounts of 3D Gaussian ellipsoids appear in the scene. To address this issue, we introduce TemporalGS, to the best of our knowledge, the first training-free plug-and-play algorithmic approach to accelerate 3DGS rendering without any post-training or post-processing, implemented on top of tile-based software rasterization. The key idea is that, instead of rendering frames independently as 3DGS, we leverage the temporal priors, represented by novel geometry and appearance buffers, etc., to reduce redundancy of Gaussian preprocessing, sorting, and rasterization operations of consecutive frames. Specifically, we propose two acceleration strategies: (1) temporal dynamic culling, which filters out Gaussians that contribute less to current frame rendering; (2) selective rendering, which renders only a small portion of tiles that cannot be approximated by the temporal priors. By adapting and interleaving these two strategies, TemporalGS yields a simple but effective plug-and-play solution for 3DGS rendering speed-up without any training. Extensive experiments show that TemporalGS achieves comparable or even better performance compared to existing state-of-the-art post-training or post-processing-based 3DGS rendering acceleration approaches. TemporalGS can significantly enhance the rendering speed of various 3DGS methods, achieving up to $1.48\times$ acceleration, while maintaining competitive rendering quality. We further extend our TemporalGS to hardware rasterization-based 3DGS to show the portability of our algorithm.

• The paper introduces a training-free module using temporal priors to accelerate 3D Gaussian Splatting rendering without retraining.
• It employs dynamic culling and selective rendering techniques to reduce redundant computations and improve FPS by up to 1.48x in complex scenes.
• The method integrates seamlessly with various hardware and software platforms while incurring minimal memory overhead.

TemporalGS: Plug-and-Play Acceleration for 3D Gaussian Splatting Rendering

Introduction and Motivation

3D Gaussian Splatting (3DGS) has become a dominant paradigm for real-time radiance field rendering, offering superior efficiency and fidelity compared to Neural Radiance Fields (NeRF). Despite its algorithmic advances, high frame-rate and low-latency rendering in large-scale or complex scenes remain computationally prohibitive due to the burden of preprocessing, sorting, and rasterization of massive Gaussian primitives. Existing acceleration techniques either require retraining, post-processing, or are tightly coupled to specific software or hardware stacks.

TemporalGS introduces a training-free, plug-and-play algorithmic module for 3DGS acceleration. It leverages temporal priors—geometry and appearance buffers derived from previously rendered frames—to minimize redundant computation in the handling of Gaussians across consecutive frames. The method decomposes into two synergistic strategies: temporal dynamic culling and selective rendering. These strategies can be interleaved adaptively to maximize rendering throughput and minimize latency, while remaining agnostic to underlying 3DGS variants and hardware rasterization APIs.

Methodology

Temporal Priors Construction

TemporalGS defines three core temporal priors: Temporal Geometry Buffer (TGB), Temporal Appearance Buffer (TAB), and Temporal Gaussian Buffer. TGB encapsulates per-pixel depth statistics (mean and standard deviation) extracted during the rendering of a reference frame. Warping operations propagate TGB from the reference frame to subsequent frames using view transformation matrices, enabling efficient occlusion reasoning.

TAB inherits the rendered image from the reference frame, and is warped to the viewpoint of the target frame by leveraging the warped depth map. Bilinear interpolation is employed to handle fractional pixel mapping. Temporal Gaussian Buffer catalogs the set of Gaussians visually active in the reference frame, serving as a high-recall candidate pool for subsequent culling.

Acceleration Strategies

Temporal Dynamic Culling (TDC)

TDC reduces Gaussian preprocessing via adaptive frustum culling and temporal occlusion culling. Adaptive frustum culling restricts consideration to Gaussians within the normalized device coordinate bounds defined by the temporal Gaussian buffer, avoiding unnecessary projections and duplications. Temporal occlusion culling further removes Gaussians lying behind warped geometry buffers in subsequent frames, utilizing standard deviation-based depth windows to preserve high recall for newly emergent surfaces.

Selective Rendering (SR)

SR minimizes sorting and rasterization by rendering only regions that deviate significantly from the prior frame. Tiles are selected for full rendering if (1) new Gaussians emerge closer than warped TGB depths, or (2) pixel-to-pixel correspondences mapping fails due to unmatched geometry. Occluded tile detection uses $\alpha$-blending thresholds on Gaussians to measure visual contribution. Unmatched tile detection employs depth filling and backward warping to probe correspondence failure. SR then stitches the warped TAB and freshly rendered tiles into the final output, with adaptive reference frame insertion triggered whenever the percentage of rendered tiles exceeds a configurable threshold $\eta$.

Figure 1: High-level comparison: vanilla 3DGS renders frames independently, while TemporalGS leverages temporal priors to efficiently render frame sequences and minimize redundant computation.

Algorithmic Integration and Hardware Portability

TemporalGS is implemented in CUDA for maximal compatibility and performance. Its operations are modular and can be inserted into existing 3DGS rasterization pipelines—both software and hardware (OpenGL, Vulkan, WebGPU). The method incurs negligible memory overhead relative to baseline 3DGS and does not require modification to upstream Gaussian reconstruction or rendering steps.

Figure 2: Visualization of frame-by-frame culling and selective rendering: temporal priors enable adaptive frustum and occlusion culling, as well as selective tile rendering for efficient frame generation.

Empirical Evaluation

TemporalGS attains up to $1.48\times$ acceleration in large-scale aerial and urban scenes, while preserving competitive image quality metrics (PSNR, SSIM, LPIPS). On the Mip-NeRF-360 benchmark, plug-in TemporalGS variants of vanilla 3DGS, LightGS, CP3DGS, and C3DGS achieve 25–33% higher FPS with only 0.38–0.52 PSNR drop relative to their original counterparts. In large-scale environments reconstructed by CityGaussian, TemporalGS plug-ins outperform post-training and post-processing acceleration baselines by achieving 81–148% higher FPS at a cost of only 0.19–0.74 PSNR drop.

Comparisons to NeoG, a temporal coherence-based method, reveal that TemporalGS achieves superior scalability and memory efficiency—NeoG doubles memory consumption and does not generalize to sorting-free Gaussian Splatting architectures, while TemporalGS incurs only 5% overhead and is agnostic to rasterization pipeline. The results also show hardware portability: integrating TemporalGS into hardware rasterization methods (fast_gauss, web-splat) yields notable acceleration, except in extreme cases where hardware pipeline optimizations are not aligned with float16 depth rendering.

Module ablation studies confirm that the selective rendering module is the primary driver of FPS acceleration, followed by temporal occlusion culling and adaptive frustum culling. Analysis of camera motion speed highlights robustness, as performance degradation is gradual with increasing motion (correlated to the percentage of newly rendered tiles). Hyperparameter sensitivity is low once the adaptive reference insertion mechanism is tuned appropriately.

Implications, Limitations, and Future Work

TemporalGS enables real-time interactive roaming in large, complex 3DGS scenes without retraining or extensive post-processing, facilitating deployment in VR/AR, urban navigation, and scalable visualization systems. Its training-free, modular architecture fits any 3DGS variant or hardware stack, making it a universal drop-in acceleration component.

The principal limitation is that its temporal redundancy assumption fails under very rapid camera movements or when $\eta$ is improperly set, resulting in increased tile rendering or flickering artifacts. The framework can be extended by integrating lightweight learning-based modules for artifact correction, and potentially adapted to dynamic scene rendering with 4D Gaussian Splatting or temporally coherent view synthesis in streaming volumetric video.

Conclusion

TemporalGS introduces a training-free, plug-and-play approach for 3DGS rendering acceleration, leveraging temporal priors and adaptive frame-local strategies to achieve substantial speed-up across varied scales and hardware environments. The methodology is technically rigorous, generalizable, and empirically demonstrates strong quality-efficiency trade-offs. This establishes a new baseline for practical, high-throughput, temporally coherent novel view synthesis in 3DGS systems (2607.03390).