F4Splat: Feed-Forward Predictive Densification for Feed-Forward 3D Gaussian Splatting

Abstract: Feed-forward 3D Gaussian Splatting methods enable single-pass reconstruction and real-time rendering. However, they typically adopt rigid pixel-to-Gaussian or voxel-to-Gaussian pipelines that uniformly allocate Gaussians, leading to redundant Gaussians across views. Moreover, they lack an effective mechanism to control the total number of Gaussians while maintaining reconstruction fidelity. To address these limitations, we present F4Splat, which performs Feed-Forward predictive densification for Feed-Forward 3D Gaussian Splatting, introducing a densification-score-guided allocation strategy that adaptively distributes Gaussians according to spatial complexity and multi-view overlap. Our model predicts per-region densification scores to estimate the required Gaussian density and allows explicit control over the final Gaussian budget without retraining. This spatially adaptive allocation reduces redundancy in simple regions and minimizes duplicate Gaussians across overlapping views, producing compact yet high-quality 3D representations. Extensive experiments demonstrate that our model achieves superior novel-view synthesis performance compared to prior uncalibrated feed-forward methods, while using significantly fewer Gaussians.

• The paper demonstrates a novel feed-forward predictive densification method that adaptively allocates Gaussian primitives for efficient 3D reconstruction.
• It utilizes a multi-scale transformer-based encoder and gradient-driven supervision to optimize Gaussian distribution based on scene complexity.
• Experiments on RE10K and ACID benchmarks show state-of-the-art results using only 10–28% of the Gaussian count compared to prior methods.

F4Splat: Feed-Forward Predictive Densification for Controllable, Efficient, and High-Fidelity Feed-Forward 3D Gaussian Splatting

Introduction

F4Splat addresses a persistent inefficiency in feed-forward 3D Gaussian Splatting (3DGS) by introducing a mechanism for spatially adaptive, predictive densification of Gaussians from uncalibrated multi-view image collections. Unlike prior pipelines based on rigid uniform allocation (pixel or voxel level), which result in redundant Gaussian primitives and inflexible scaling, F4Splat replaces this paradigm with a densification-score-guided allocation framework. This mechanism enables both explicit control over the Gaussian budget at inference and compact, high-fidelity 3D reconstructions—even under stringent resource constraints. This essay presents a detailed exposition of the proposed architecture, optimization strategy, experimental validation, and implications.

Limitations of Uniform Allocation in Feed-Forward 3DGS

Existing feed-forward 3DGS methods leverage supervised pre-training to perform scene reconstruction in a single forward pass from sparse views. However, their standard uniform pixel-to-Gaussian or voxel-to-Gaussian pipelines create significant inefficiencies: over-allocation in homogeneous regions and duplicate primitives across overlapping images, leading to suboptimal utilization of the Gaussian budget. AnySplat allows for partial Gaussian-count control via voxel size adjustment but still exhibits spatial uniformity and requires retraining for different resolutions or budgets. The inability to adaptively allocate Gaussians according to fine-grained scene content and input coverage remains a major bottleneck in achieving both compact and expressive 3D representations.

Adaptive Densification via Predictive Densification Scores

F4Splat proposes a feed-forward architecture that directly predicts per-region densification scores for selective Gaussian allocation. The concept is formalized as a controllable mapping:

$$(\{\mathbf{g}_g\}_{g=1}^{N_{\mathcal{G}}},\ \{ \hat{\mathbf{P}}_i \}_{i=1}^{N_{\text{ctx}}}) = \mathbf{F}_\theta(\{\mathbf{I}^\text{ctx}_i\}_{i=1}^{N_{\text{ctx}}},\ \bar{N}_\mathcal{G})$$

where the target Gaussian budget $\bar{N}_\mathcal{G}$ can be efficiently enforced at inference time without retraining.

Given a set of uncalibrated context images, the network predicts dense multi-scale Gaussian parameter maps, pose estimates, and densification score maps. These scores reflect both geometric complexity and view redundancy, ascertained by learning from gradient-based signals derived from the rendering loss. Gaussian primitives are spatially allocated by thresholding the densification map, adaptively concentrating representational capacity on complex or underdetermined regions and minimizing duplication in overrepresented areas.

Figure 1: Comparison of Gaussian allocation under different target Gaussian budgets; adaptive densification preserves detail in challenging areas without over-allocating to simple regions.

Figure 2: Overview of the F4Splat pipeline, showing joint camera, Gaussian parameter, and densification score prediction, with adaptive budget enforcement.

Architecture and Training Pipeline

The pipeline consists of:

• Geometry Backbone: A multi-view transformer-based encoder (initialized from VGGT) extracts geometric features and predicts camera poses from uncalibrated inputs.
• Multi-Scale Prediction: Hierarchical feature maps feed into parallel parameter heads for Gaussian center, parameters, and densification scores, enabling resolution-adaptive representation.
• Spatially Adaptive Allocation: Masking and thresholding the densification maps at multiple scales ensures non-overlapping, compact allocations consistent with the target budget.

The densification supervision is grounded in the homodirectional rendering gradient with respect to scene parameters—mirroring adaptive density control in optimization-based 3DGS. During training, log-normed gradients on the rendering loss provide supervisory signals for learning densification score prediction. Training leverages aligned novel views for robust generalization and camera pose prediction incorporates explicit similarity alignment to compensate for coordinate system drift.

Figure 3: Densification score maps indicate where additional Gaussian primitives are allocated, capturing detail and minimizing redundancy across input context images.

Figure 4: Computation of the rendering-loss-based gradient that supervises densification score prediction in a feed-forward, amortized manner.

Experimental Validation

Experiments were conducted on RealEstate10K (RE10K) and ACID benchmarks, focusing on single- and multi-view uncalibrated scenarios. Results indicate robust generalization across datasets and input conditions.

Key findings:

• F4Splat achieves state-of-the-art or competitive PSNR/SSIM/LPIPS in uncalibrated and pose-free regimes using only 10–28% the number of Gaussians required by prior art (e.g., AnySplat, VicaSplat), demonstrating a substantial increase in allocation efficiency. As an example, F4Splat matches or exceeds the rendering quality of baselines at 1/4 Gaussian budget and maintains high performance even with two-view inputs.
• Qualitative comparisons reveal sharper geometries and superior preservation of fine structure in complex regions, with a notable reduction in blurring and redundancy.

  Figure 5: Qualitative comparisons of novel view synthesis on RE10K; F4Splat matches or surpasses baseline quality using dramatically fewer Gaussians.

  

  Figure 6: Additional qualitative results show that adaptive allocation consistently yields high-fidelity renderings under tight Gaussian constraints.

Ablation studies confirm the necessity of learned densification scoring, level-wise supervision, and scene-scale regularization for both stability and performance. Adopting random or simple frequency-based allocations degrades metrics significantly, highlighting the utility of gradient-based, scene-aware densification as learned by F4Splat.

Theoretical and Practical Implications

By reframing Gaussian allocation as a predictive, budget-controllable, and spatially adaptive process, F4Splat bridges the gap between highly efficient optimization-based methods and real-time, single-pass, generalizable pipelines. Its architecture provides explicit budget control, a property critical for scalable deployment on resource-limited platforms or for applications demanding strict quality-resource trade-offs (e.g., mobile AR/VR, large-scale map fusion).

The predictive densification score approach enables amortization of adaptive density allocation, hitherto only accessible through iterative optimization schemes, in a fully differentiable manner. This marks a key step towards generalizable, controllable photo-realistic view synthesis systems that remain tractable.

Future Directions

Potential extensions include integrating semantic-aware densification, dynamic budget adaptation based on hardware profiles, synergistic combination with 3DGS compression and pruning frameworks, and joint optimization with higher-level scene understanding tasks. Furthermore, unsupervised or weakly-supervised adaptation of densification policies to domains with limited annotated data remains an open research question. These lines of inquiry could further increase both the fidelity and efficiency of real-time 3D scene understanding in unstructured environments.

Conclusion

F4Splat presents a technically robust, feed-forward framework for adaptive, predictive, and compact 3D Gaussian Splatting from sparse, uncalibrated images. By learning a densification-score-guided allocation policy, F4Splat achieves explicit, fine-grained control over the Gaussian budget, yielding high-fidelity novel view synthesis with minimal redundancy or quality loss. Its numerical and qualitative performance validates spatially adaptive allocation as a superior alternative to uniform assignment for compact 3D scene representations, establishing a strong foundation for subsequent developments in efficient neural graphics primitives and controllable scene reconstruction pipelines (2603.21304).