Gaussian Splat Initialization

• Gaussian Splat Initialization is the process of positioning and shaping anisotropic Gaussian primitives to approximate true scene geometry using methods like SfM, MVS, or learning-based predictors.
• It employs geometric and depth-based priors to set initial covariances and orientations, which significantly improve fidelity and reduce reconstruction error.
• Recent strategies integrate curvature, segmentation, and SLAM-driven techniques to enhance optimization efficiency and achieve superior performance in challenging visual conditions.

Gaussian Splat Initialization is a critical stage in Gaussian Splatting-based scene representations, governing the placement, shape, and density of the initial Gaussian primitives prior to differentiable optimization. Diverse initialization schemes exist to address scene geometry, convergence speed, and downstream quality, particularly in challenging settings (e.g., limited views, difficult textures, or noisy pose/depth priors). Recent research demonstrates that initialization strategies substantially impact both the fidelity and efficiency of photo-realistic reconstruction and surface recovery.

1. Foundations of Gaussian Splat Initialization

In 3D or 2D Gaussian Splatting, scenes are represented by an explicit set of anisotropic Gaussians parameterized by a 3D mean $\mu_i$, covariance $\Sigma_i$, opacity $\alpha_i$, and either direct or view-dependent color coefficients (e.g., spherical harmonics) (Foroutan et al., 2024, Desiatov et al., 21 Mar 2026). The initialization step specifies the spatial locations, scales (covariances), and sometimes opacities of these splats, either by sampling, geometric inference, variational methods, or learned predictors. The effectiveness of downstream optimization strongly depends on how closely these initial splats approximate the true scene geometry and spatial density patterns.

Initialization schemes fall into three principal categories:

• Geometry-driven using classical reconstruction: Structure-from-Motion (SfM), Multi-view Stereo (MVS), or monocular depth/normal estimates (R. et al., 1 May 2026, Kim et al., 16 Jun 2025, Wang et al., 1 Jul 2025, Foroutan et al., 2024, Gao et al., 2 Jan 2025).
• Learning-based or neural pointwise estimation: Data-driven mappings from input imagery to dense point estimates or confidence-weighted structures (Hofer et al., 3 Feb 2026, Liu et al., 8 May 2025, Li et al., 5 Sep 2025).
• Random and regularized methods: Uniform sampling or systematic placement, sometimes paired with progressive “coarse-to-fine” regularization to mimic the benefits of geometry (Jung et al., 2024).

Initialization is distinct from densification: it prepares the initial primitive set, which is then refined and possibly expanded during gradient optimization and adaptive density control.

2. Geometry-Aware and Depth-/Normal-Guided Strategies

Surface-aware initialization leverages geometric priors to reduce coverage gaps and to place Gaussians close to the true scene manifold. 2D-SuGaR exemplifies this approach by densely seeding 2D Gaussians from dense monocular depth maps and predicted normals (e.g., Metric3Dv2), correcting for the scale ambiguity inherent to monocular inference via per-image median alignment to SfM-derived metric depths (R. et al., 1 May 2026). Each sampled image pixel is back-projected to 3D by

$$\vec{X}_i = s_i \cdot d_{\mathrm{mono}}(u_i, v_i) \cdot K^{-1}[u_i, v_i, 1]^T,$$

where $s_i$ is an image-specific scale, and $K$ is the camera intrinsic matrix.

Orientation for each surfel is initialized from the predicted normal $\tilde{N}(u_i,v_i)$, producing a local rotation $R_i$ that aligns the Gaussian’s major axes to the estimated surface tangent. In-plane scales $(\sigma_x, \sigma_y)$ either follow the local pixel footprint or are fixed to the average, and the out-of-plane scale $\sigma_z$ is set to a small value (e.g., $\Sigma_i$0) to mimic a thin surface element.

Such dense depth-guided seeding ensures coverage in textureless regions and resilience when traditional SfM fails, as empirically demonstrated by a $\Sigma_i$15% reduction in Chamfer Distance over vanilla 2DGS before mesh refinement, and up to 16% after the joint refinement stage (R. et al., 1 May 2026).

3. Multi-View Stereo, Learned, and Segmentation-Driven Initializations

Multi-view Stereo-guided pipelines (e.g., PatchMatch-MVS with consistency filtering) provide dense, reliable surface point sets that are directly used as Gaussian centers, followed by voxel-based subsampling to balance coverage and memory (Kim et al., 16 Jun 2025). MVS-based initialization demonstrates improvements of +5–8 F1 points and $\Sigma_i$2–$\Sigma_i$3 mm reduction in Chamfer Distance compared to sparse SfM, even before subsequent regularizer losses are introduced (Kim et al., 16 Jun 2025).

Deep learning-based alternatives include:

• Pointwise Dense Neural Estimators: Pi-GS integrates $\Sigma_i$4, a permutation-equivariant network that regresses dense 3D points and associated uncertainty per scene, enabling isotropic Gaussian initialization (variance $\Sigma_i$5) and reliable color bootstrapping via image projection (Hofer et al., 3 Feb 2026).
• Learned Splat Initializers: QuickSplat trains a 3D sparse UNet to infer dense voxel-encoded features from sparse SfM input, decoded into Gaussian centroids, scales, colors, and opacities; this network fills regions that typical SfM under-samples (e.g., large textureless walls), achieving an $\Sigma_i$6 speedup and halving the depth error versus 2DGS (Liu et al., 8 May 2025).

Region segmentation can further help by identifying redundant or structurally unimportant 3D clusters for downsampling. SDI-GS applies fast 2D DBSCAN on RGB space per image, cross-view label consistency to cluster points, and stratified sampling per cluster, reducing memory and training time while retaining scene fidelity (Li et al., 15 Sep 2025).

4. Covariance, Anisotropy, and Curvature-Driven Initialization

Geometry-guided covariance initialization has a direct effect on the early-stage coverage and noise robustness of the splat distribution. GeoSplat (Li et al., 5 Sep 2025) and related works move beyond simple isotropic (sphere or disc) covariances, using principal curvatures $\Sigma_i$7 and their directions $\Sigma_i$8:

• For a surface patch, the largest in-plane variance aligns with the principal direction of smallest curvature (i.e., locally flattest), while out-of-plane variance is minimized.
• Scales are computed as

$\Sigma_i$9

yielding a covariance $\alpha_i$0 aligned to the tangent-normal basis(Li et al., 5 Sep 2025).

This curvature-aligned initialization closes holes on flat regions (via large, nearly isotropic splats) and preserves detail on high-curvature zones (via tight, anisotropic splats); empirical ablations show consistent gains of $\alpha_i$1–$\alpha_i$2 dB in PSNR compared to isotropic or random scale initializations.

5. Data-Driven, SLAM, and Mixed-Primitive Pipelines

Gaussian splat initialization can be further refined using data-driven scene analysis and real-time systems:

• SLAM-Coupled Pipelines: GSO-SLAM fuses visual odometry, semi-dense depth, and image gradients, jointly fitting 2D image-plane Gaussians to spatial gradients, then solving for 3D anisotropic covariances by multi-view linear least squares, regularized by eigenvalue flooring and consistency checks (Yeon et al., 12 Feb 2026). This initialization is within a few gradient descent steps of the converged optima, doubling early convergence speed.
• Mixed-Primitive Initialization: Some recent systems extend beyond point-ellipsoid primitives, initializing from local clusters of up to three spatially and chromatically related points to instantiate Gaussian ellipses (single-vertex), lines (two-vertex), or triangles (three-vertex), with shared covariance and SH color (Qu et al., 15 Jul 2025). Vertex pruning then simplifies primitives if their structure becomes degenerate, further compacting the representation.

A summary table illustrates key differences:

Method	Center Source	Covariance Type	Special Heuristics
SfM/COLMAP	Sparse SfM	Isotropic, neighbor-based	None
2D-SuGaR	Mono depth+normals	Thin surfel, normal-aligned	Scale-alignment, DBSCAN pruning
GeoSplat	SfM/mesh + curv	Curvature-aligned aniso	KNN manifold varifold est.
QuickSplat	Learned	Learned per-voxel	Densifier/Optimizer nets
Pi-GS	Dense $\alpha_i$3	Isotropic, uncertainty	Depth/normal warping
SDI-GS	Mono+segment	Isotropic, depth-weighted	2D/3D segment sampling/prune
GSO-SLAM	VO+keyframes	Anisotropic, gradient-based	Multi-view covariance LSQ
Mixed-Primitive	SfM cluster	Shared, primitive-type	Distance/color clustering

6. Random, Relaxed, and SfM-Free Initializations

Relaxation of the classical “geometry-prior” requirement has been addressed by RAIN-GS (Jung et al., 2024) and related works, which demonstrate that even extremely sparse and random seeding (e.g., $\alpha_i$4–$\alpha_i$5 Gaussians with large variances) can achieve competitive final performance, provided the optimization pipeline enforces a progressive low-pass filtering and delayed introduction of high-frequency harmonics. Progressive reduction of rendered Gaussian support (via a filter parameter $\alpha_i$6 proportional to $\alpha_i$7) encourages the network to first fit the global low-frequency structure, facilitating “coarse-to-fine” optimization even in the absence of geometric anchors. RAIN-GS achieves +4.7 dB PSNR gains over naïve random seeding and approaches SfM-initialized results.

NeRF-based volumetric initialization (e.g., via Instant-NGP, NerfAcc) further enables dense, surface-centered Gaussian seeding. By sampling NeRF surface densities and radiances, and optionally distilling depth supervision during splatting optimization, NeRF-guided initialization achieves near or even superior PSNR/SSIM/LPIPS to standard SfM, while handling low-texture areas (Foroutan et al., 2024).

7. Critical Analysis, Ablations, and Best Practices

Systematic evaluation (e.g., (Desiatov et al., 21 Mar 2026)) demonstrates that:

• Densification is generally more critical than initialization density: robust densifiers (MCMC, IDHFR) adaptively refine geometry even from sparse initial sets, whereas naïve gradient-based splitting (AbsGS) benefits strongly from dense, accurate initialization.
• Initialization with dense priors (MVS, mono-depth, neural estimators) particularly aids generalization to under-sampled views and scene regions not well-covered in the training trajectory.
• Specialized schemes (curvature, segmentation, SLAM-driven) further enhance coverage of challenging geometries, accelerate convergence, and yield compact, high-fidelity splat clouds.
• Initialization strategy must be matched to both the target application (speed, memory, fidelity) and the densification/controller mechanism; over-provisioning with dense or redundant Gaussians offers diminishing returns once a competent densifier is present.

Ablative studies (e.g., in 2D-SuGaR (R. et al., 1 May 2026), GeoSplat (Li et al., 5 Sep 2025), GDGS (Wang et al., 1 Jul 2025), and EasySplat (Gao et al., 2 Jan 2025)) consistently report that surface-/geometry-aware initializations produce faster optimization, better coverage, and improved quantitative metrics (e.g., Chamfer distance, PSNR/SSIM/LPIPS) over random or isotropic schemes, with final gains of 5–20% in geometric accuracy depending on the downstream pipeline.

8. Conclusion

Gaussian Splat Initialization is a foundational determinant of reconstruction quality and optimization efficiency in both 2D and 3D Splatting pipelines. Modern methods exploit dense multi-view, monocular, or learning-based geometric priors, sophisticated covariance estimation (including curvature and normal guidance), and region-wise sampling/pruning to ensure optimal scene coverage and early convergence. Advances in initialization, particularly when tightly integrated with adaptive density control, have rendered Gaussian Splatting robust to challenging inputs, sparse or noisy views, and large-scale environments, establishing the importance of principled, data-driven initialization frameworks across state-of-the-art systems (R. et al., 1 May 2026, Kim et al., 16 Jun 2025, Jung et al., 2024, Desiatov et al., 21 Mar 2026, Liu et al., 8 May 2025, Hofer et al., 3 Feb 2026, Gao et al., 2 Jan 2025, Wang et al., 1 Jul 2025, Li et al., 5 Sep 2025, Li et al., 15 Sep 2025, Qu et al., 15 Jul 2025).