Volumetric Spherical Gaussians

• Volumetric Spherical Gaussians are functions on the unit sphere that offer an efficient and compact model for representing volumetric color, surface, and structure in 3D scenes.
• They reduce computational and memory costs by using simple parameterizations, outperforming traditional spherical harmonics in key metrics such as speed and storage.
• They support advanced rendering techniques including multi-lobe and hybrid representations, enabling real-time neural rendering and precise 3D reconstruction.

Volumetric Spherical Gaussians (SGs) are a class of mathematical and computational primitives for representing, reconstructing, and rendering volumetric content in computer graphics and vision. By specializing the classical multidimensional Gaussian to isotropic or axis-constrained forms on spheres, SGs enable highly efficient, compact, and analytically tractable models for volumetric color, surface, and structure in 3D scenes. Recent research demonstrates their advantages as both alternatives and complements to established basis expansions such as spherical harmonics, particularly in real-time neural rendering pipelines, novel view synthesis, and 3D reconstruction.

1. Mathematical Definition and Parameterization

A Spherical Gaussian lobe is defined as a function on the unit sphere $S^2$, parameterized by a mean direction $\boldsymbol\mu \in S^2$, sharpness or bandwidth $s \geq 0$, and per-channel amplitude or color $\mathbf{a} \in \mathbb{R}^3$: $$G(\mathbf{v}; \boldsymbol\mu, s, \mathbf{a}) = \mathbf{a} \, \exp\big(s\, ( \boldsymbol\mu \cdot \mathbf{v} - 1 )\big)$$ for viewing direction $\mathbf{v}$. This unnormalized form is preferred in volumetric rendering pipelines, as the scale is absorbed in $\mathbf{a}$ and does not require an explicit normalization constant.

For practical color modeling, a view-independent diffuse term $D \in \mathbb{R}^3$ is often included: $$C(\mathbf{v}) = D + \sum_{i=1}^{n} G(\mathbf{v}; \boldsymbol\mu_i, s_i, \mathbf{a}_i)$$ with $n$ the number of SG lobes. In SG-Splatting and MEGS$^2$, $n$ is typically 1–3, trading representational flexibility for storage and evaluation cost (Wang et al., 2024, Chen et al., 7 Sep 2025).

2. Comparison with Spherical Harmonics and Gaussian Splatting

The standard in 3D Gaussian Splatting (3DGS) for representing view-dependent color is the use of low-order spherical harmonics (SH), for instance with order $L=3$. For each RGB channel, this requires $(L+1)^2$ coefficients, i.e., 48 floats per Gaussian. Computing the projected color then entails a dense linear combination over all SH basis functions.

Spherical Gaussians provide two principled advantages:

• Parameter Efficiency: A single SG lobe requires only 7 floats (amplitude $\mathbf{a}$, mean $\boldsymbol\mu$, sharpness $s$); three lobes with a diffuse term total 24 floats—half the cost of SH.
• Evaluation Cost: Color projection per Gaussian involves a single dot product, an exponential, and per-channel scalings; SH requires 48 basis evaluations and multiplications (Wang et al., 2024, Chen et al., 7 Sep 2025). Empirical ablations show a ∼47% memory reduction and ∼50% speedup in shader evaluation when SH is replaced by Spherical Gaussians (DeepBlending benchmark, SG-Splatting (Wang et al., 2024)).

3. Volumetric Rendering and Scene Representation

A typical volumetric rendering pipeline with Spherical Gaussians leverages classic volume integration: $$C(\mathbf{x}) = \sum_{i=1}^N c_i\,\alpha_i(\mathbf{x})\,\prod_{j<i}\big(1-\alpha_j(\mathbf{x})\big)$$ where $c_i$ is the SG-based color for Gaussian $i$, $\alpha_i(\mathbf{x})$ is the opacity at projected image point $\mathbf{x}$, and the front-to-back product implements the standard transmittance and blending as in neural radiance fields (NeRF) (Chen et al., 7 Sep 2025).

For edge or structure reconstruction (SGCR), the isotropic volumetric SG is simplified as

$$SG_i(x) = \exp\left( -\frac{\|x-p_i\|^2}{2 r_0^2} \right)$$

with $r_0$ fixed across all primitives (Yang et al., 7 May 2025).

4. Multi-Lobe and Hybrid Representations

Complex view-dependent effects, such as multi-peak specularities, may be poorly captured by a single lobe. Both SG-Splatting and MEGS$^2$ introduce multi-lobe schemes:

• Orthogonal Lobes: In SG-Splatting, three mutually orthogonal axes $\boldsymbol\mu_i$, with distinct amplitudes and sharpnesses $(\mathbf{a}_i, \lambda_i)$, are combined

$$C(\mathbf{v}) = D + \sum_{i=1}^3 \mathbf{a}_i\,\exp(\lambda_i(\mathbf{v} \cdot \boldsymbol\mu_i - 1))$$

Orthogonality constraints prevent degeneracy and support more accurate surface reflectance modeling (Wang et al., 2024).

• Arbitrary-Axis Lobes: MEGS$^2$ lifts the constraint, letting each $\boldsymbol\mu_i$ be free on $S^2$; soft pruning selects the minimal necessary number for accuracy (Chen et al., 7 Sep 2025).

Low-frequency errors, particularly with large smooth gradients, are addressed by a mixed scheme: combining SG lobes for high- and mid-frequency content with low-degree SH for base color, chosen adaptively per-Gaussian (Wang et al., 2024).

5. Optimization, Pruning, and Integration

SG-based color blocks can be retrofitted in existing 3DGS pipelines by least-squares fitting SGs to the trained SH colors (sampled over viewing directions), followed by scene-level fine-tuning. Geometry (means, covariances) and opacity fields are left unaltered (Wang et al., 2024).

MEGS$^2$ introduces a unified soft-pruning approach, jointly learning to zero out entire Gaussians (by opacity) and individual lobes (by sharpness) under a global memory budget constraint. An ADMM-like method alternates between continuous optimization and hard thresholding to achieve strict VRAM budgets in both static storage and active render time (Chen et al., 7 Sep 2025). Rapid pruning and compact representation enable 2× reduction in render VRAM vs. prior compressed 3DGS and >8× over vanilla 3DGS at negligible quality loss.

SGCR's pipeline exploits the spherical constraint for robust grid-based initialization, periodic densification, and two-phase optimization, yielding highly accurate edge reconstructions and enabling direct parametric curve extraction (Yang et al., 7 May 2025).

6. Quantitative Performance and Application Domains

Published results substantiate the efficiency-accuracy tradeoffs:

• SG-Splatting (3 SG + D): 29.570 PSNR, 0.901 SSIM, 0.247 LPIPS, 344 FPS, 357 MB on DeepBlending—offering a 48% rendering speedup and 46% memory cut vs. L=3 SH (Wang et al., 2024).
• MEGS$^2$: static VRAM drop from 648 MB to 416 MB on Mip-NeRF360 (35%); further pruning to 55 MB yield ≤0.02 dB PSNR drop (Chen et al., 7 Sep 2025).
• SGCR: geometric F-score 0.9260 vs 0.9210 (EMAP), Chamfer distance 0.0280 vs 0.0291 (EMAP); generation time 87 s vs 1–2 hr for image-based baselines (Yang et al., 7 May 2025).

Spherical Gaussians are thus validated for:

• Neural scene rendering in photorealistic view synthesis.
• Highly compact, real-time applications and edge deployment.
• Explicit 3D curve/edge structure reconstruction.
• Multi-lobe and hybrid modeling for physically accurate reflectance.

7. Limitations and Future Directions

Current formulations exhibit minor quality drops (∼0.1–0.2 dB PSNR) versus full SH, particularly in smooth shading regimes. Without orthogonality, lobes may collapse, causing local artifacts. Very sharp lobes ($\lambda \gg 1$) lack explicit energy normalization, risking under/overshoot near shadow discontinuities (Wang et al., 2024).

Suggested research directions include:

• Learning per-Gaussian number of lobes, possibly with anisotropic SGs.
• Improved and principled SG initialization (e.g., moment matching).
• Adaptive lobe splitting and more accurate SH-to-SG fitting.
• GPU-specific fused instructions for fast dot-exponential evaluation.
• Broad empirical evaluation for new scene classes, especially dynamic and video content (Wang et al., 2024, Chen et al., 7 Sep 2025).

A plausible implication is that the volumetric SG paradigm will continue to displace high-order SH in realistic, resource-constrained rendering and serve as a basis for increasingly explicit, structure-aware neural graphics systems.