R2-Gaussian Tomographic Reconstruction

• R2-Gaussian is a tomographic reconstruction framework that corrects the integration bias inherent in standard 3D Gaussian splatting for unbiased CT density recovery.
• It employs an analytically correct forward model and GPU-accelerated rasterization, enabling efficient, high-fidelity volumetric reconstructions at speeds over 100 FPS.
• Empirical results show significant improvements in PSNR and SSIM metrics compared to traditional and NeRF-based methods, with potential extensions to other imaging modalities.

R2-Gaussian refers to a tomographic reconstruction framework that extends 3D Gaussian splatting (3DGS) to physically consistent, unbiased recovery of volumetric density fields from sparse X-ray computed tomography (CT) data by explicitly addressing and rectifying the inherent integration bias in standard 3DGS rendering pipelines. This approach establishes a mathematically correct forward model for line integrals of anisotropic 3D Gaussian primitives and introduces practical, GPU-accelerated rasterization and voxelization algorithms for efficient, high-fidelity reconstruction (Zha et al., 2024, Chen et al., 1 Feb 2026).

1. Background: 3D Gaussian Splatting and the Integration Bias

In conventional 3DGS, a scene is parameterized by a set of weighted, anisotropic Gaussians defined by their centers $\mathbf{p}_i$, covariances $\Sigma_i$, and weights $c_i$:

$$G^3_i(\mathbf{x}) = c_i \exp\Big(-\tfrac{1}{2} (\mathbf{x} - \mathbf{p}_i)^\top \Sigma_i^{-1} (\mathbf{x} - \mathbf{p}_i)\Big)$$

Rendering in computer vision tasks (e.g., novel view synthesis) proceeds by projecting these Gaussians to 2D via an affine transformation and compositing them using alpha blending; this approach is highly efficient on modern hardware and outperforms NeRF techniques in certain settings. However, the alpha opacity model is tailored for optical radiance accumulation, not the physical line integrals of density measured in X-ray CT. When integrating anisotropic Gaussians along rays, an additional covariance-dependent scaling—referred to as integration bias—arises, which is neglected in standard 3DGS. This results in density estimates that vary by view angle, undermining accurate tomographic reconstruction (Zha et al., 2024).

2. Forward Model and Rectification of Line Integrals

For X-ray CT, the ideal measurement at detector pixel $\mathbf{u}$ corresponds to the line integral of the attenuation field $\sigma(\mathbf{x})$ along the ray:

$$I(\mathbf{r}) = \int_{t_n}^{t_f} \sigma(\mathbf{r}(t))\,dt, \quad \mathbf{r}(t) = \mathbf{o} + t\mathbf{d}.$$

If $\sigma(\mathbf{x})$ is a sum of 3D Gaussians, analytical integration along the ray yields:

$$L_i(\mathbf{r}) = c_i\,(2\pi)^{1/2} |\Sigma_i|^{1/2} \exp\left(-\tfrac{1}{2} (\mathbf{m} - \mathbf{p}_i)^\top (\Sigma_i^{2\times2})^{-1} (\mathbf{m} - \mathbf{p}_i)\right)$$

where $\mathbf{m}$ is the 2D mean and $\Sigma_i^{2\times2}$ the upper left $2 \times 2$ block of the covariance. The crucial scaling factor $\mu_i = (2\pi)^{1/2} |\Sigma_i|^{1/2}$ is missing in the standard 3DGS, causing the observed integration bias (Zha et al., 2024, Chen et al., 1 Feb 2026).

The R2-Gaussian framework enforces that the stored $c_i$ always corresponds to the true 3D Gaussian's density at its center, and computes

$w_i = \mu_i c_i$

when projecting to 2D. Thus, the projection on the detector is:

$$I(\mathbf{u}) = \sum_{i=1}^M \tilde{K}_i(\mathbf{u}),$$

with

$$\tilde{K}_i(\mathbf{u}) = c_i (2\pi)^{1/2} |\Sigma_i|^{1/2} \exp\left(-\tfrac{1}{2}(\mathbf{u} - \mathbf{m}_i)^\top (\Sigma_i^{2\times2})^{-1}(\mathbf{u} - \mathbf{m}_i)\right).$$

This rectification strictly decouples density recovery from view angle and achieves unbiased, physically consistent tomographic measurement simulation.

3. Efficient Differentiable Implementation

R2-Gaussian implements a CUDA-accelerated, tile-based voxelizer to render the density field into discrete volumes for regularization and final output. The steps include:

• Partitioning 3D space into tiles (e.g., $8 \times 8 \times 8$).
• Culling Gaussians whose support does not intersect a tile using Mahalanobis distances.
• Within each tile, running one thread per voxel to accumulate $\sigma(\mathbf{x}) = \sum_i G^3_i(\mathbf{x})$ over relevant Gaussians.
• Storing results in the output volume and supporting backpropagation by reusing the computation graph.

The rasterization and volume assembly are fully differentiable, permitting the use of 3D spatial priors (e.g., total variation) that regularize the reconstruction. This yields computational throughput exceeding 100 FPS for volume projections and enables efficient optimization (Zha et al., 2024).

4. Optimization, Training Regime, and Regularization

Initialization is performed via a fast FDK (Feldkamp–Davis–Kress) reconstruction. High-density points are sampled for the set of Gaussian centers $\mathbf{p}_i$, neighborhood distances set $\Sigma_i$, and FDK values initialize $c_i$ (downscaled to avoid overlap artifacts). During training, all parameters $\{c_i, \mathbf{p}_i, \Sigma_i\}$ are optimized by minimizing a composite loss:

$$\mathcal{L}_{\mathrm{total}} = \|I_r - I_m\|_1 + \lambda_{\mathrm{ssim}}\mathcal{L}_{\mathrm{DSSIM}}(I_r, I_m) + \lambda_{\mathrm{tv}} \sum_{x,y,z} |\nabla \mathbf{V}(x,y,z)|$$

where $I_r$ is the rectified projection, $I_m$ is the measured one, $\lambda_{\mathrm{ssim}}=0.25$, and $\lambda_{\mathrm{tv}}=0.05$ are empirically chosen. Adam is used over 30k iterations with exponential learning rate decay; parameters with high training gradients are adaptively split to enhance reconstruction fidelity (Zha et al., 2024).

5. Empirical Performance and Comparison

On a suite of 15 CT cases (organs, insects, industrial parts), R2-Gaussian achieves:

• PSNR 38.9 dB and SSIM 0.959 in 8 minutes (30k iters, $12\times$ faster than state-of-the-art NeRF-based SAX-NeRF at 13 hours and PSNR 38.07 dB).
• Even at 2.5 minutes (10k iters), PSNR 38.29 dB, surpassing analytical and iterative baselines (e.g., SART, ASD-POCS).
• Ablation studies show that omitting rectification (i.e., using X-3DGS) reduces reconstructed-density PSNR by 7.6 dB, demonstrating the necessity of correcting the integration bias for quantitative volume recovery.
• Applying 3D total variation regularization via the differentiable voxelizer increases SSIM by ~0.02 (Zha et al., 2024).

6. Relationship to Ray-Tracing and Extensions Beyond Affine Collapse

R2-Gaussian employs a local affine approximation for efficiency, collapsing 3D Gaussians to 2D via Jacobian-based mappings and blending. However, this approach is an approximation and can introduce minor errors in line-integral consistency for highly anisotropic or oblique Gaussians. Recent work proposes replacing the affine splatting step with analytic Gaussian ray tracing, using closed-form integrals along detector rays:

$$I(o, d) = \sum_i \rho_i \sqrt{2\pi / A_i} \exp\left( -\frac{1}{2}(C_i - B_i^2 / A_i) \right)$$

where $A_i = d^\top \Sigma_i^{-1} d$, $B_i = d^\top \Sigma_i^{-1} (o - \mu_i)$, $C_i = (o - \mu_i)^\top \Sigma_i^{-1} (o - \mu_i)$ for each Gaussian. This ray-tracing generalization eliminates the affine collapse error and supports arbitrary PET/CT geometries and corrections (e.g., PET arc correction), with improved numerical accuracy at practical computational cost (Chen et al., 1 Feb 2026).

7. Broader Implications and Future Directions

By explicitly identifying and rectifying the integration bias in Gaussian-based volumetric rendering, R2-Gaussian enables unbiased, physically meaningful tomographic density reconstruction at competitive speed and accuracy. The analytic framework is readily extensible—a plausible implication is that any task requiring integrals over learned density fields (e.g., MRI reconstruction, volumetric surface extraction, simulation of 4D time-varying densities) potentially benefits from this bias correction.

Open future work includes extension to modalities with complex scanning geometries, non-X-ray imaging, calibration-error handling, limited-angle scan extrapolation, and generalized rectified splatting for other scientific domains (Zha et al., 2024, Chen et al., 1 Feb 2026).

---

Key references:

• "R$^2$-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction" (Zha et al., 2024)
• "Radioactive 3D Gaussian Ray Tracing for Tomographic Reconstruction" (Chen et al., 1 Feb 2026)