UTrice: Unifying Primitives in Differentiable Ray Tracing and Rasterization via Triangles for Particle-Based 3D Scenes

Abstract: Ray tracing 3D Gaussian particles enables realistic effects such as depth of field, refractions, and flexible camera modeling for novel-view synthesis. However, existing methods trace Gaussians through proxy geometry, which requires constructing complex intermediate meshes and performing costly intersection tests. This limitation arises because Gaussian-based particles are not well suited as unified primitives for both ray tracing and rasterization. In this work, we propose a differentiable triangle-based ray tracing pipeline that directly treats triangles as rendering primitives without relying on any proxy geometry. Our results show that the proposed method achieves significantly higher rendering quality than existing ray tracing approaches while maintaining real-time rendering performance. Moreover, our pipeline can directly render triangles optimized by the rasterization-based method Triangle Splatting, thus unifying the primitives used in novel-view synthesis.

• The paper presents a novel framework that unifies differentiable ray tracing and rasterization by using triangles as direct, optimizable primitives.
• It replaces proxy geometry with a triangle window function that supports efficient gradient propagation and BVH construction for particle-based scenes.
• Empirical results on datasets like Mip-NeRF 360 demonstrate improved perceptual quality and competitive performance compared to existing methods.

UTrice: Unifying Differentiable Ray Tracing and Rasterization via Triangle Primitives for Particle-Based 3D Scenes

Introduction

The paper "UTrice: Unifying Primitives in Differentiable Ray Tracing and Rasterization via Triangles for Particle-Based 3D Scenes" (2512.04421) presents a differentiable rendering framework leveraging triangles as unified primitives for both rasterization and ray tracing in particle-based 3D scene optimization. This contrasts with prior approaches such as 3D Gaussian Splatting (3DGS), 2D Gaussian Splatting (2DGS), Triangle Splatting (3DTS), and 3D Gaussian Ray Tracing (3DGRT), which depend on Gaussian kernels and proxy geometries leading to increased overhead and system complexity.

UTrice establishes a pipeline where triangles are directly used as optimizable rendering primitives, eliminating the need for intermediate geometry representations during BVH construction and intersection testing. The framework is compatible with both rasterization and ray tracing-based optimization flows, facilitating high-quality novel-view synthesis, physically-based rendering effects, and generalization to complex camera models.

Figure 1: Overview of the UTrice pipeline, showing triangle initialization from SfM point clouds, BVH construction in OptiX, and iterative optimization via differentiable ray tracing.

Methodology

Differentiable Triangle Primitives

Central to UTrice is the usage of differentiable triangles. Each triangle is parameterized by its three vertices, spherical harmonics-encoded color, opacity, and a smoothness factor $\sigma$. The triangle window function establishes a smooth spatial response over the interior, controlled by $\sigma$; low values yield opaque behavior, while higher $\sigma$ facilitates sensitivity to shape variations, crucial for gradient propagation and optimization.

Figure 2: Effect of $\sigma$ on the triangle window function, illustrating differentiability with respect to geometry for varying interior-point sensitivity.

The window function is defined in world space, ensuring compatibility with rasterization-optimized primitives and enabling seamless integration with 3DTS-derived triangles. The response at a given intersection is computed via:

$$I(\mathbf{p}) = \mathrm{ReLU} \left( \frac{\phi(\mathbf{p})}{\phi(\mathbf{s})} \right)^{\sigma}$$

where $\mathbf{s}$ is the triangle incenter, and $\phi(\mathbf{p})$ measures the signed distance from the point to triangle edges.

Ray Tracing Pipeline

The ray tracing component utilizes OptiX for BVH construction and triangle intersection queries. Unlike 3DGRT, which requires proxy geometry for particle support, UTrice builds the BVH using triangle vertices and indices directly, simplifying both performance and integration. The k-closest intersection buffer aids in compositing semi-transparent primitives along each ray direction, and custom CUDA kernels provide efficient gradient computation for triangle optimization.

Figure 3: Compatibility demonstration: Triangle primitives from rasterization-based optimization can be directly ray-traced with no additional preprocessing.

This design enables UTrice to accommodate general camera and lens models, as rays are supplied externally, not dictated by a pinhole projection. The approach is extensible to modalities such as LiDAR simulation.

Gradient Propagation and Optimization

Triangle optimization is formulated via backpropagation through the triangle window function, enabling gradients to update vertex positions and shape parameters. The framework implements stable gradient chains from ray response to edge normals (see Eq. 5), enabling rotation and scaling of triangles toward minimizing pixel-wise rendering loss.

Figure 4: Visualization of gradient propagation, showing how vertex updates reflect triangle rotation and scaling aligned to ground-truth geometry.

Pruning and densification are managed via opacity thresholds, transmittance-weighted window responses, and view hit counts. The authors introduce a novel world-space occlusion metric accounting for camera-triangle distance during MCMC-based densification, resolving sizing biases inherent to image-space measures.

Figure 5: Occlusion comparison: Triangles of differing sizes at varying distances exhibit similar occlusion contributions under world-space occlusion metric.

Empirical Results

Rendering Quality and Quantitative Benchmarks

UTrice is evaluated on Mip-NeRF 360 and Tanks and Temples datasets, comparing against 3DGRT, 3DGS, 2DGS, and 3DTS. On perceptual metrics (LPIPS), UTrice demonstrates consistent improvements over 3DGRT, showing $\approx 15\%$ lower LPIPS values, especially in regions of fine detail and challenging lighting. SSIM scores are also competitive, with negligible differences in PSNR attributed to Gaussian kernel smoothing artifacts in competing methods.

Figure 6: Novel-view synthesis results showing better detail preservation and reduced smoothing artifacts compared to 3DGRT, particularly in high-frequency regions.

Triangle-based rasterization approaches (3DTS) have comparable PSNR but suffer in color fidelity and detail under ray tracing. UTrice’s differentiable triangle tracer produces sharper textures and avoids color noise in distant or glass-laden regions.

Figure 7: Novel-view color comparison. 3DTS produces noisy colors, but UTrice maintains consistency with ground truth under ray tracing.

Rendering Performance

With quality-oriented configurations, UTrice achieves near real-time speeds on high-end GPUs, approximately $30\%$ slower than 3DGRT but with unoptimized code paths. Further engineering optimizations are anticipated to close this gap.

Ablation Studies

Removal of individual loss terms ($\mathcal{L}_n$, $\mathcal{L}_s$) causes minor quality degradation, while disabling world-space occlusion and view pruning fundamentally disrupts convergence and optimization stability. This validates the necessity of these contributions for robust training and memory efficiency.

Ray Tracing Effects and Generalization

Once the scene is optimized using triangle primitives, UTrice can render physically-based effects such as depth of field, environment lighting, reflection, and refraction with minimal system complexity.

Figure 8: Ray tracing effects, including environment lighting, depth of field, reflection, and refraction enabled by optimizable triangle primitives.

Implications and Future Directions

UTrice offers direct compatibility between rasterization and ray tracing, lowering the barrier to integrating advanced rendering effects into scene optimization pipelines and bridging the gap between real-time and physically-based rendering in differentiable frameworks.

On the practical side, triangle primitives simplify BVH construction and memory management for complex scene geometry. Theoretical implications include the prospect of unified radiance field representations amenable to both hardware-accelerated rasterization and differentiable ray-based rendering. Applications in robotics, autonomous driving (LiDAR simulation), and real-time view synthesis for AR/VR are direct beneficiaries.

Key limitations include increased primitive count, loss of mesh connectivity leading to redundant vertex storage and higher memory usage, and lack of robust handling for degenerate/excessive triangles. Future work should address connected mesh optimization, primitive reduction, and pipeline robustness, which could foster further scalability and efficiency in differentiable rendering.

Conclusion

UTrice advances differentiable 3D scene optimization by using triangles as unified primitives across rasterization and ray tracing. The framework eliminates proxy geometry, simplifies BVH construction, and enables high-fidelity, generalizable rendering with support for complex camera models. Empirical results demonstrate improved perceptual quality and competitive rendering speeds versus prior particle-based ray tracing approaches. UTrice provides a foundation for further research into unified, scalable, differentiable rendering pipelines with practical extensions to dynamic, non-rigid, and multi-modal scene representations.