Differentiable Mesh Splatting Renderer

• Differentiable mesh splatting renderer is a framework that uses explicit mesh primitives and soft splatting kernels for efficient end-to-end gradient propagation.
• It applies a differentiable soft indicator and volumetric compositing to optimize vertex positions, appearance, and even mesh topology through multi-stage training.
• The approach outperforms point-based and Gaussian methods in photorealism and efficiency, making it ideal for real-time graphics, simulation, and precise surface reconstruction.

A differentiable mesh splatting renderer is a class of rendering framework that exploits both the geometric fidelity of surface meshes and the efficiency and gradient propagation of splatting-based image formation. These renderers realize truly end-to-end optimization: surface geometry and appearance, represented as explicit mesh primitives (often triangles), are rendered using soft, differentiable splatting kernels or pseudo-volumetric compositing, enabling gradients to flow from image-level losses directly to mesh geometry and attributes via GPU-friendly algorithms. This paradigm closes the historical gap between volumetric field rendering and classic mesh graphics, offering photorealistic view synthesis, precise surface reconstruction, and compatibility with real-time or physics-ready assets (Guédon et al., 30 Jun 2025, &&&1&&&, Sheng et al., 23 Jun 2025, Held et al., 7 Dec 2025, Held et al., 25 May 2025, Zhang et al., 29 Jan 2026, Lin et al., 2024, Gu et al., 2024, Cole et al., 2021, Guo et al., 1 Dec 2025, Ma et al., 2024, Byrski et al., 15 Mar 2025).

1. Fundamental Principles and Mathematical Formulation

Differentiable mesh splatting renderers generalize the splatting paradigm by applying it to mesh-based primitives—chiefly triangles—rather than point clouds or volumetric grids. Each triangle (or mesh facet) is equipped with: 3D vertex positions, color/appearance attributes (e.g., spherical harmonics coefficients), an opacity, and a “softness” parameter controlling the spatial extent of its splat.

The core operation projects each triangle onto the image plane, computes a differentiable coverage mask via a soft indicator or window function, and blends the triangle's color with prior covered content using volumetric alpha compositing. The central mathematical forms include:

• Triangle Splat Window (projected SDF-based):

$$I(p) = \Bigl[\mathrm{ReLU}\bigl(\phi(p) / \phi(s)\bigr)\Bigr]^{\sigma}$$

where $\phi(p)$ is the signed distance from pixel $p$ to the triangle edge set, $s$ is the incenter in 2D, and $\sigma$ modulates boundary smoothness. For 2DTS (Sheng et al., 23 Jun 2025), the “eccentricity” function $e_i(x)$ based on barycentric coordinates replaces the SDF.

• Compositing:

$$C(p) = \sum_{n=1}^N [\,c_{T_n}\,o_{T_n}\,I_n(p)\,] \prod_{i=1}^{n-1} [1 - o_{T_i}\,I_i(p)]$$

for triangles sorted (typically) by depth, with $c_{T_n}$ the (potentially barycentrically interpolated) color, $o_{T_n}$ the per-triangle opacity, and $I_n(p)$ the soft mask.

All operations—projection, mask evaluation, compositing—are differentiable with respect to vertex positions, appearance, sharpness, and even mesh topology (if adaptively remeshed).

2. Architecture and Optimization Pipeline

All practical mesh splatting frameworks follow a multi-stage optimization process combining mesh topology management, soft-to-hard solidification, and volumetric rendering:

• Initialization: The mesh is seeded via Delaunay triangulation, dense MVS, or extracted from a volumetric SDF field (e.g., via Marching Tetrahedra) (Lin et al., 2024, Guédon et al., 30 Jun 2025, Held et al., 29 Sep 2025, Zhang et al., 29 Jan 2026).
• Stage 1 (Soft Triangles/Soup): The mesh consists of unconnected or loosely connected triangles with low initial opacity and large splat bandwidth/softness (Held et al., 29 Sep 2025, Held et al., 7 Dec 2025, Held et al., 25 May 2025).
• Stage 2 (Connectivity and Opaqueness): Connectivity is enforced by shared-vertex parameterization or Delaunay-based extraction; opacity and splat sharpness are annealed toward 1 and near-discrete (i.e., solid) values (Held et al., 25 May 2025, Held et al., 29 Sep 2025).
• Stage 3 (Fine Tuning): Densification (via midpoint subdivision of high-content triangles), pruning (removal of low-opacity or under-contributing faces), and regularization (surface consistency, Laplacian smoothness, normal alignment) drive the mesh toward well-conditioned, artifact-free geometry (Held et al., 29 Sep 2025, Held et al., 7 Dec 2025, Zhang et al., 29 Jan 2026).

The full training pseudocode for Triangle Splatting+, for example, involves batching over multi-view images, per-pixel splatting and compositing, backpropagation of photometric and perceptual losses (e.g., DSSIM), and mesh regularization, with explicit scheduling for the window sharpness parameter $\sigma$, opacity floors, and mesh densification (Held et al., 29 Sep 2025, Sheng et al., 23 Jun 2025).

3. Gradients, Differentiability, and Mesh-Parameter Backpropagation

Mesh splatting renderers deliver true differentiability with respect to geometric (vertex positions), appearance, and compositing parameters through:

• Analytic derivatives of the splat indicator function with respect to vertex positions via edge-normal parametrization and signed distance field (SDF) calculus:

$$\frac{\partial I_k}{\partial \text{vertex}} = \frac{\partial I_k}{\partial \phi} \frac{\partial \phi}{\partial \text{vertex}}$$

with $$\frac{\partial I_k}{\partial \phi} = \sigma\,\mathrm{ReLU}(\phi / \phi(s))^{\sigma-1}\, (1/\phi(s))$$ and $\frac{\partial \phi}{\partial \text{vertex}}$ tracing edge-normal updates.

• Chain rule accumulation through depth-sorted compositing, distributing gradients from $$\partial C/\partial o_{T_n}, \partial C/\partial c_{T_n}$$ to all referenced vertices and appearance parameters.
• In hybrid volumetric-mesh pipelines, gradients also flow from soft pseudo-volumetric layers (as in mesh softening (Zhang et al., 29 Jan 2026)) or from SDF-learned fields (as in Mesh-in-the-Loop Gaussian Splatting (Guédon et al., 30 Jun 2025)) to mesh vertex positions, face connectivity, and associated appearance.
• Mesh-connected methods (e.g., continuous remeshing, Laplacian smoothing) further propagate gradients through remeshing operations, though most schemes treat connectivity updates as non-diffable and freeze topology after a certain point.

In all cases, the differentiable renderer may leverage custom CUDA kernels for splatting, compositing, and sorting, or rely on auto-diff frameworks (PyTorch/TensorFlow) coupled to personalized rasterizers or OpenGL-primitive backends (Held et al., 7 Dec 2025, Guo et al., 1 Dec 2025, Cole et al., 2021, Byrski et al., 15 Mar 2025).

4. Comparison to Alternative Splatting and Mesh Reconstruction Methods

Differentiable mesh splatting outperforms point or Gaussian-based splatting in several dimensions:

• Rendering and Reconstruction Quality: Mesh-based approaches provide crisp geometric boundaries, support for fine geometric details, and improved photorealism metrics (higher PSNR, SSIM, and better LPIPS) without the blurring or tailing artifacts from Gaussian models (Sheng et al., 23 Jun 2025, Held et al., 25 May 2025, Held et al., 7 Dec 2025).
• Mesh Usability: Resulting meshes are readily importable into standard AR/VR, simulation, and real-time graphics engines, supporting depth buffering, occlusion culling, and physics, whereas Gaussian clouds lack direct pipeline compatibility (Held et al., 7 Dec 2025, Guédon et al., 30 Jun 2025).
• Efficiency: Training times are as low as 20–50 minutes (e.g., MeshSplatting: 48 min, Triangle Splatting+: 25–39 min on standard benchmarks); exported meshes are lighter-weight and easier to densify or prune (Held et al., 7 Dec 2025, Held et al., 29 Sep 2025).
• Memory and Scalability: Mesh representations are more memory-efficient for complex scenes and avoid the explosion in the number of primitives needed for volumetric fidelity (Guédon et al., 30 Jun 2025, Held et al., 25 May 2025).
• Novelty and Control: The explicit mesh structure allows scene deformation, editing, physics simulation, and targeted mesh manipulation, as in REdiSplats, MaGS, and TagSplat frameworks (Byrski et al., 15 Mar 2025, Ma et al., 2024, Guo et al., 1 Dec 2025).

The transition from volumetric or point-based splatting to mesh-guided splatting (including mesh-softened volumetric approaches such as in (Zhang et al., 29 Jan 2026)) enables direct geometry extraction and superior downstream usability, with maintained real-time rendering throughput.

5. Specialized Variants and Hybrid Frameworks

Several research directions have extended basic mesh splatting to support additional geometry and appearance modeling:

• Pseudo-volumetric and Mesh Softening: Mesh Splatting (Zhang et al., 29 Jan 2026) "softens" a base mesh into multiple, thin, semi-transparent offset layers, enabling volumetric rendering with mesh-centric differentiability and improved stability for inverse rendering.
• Mesh-adsorbed and Deformable Gaussians: MaGS binds splats to mesh faces with flexible barycentric constraints, supporting non-rigid deformations and compatibility with simulation priors (ARAP, SMPL, physics engines) (Ma et al., 2024).
• Topology-aware Dynamic Mesh Splatting: TagSplat maintains connectivity-aware Gaussian splats for dynamic mesh modeling, incorporating densification, pruning, temporal coherence, and differentiable mesh rasterization, providing robust 4D keypoint tracking (Guo et al., 1 Dec 2025).
• Hybrid Volumetric-Mesh Extraction: MILo performs differentiable Delaunay triangulation and Marching Tetrahedra over Gaussian-driven pivots at each iteration, ensuring mesh and volumetric field consistency (Guédon et al., 30 Jun 2025).
• Tetrahedron Splatting: TeT-Splatting generalizes the differentiable splatting renderer to deformable tetrahedral grids, supporting structured mesh extraction with real-time rasterization and precise, robust mesh outputs (Gu et al., 2024).

These approaches combine the strengths of both explicit connectivity (required for animation, kinematic priors, and simulation) and soft, robust view synthesis inherited from volumetric splatting.

6. Quantitative Performance and Experimental Outcomes

The following table summarizes key empirical results reported in the literature for differentiable mesh splatting and selected competitors:

Method	Mesh-based PSNR↑	SSIM↑	LPIPS↓	Chamfer (DTU)↓	Training Time	#Vertices	FPS@HD
MeshSplatting (Held et al., 7 Dec 2025)	24.78 (Mip-N360)	0.728	0.310	Best-5/15 scenes	48 min	3 M	220
Triangle Splat+ (Held et al., 29 Sep 2025)	25.21 (Mip-N360)	0.742	0.294	—	25–39 min	2 M	400
MILo (dense) (Guédon et al., 30 Jun 2025)	24.09 (Mip-N360, MiLo)	0.688	0.323	0.68	110 min	7 M	170
2DTS (Sheng et al., 23 Jun 2025)	25.58 (Mip-N360-Outdoor)	—	—	0.0519	28–131 min	68–5800K	—
Mesh Splatting (Zhang et al., 29 Jan 2026)	0.62 (DTU, Chamfer cm)	—	—	0.62	12–23 min	0.3 M	—

Reported results demonstrate that differentiable mesh splatting achieves superior perceptual quality, lower mesh complexity, and faster training than prior volumetric or point-based methods, with smooth integration into application pipelines (Held et al., 7 Dec 2025, Held et al., 29 Sep 2025, Zhang et al., 29 Jan 2026).

7. Limitations, Open Challenges, and Outlook

Despite state-of-the-art results, current differentiable mesh splatting methods exhibit several intrinsic limitations:

• Background Completion: Mesh construction depends on input point cloud or MVS completeness; sparse or occluded background regions may be under-filled (Held et al., 7 Dec 2025).
• Transparency and Semi-conducting Materials: Fully opaque splatting cannot reproduce semi-transparent phenomena such as glass and water (Held et al., 29 Sep 2025).
• Non-manifoldness and Watertightness: Meshes are not always watertight; minor non-manifold patches or small artifacts may persist, motivating further regularization (Zhang et al., 29 Jan 2026, Sheng et al., 23 Jun 2025).
• Dynamic Scenes: While extensions exist for dynamic geometry and 4D reconstructions, maintaining temporal/topological coherence across frames introduces additional regularization and computational demands (Guo et al., 1 Dec 2025).
• Visibility Sorting: Standard centroid-based depth sorting can result in pop artifacts under viewpoint changes; per-pixel or hierarchical sorting remains an area of research (Held et al., 25 May 2025).

Future directions include joint learning of per-triangle textures or BRDFs, watertightness constraints and mesh topology priors, hybrid implicit/explicit background modeling, and further bridging of mesh splatting with real-time simulation, ray tracing, and VR/AR application pipelines (Held et al., 7 Dec 2025, Held et al., 25 May 2025, Byrski et al., 15 Mar 2025).

---

Key References:

(Held et al., 25 May 2025, Held et al., 29 Sep 2025, Held et al., 7 Dec 2025, Sheng et al., 23 Jun 2025, Guédon et al., 30 Jun 2025, Lin et al., 2024, Zhang et al., 29 Jan 2026, Byrski et al., 15 Mar 2025, Ma et al., 2024, Guo et al., 1 Dec 2025).