Forget Superresolution, Sample Adaptively (when Path Tracing)

Abstract: Real-time path tracing increasingly operates under extremely low sampling budgets, often below one sample per pixel, as rendering complexity, resolution, and frame-rate requirements continue to rise. While super-resolution is widely used in production, it uniformly sacrifices spatial detail and cannot exploit variations in noise, reconstruction difficulty, and perceptual importance across the image. Adaptive sampling offers a compelling alternative, but existing end-to-end approaches rely on approximations that break down in sparse regimes. We introduce an end-to-end adaptive sampling and denoising pipeline explicitly designed for the sub-1-spp regime. Our method uses a stochastic formulation of sample placement that enables gradient estimation despite discrete sampling decisions, allowing stable training of a neural sampler at low sampling budgets. To better align optimization with human perception, we propose a tonemapping-aware training pipeline that integrates differentiable filmic operators and a state-of-the-art perceptual loss, preventing oversampling of regions with low visual impact. In addition, we introduce a gather-based pyramidal denoising filter and a learnable generalization of albedo demodulation tailored to sparse sampling. Our results show consistent improvements over uniform sparse sampling, with notably better reconstruction of perceptually critical details such as specular highlights and shadow boundaries, and demonstrate that adaptive sampling remains effective even at minimal budgets.

• The paper introduces a stochastic adaptive sampling method that non-uniformly allocates path tracing samples to enhance performance at sub-1-spp regimes.
• It details an integrated pipeline combining a sampler, renderer, and gather-based denoiser with learned demodulation and tonemapping-aware perceptual losses.
• Quantitative evaluations demonstrate significant improvements in PSNR and perceptual quality over uniform sampling and superresolution methods at low sampling budgets.

Adaptive Sampling Versus Superresolution in Real-Time Path Tracing

Motivation and Problem Statement

Advances in real-time path tracing are outpacing traditional rasterization pipelines, but tight computational budgets mean that rendering often occurs at below one sample per pixel (spp). Existing production pipelines solve this problem with superresolution, rendering at reduced spatial resolution and upscaling, which uniformly sacrifices detail and lacks explicit noise-awareness. Monte Carlo noise, reconstruction difficulty, and perceptual importance are spatially heterogeneous, especially in path-traced scenes. The paper "Forget Superresolution, Sample Adaptively (when Path Tracing)" (2602.08642) proposes abandoning uniform superresolution in favor of adaptive sparse sampling–allocating path tracing samples non-uniformly on pixels where they matter most–and introduces a full pipeline for end-to-end learning of such allocations at extreme sparsity.

Pipeline Overview and Innovations

The adaptive pipeline consists of a sampler network, renderer, and denoiser (Figure 1). The sampler uses GBuffer features and temporal feedback to predict a continuous sample density map. After stochastic discretization, the renderer path traces according to this density, producing sparse and noisy pixel data. The denoiser, which shares latent features with the sampler, reconstructs the final output using a pyramidal gather filter and learned demodulation, with the reconstructed (tonemapped) image fed back to the sampler in subsequent frames. Tonemapping-aware perceptual losses are used in training to prioritize allocation of samples in regions with high perceptual visibility.

Figure 1: Overview of the adaptive pipeline, from GBuffer and previous frame inputs, through the sampler network, stochastic allocation, rendering, denoising, tonemapping, and perceptual feedback.

The crux of the approach is a stochastic formulation for sample placement: pixel-wise sample densities are interpreted as probabilities, making allocation differentiable at sub-1-spp. Previous deterministic rounding methods fail at low budgets due to gradient sparsity and bias; the new stochastic method enables robust gradient flow, as visualized and compared in Figure 2.

Figure 2: Visual analysis of the relaxed estimator under varying sampling probabilities and temperature parameters, demonstrating gradient stability and faithful approximation to the true expected gradient.

Differentiable Sparse Adaptive Sampling

The estimator reframes sample allocation as an exploration-exploitation problem, leveraging stochastic rounding rather than deterministic thresholds. For a fractional density $s_i$, an extra sample is allocated with probability $p_i$ (where $p_i = s_i - \lfloor s_i \rfloor$). The estimator normalizes by the predicted density, maintaining unbiasedness and allowing backpropagation. A relaxed approximation, using a clipped linear ramp (temperature parameter $\lambda$), further enables efficient computation and avoids the pitfalls of Gumbel-Softmax relaxations, preserving statistical properties required in Monte Carlo settings.

Gradient comparison demonstrates that the relaxed estimator achieves stability and accuracy superior to finite difference and straight-through approaches, mitigating biases and regularization mismatches.

Tonemapper-Aware Training and Perceptual Loss Formulation

Classic per-pixel losses in HDR space (L1, L2, SMAPE) are misaligned with human perception and exacerbate oversampling or undersampling artifacts. The paper introduces a family of differentiable, parametric tone mapping operators (TMO) and employs modern perceptual losses (MILO) computed in the tonemapped sRGB domain. The TMO shapes are highly expressive (Figure 3) and allow backpropagation through the entire pipeline. This boosts training stability and sample allocation alignment with visual salience.

Figure 3: Differentiable tone-mapping operator curves parameterized by $s$ (toe) and $h$ (shoulder), showing compression of luminance tails as a function of log radiance.

Denoising Architecture: Gather Pyramid and Learned Demodulation

An architectural shift from scattering to gathering kernels in pyramidal denoising filters ensures robustness in the sparse regime. Each output pixel gathers from neighbors, distributing the reconstruction task evenly regardless of sampling density (as evidenced in Figure 4). Furthermore, the denoiser learns its own demodulation maps, allowing texture preservation and geometric shading cues even when albedo decomposition from the renderer is imprecise or unavailable.

Figure 4: Learned demodulation preserves texture saturation and color bleeding, while gather pyramids outperform scatter pyramids, minimizing ringing and oversharpening.

Numerical Evaluation and Comparative Results

Extensive quantitative assessment demonstrates the superiority of the adaptive method over both uniform sampling and superresolution. At 0.25 spp (equivalent to 50% superresolution), the adaptive variant achieves 25.41 dB PSNR and 7.056 JOD in CVVDP, substantially outperforming DLSS, JNDS, and NPPD baselines. Consistent gains are maintained across targets from 0.11 spp up to 4 spp.

Resolution scaling experiments show that adaptive sampling capitalizes on increased spatial resolution, as the sampler tailors allocations to high-frequency regions, mitigating blurring and loss of detail (see Figure 5 and Figure 6). The density maps illustrate the concentration of samples on perceptually salient features.

Figure 5: Adaptive sampling preserves edges and texture at low budgets, outperforming DLSS and JNDS, with density maps highlighting selective allocation.

Figure 6: Across varying budgets and scenes, adaptive sampling consistently recovers fine details and allocates samples to regions of high error and perceptual importance.

Component ablations confirm that the stochastic estimator, perceptual loss, gather pyramid, learned demodulation, and recurrent features are all critical for optimal performance and stability. Using surrogate gradients or classic losses degrades quality, especially in the sparse regime.

Robustness and Practical Implications

Robustness analysis (Figure 7) shows that adaptive sampling improves quality in complex, high-error regions while only marginally degrading performance in smooth, low-error areas. Mean absolute error is sharply reduced where sampling increases, indicating effective targeting of the computational budget.

Figure 7: Best-worst case analysis demonstrates adaptive sampling's concentration of computation on high-error regions, achieving significant error reduction with minimal penalty elsewhere.

Inference experiments with lightweight networks show that the approach scales well; even real-time-capable architectures benefit from adaptive allocation.

GBuffer rendering overhead is minimal compared to the cost of path tracing, and high-resolution GBuffers significantly assist denoiser and sampler performance.

Temporal stability is enhanced by recurrent features and perceptual alignment, with sample density maps naturally compensating for dynamic specularities and disocclusions. Future progress in motion vector estimation and GBuffer-free extrapolation is poised to further optimize efficiency.

Theoretical and Practical Implications

From a theoretical standpoint, the stochastic formulation resolves the longstanding challenge of differentiability in sparse adaptive sampling, fully decoupling sample allocation from deterministic bias and gradient vanishing. The integration of differentiable tonemapping and perceptual losses directly aligns optimization with human visual perception, a departure from legacy approaches.

Practically, the method points toward a new paradigm for real-time path tracing: spatially targeted rendering yields higher fidelity and perceptual quality than both uniform sparse sampling and superresolution. The pipeline is robust even at extreme sparsity, and network architectures can be flexibly scaled without substantial quality loss. The learnable demodulation and gather-based filters signal avenues for further innovation as rendering pipelines become more heterogeneous.

Speculation and Future Directions

Adaptive sampling could be integrated with dynamic scene understanding for temporal anticipation, allowing even more aggressive sparsity regimes. Advances in video perceptual metrics may supersede MILO, unlocking further gains. As rendering engines adopt increasingly complex materials and effects (including neural materials and volumetrics), learned allocation strategies will become indispensable for efficient computation. Transformer-based denoisers and improved temporal networks may further enhance scaling and stability.

Conclusion

The presented work establishes a stable, perceptually aligned, end-to-end adaptive sampling and denoising pipeline for real-time path tracing under extreme sparsity. The stochastic formulation for sample placement, perceptual loss-guided allocation, gather pyramidal denoising, and learned demodulation collectively enable superior recovery of detail and perceptually critical features compared to superresolution and previous adaptivity schemes, robust across spatial resolutions and budgets. The practical implications are significant: efficient rendering with minimal samples and maximal visual fidelity in advanced graphical applications is now achievable.