On the complexity of standard and waste-free SMC samplers

Abstract: We establish finite sample bounds for the error of standard and waste-free SMC samplers. Our results cover estimates of both expectations and normalising constants of the target distributions. We consider first an arbitrary sequence of distributions, and then specialise our results to tempering sequences. We use our results to derive the complexity of SMC samplers with respect to the parameters of the problem, such as $T$, the number of target distributions, in the general case, or $d$, the dimension of the ambient space, in the tempering case. We use these bounds to derive practical recommendations for the implementation of SMC samplers for end users.

• The paper derives finite-sample complexity bounds for expectation estimation by leveraging spectral gap and mixing time properties in SMC samplers.
• It introduces a waste-free SMC approach that reuses the full Markov trace to improve variance reduction and computational efficiency.
• It provides non-asymptotic guarantees for normalizing constant estimation, with insights on enhancements via median-of-means aggregation.

Complexity Analysis of Standard and Waste-Free SMC Samplers

Introduction and Overview

This work provides a comprehensive finite-sample analysis for both standard and waste-free Sequential Monte Carlo (SMC) samplers, with a focus on the complexity required to achieve prescribed error bounds for the estimation of expectations and normalizing constants. The authors develop explicit complexity rates as a function of key problem parameters (such as the number of bridging distributions $T$, the ambient dimension $d$, and the spectral gap $\gamma$ or mixing times) and analyze practical implications for algorithm configuration in high-dimensional and tempering scenarios.

A primary contribution is the derivation of finite-sample bounds for both SMC variants, yielding, for the first time, non-asymptotic guarantees for normalizing constant estimators in the SMC context under realistic Markov kernel assumptions, including scenarios where the spectral gap may be weak or absent.

Standard versus Waste-Free SMC: Formalization and Key Distinctions

Standard SMC recursively propagates $M$ interacting particle Markov chains across a sequence of target distributions $\pi_0, \ldots, \pi_T$. At each stage, particles are resampled on the terminal states, possibly discarding intermediate Markov transitions.

Waste-free SMC, in contrast, retains all $N=MP$ states generated in each Markov chain at each iteration. The entire set is reweighted and resampled, leveraging the full Markov trace. This strategy was originally motivated by empirical observations of improved variance but lacked finite-sample complexity analysis prior to this work.

The paper's main technical novelty is a recursive coupling construction that enables tracking the evolution of the sampling law in waste-free SMC, controlling warmness, and delivering rigorous bounds on the error and required computational budget.

Finite-Sample Guarantees and Complexity Bounds

The primary technical results are finite-sample, high-probability bounds for estimation errors of both moments and normalizing constants, giving explicit scaling relations with respect to $\varepsilon$ (accuracy), $\eta$ (failure probability), $T$, $d$, $d$0, $d$1, and mixing properties.

The results are organized by estimation objective (moments vs. normalizing constants), sampler variant, and Markov kernel assumption:

Moments

• Standard SMC: Under a uniform spectral gap $d$2, estimation of expectations $d$3 to error $d$4 (with probability $d$5) requires $d$6 Markov steps.
• Waste-Free SMC: The corresponding complexity is $d$7, yielding a $d$8 improvement.

A flexible "greedy" variant of waste-free SMC, where $d$9 is large only in the final iteration, further improves the leading complexity term to $\gamma$0 for the expectation with respect to $\gamma$1.

Normalizing Constants

• Standard SMC (under mixing-time bounds): Estimation of $\gamma$2 up to accuracy $\gamma$3 and failure probability $\gamma$4, with warm-start mixing time $\gamma$5, costs $\gamma$6 Markov transitions.
• Waste-Free SMC (under spectral gap): The estimator for $\gamma$7 is controlled via second-moment methods and union bounds, showing that $\gamma$8 Markov steps suffice. Via a median-of-means aggregation, the cost is improved to $\gamma$9.

The analysis derives a lower bound matching $M$0 for idealized (independent) settings, leaving a gap of order $M$1 in the dependency for waste-free SMC as an open problem.

Specialization to Geometry and High-Dimensional Scaling

The tempering case, where $M$2 interpolates between a tractable base and the target distribution via a schedule $M$3, is central to applications such as Bayesian computation and log-concave volume estimation. Under geometric tempering with strong log-concavity and smoothness, the authors show:

• The number of required intermediate distributions is $M$4.
• For random walk Metropolis kernels, spectral gaps scale as $M$5, giving query complexities of up to $M$6 for waste-free SMC and $M$7 for robust median-based normalizer estimates.
• For MALA or pCN-type kernels, which can deliver improved dimension dependency, standard SMC with robust median-of-means aggregation achieves $M$8 complexity for estimating $M$9.

The analysis highlights the optimal or near-optimality of these rates compared to the lower bounds established for volume computation and log-concave normalization [10.1214/18-EJS1411, (Ge et al., 2019)].

Technical Insights: Proof Techniques and Key Lemmas

A central contribution is the extension of finite-sample coupling and warmness control, previously established only for standard SMC, to the waste-free SMC context. The work leverages:

• Pathwise couplings to stationary chains, controlling the propagation of deviations across iterations.
• Non-asymptotic concentration inequalities for Markov chains with spectral gaps [JMLR:v22:19-479, (Jiang et al., 2018)], refined for one-sided deviations and heavy-tailed reweighting.
• Advancement from sub-Gaussian to variance-based (second moment) bounds, enabling tight analysis of heavy-tailed importance sampling ratios and robust estimators for $\pi_0, \ldots, \pi_T$0.

These technical ingredients enable the authors to close key gaps in the literature, especially on finite-sample analysis of normalizing constant estimators, and systematize the choice of algorithmic parameters in practice.

Implications and Perspectives

Practically, the results validate the empirical variance reduction of waste-free SMC, quantifying its advantage using a reduced query complexity for expectation estimation at fixed error. The greedy allocation strategy suggests focusing computational effort on the final bridging step, a principle with significant computational implications.

For normalizing constant estimation—a notoriously difficult problem in high dimensions—the work recommends using standard SMC with robust median-based estimators, especially when employing fast-mixing kernels such as MALA or pCN. The analysis identifies scenarios where the waste-free SMC complexity is competitive and lays out precise conditions (e.g., log-concavity, spectral gaps, mixing regime) where one strategy should be preferred over another.

From a theoretical perspective, the extension to mixing-time-based rather than spectral gap-based bounds opens the framework to the analysis of optimal-transport-based or non-convex scenarios and invites future work to close the remaining complexity gaps.

Conclusion

"On the complexity of standard and waste-free SMC samplers" (2604.03352) bridges a critical gap in the quantitative analysis of SMC algorithms by deriving explicit, non-asymptotic complexity bounds for both expectation and normalizing constant estimation under finite computational budgets. Through a combination of pathwise coupling, variance analysis, and concentration inequalities suitable for dependent samples, the work systematizes the tuning of SMC samplers and provides practical guidelines for scalable inference in high-dimensional and strongly log-concave settings.

An important remaining open question is whether the $\pi_0, \ldots, \pi_T$1-factor gap in normalizing constant estimation for waste-free SMC can be eliminated via further dependence and coupling structure refinement, or whether this gap is intrinsic. The results set the basis for similar finite-sample analysis in adaptive SMC, multimodal and non-log-concave contexts, as well as for robust estimation strategies in the presence of heavy-tailed importance ratios.