Deterministic Backtesting Configuration

• Deterministic backtesting configuration is a rigorously specified process that guarantees unique, reproducible results by eliminating path-dependence and ambiguity.
• It underpins quantitative finance by validating trading strategies, calculating risk measures like Expected Shortfall, and ensuring regulatory compliance.
• The framework employs finite test suites and explicit ambiguity resolution to certify backtest engine correctness across diverse market scenarios.

A deterministic backtesting configuration is a rigorously specified process, algorithm, or test suite ensuring that a sequence of historical data, trading rules, and risk metrics yields unique, reproducible outcomes under precise assumptions—free of path-dependence, tie-breaking ambiguity, or Monte Carlo sampling. Such configurations are foundational in quantitative finance and risk regulation, underpinning the formal validation of trading strategies, risk measures such as Expected Shortfall (ES), and the operational correctness of backtest engines deployed in both equity and derivatives trading. Determinism ensures regulatory acceptability, reproducibility of results, robust statistical interpretation, and forms a basis for formal proofs of correctness and fairness.

1. Formal Properties of Deterministic Backtesting

Deterministic backtest engines and frameworks are characterized by:

• Uniqueness of Results: For each input (data sequence, configuration parameters, order logic), the outcome (trade list, P&L, or risk statistic) is uniquely determined with no dependence on random seeds, sampling, or subjective tie-breaks.
• Well-Posedness under Model and Data Constraints: Backtest execution relies on precise modeling of allowed data series (e.g., candle OHLCV, intra-candle price paths), order types, liquidity assumptions (immediate fill or “perfect liquidity”), and rule-based ambiguity resolution for simultaneously triggered orders or "situations not unique" (SNUs) (Löw et al., 2015, Maier-Paape et al., 2014).
• Reproducibility and Auditability: The system’s outputs can be traced and certified by deterministic procedures and, if needed, formally proven correct via finite test suites covering all execution branches (Löw et al., 2015, Maier-Paape et al., 2014).

2. Regulatory Risk-Measure Backtests: Deterministic ES Frameworks

Deterministic configurations for backtesting risk measures, particularly ES, provide robust regulatory tools for validating internal models under the Basel framework. A canonical example is the deterministic three-zone Expected Shortfall backtest proposed by Moldenhauer and Pitera:

• Secured Position Construction: Let the observed P&L on day $t$ be $\mathrm{P\!{\scriptstyle L}}_t$, with capital reserve $\hat\rho_t$. Define the “secured position” $X_t = \mathrm{P\!{\scriptstyle L}}_t + \hat\rho_t$.
• Backtest Statistic $K_\mathrm{nominal}$: Sort $X_{(1)} \leq \ldots \leq X_{(n)}$ over a window of $n$ days; compute cumulative sums $S_k = \sum_{i=1}^{k} X_{(i)}$. Define $K_\mathrm{nominal} = \max\{k: S_k < 0\}$, whose value determines the model’s status.
• Monotonicity and Duality: Because ES is monotonic in $\alpha$, $K$ effectively inverts the acceptance threshold: the minimal $\alpha^*$ such that the empirical $\widehat{\mathrm{ES}}_n^{\alpha^*}(X) \leq 0$ (Moldenhauer et al., 2017).

This backtest is explicitly deterministic: the observed $X_t$ sequence, given reserves $\hat\rho_t$, yields a unique $K_\mathrm{nominal}$ and traffic-light color (green/yellow/red) without any randomization or resampling.

Zone	$K_\mathrm{nominal}$	$G_n$	Typical Null Probability
Green	$\leq 11$	$<0.044$	$\approx 90\%$
Yellow	$12$–$24$	$0.048$–$0.096$	$\approx 10\%$
Red	$\geq 25$	$\geq 0.10$	$\approx 0.01\%$

This deterministic traffic-light scheme mirrors Basel VaR traffic-lights and ensures that only model underestimation (conservativeness failure) is flagged (Moldenhauer et al., 2017).

3. Finite Test Suites and Formal Engine Certification

Deterministic backtesting further includes engines whose end-to-end correctness is certifiable over all admissible trading scenarios by reduction to a finite test set (Löw et al., 2015):

• Model and Sub-level Discretization: All relevant prices (open, close, high, low, order levels) are restricted to a finite grid (order levels and interleaved sub-levels). Only the ordinal relationships among these grid points matter for execution logic.
• Model Candle and Path Enumeration: For $m$ order levels, one constructs all “model candles” and bounded-length “model price paths” (IPMS). These exhaustively encode all possible execution branches and ambiguity patterns.
• Monotone Transformation Invariance: Engines are configured to be stable under monotone (strictly increasing) price transformations, ensuring generality beyond specific price points.
• Deterministic Verification Algorithm: Each model candle and IPMS is processed deterministically by the engine; outputs are compared to a reference table generated by a certified reference implementation. Pass/fail status is unique and repeatable (Löw et al., 2015).

The following succinctly characterizes model-candle sufficiency:

“If for every model candle and every mode (best, worst, ignore) the engine produces the correct result as per the ground-truth reference, then it produces the correct result on all real candles and all modes.” (Löw et al., 2015)

4. Ambiguity Resolution in Candle-Only Backtesting

When backtesting trading strategies on OHLC (“candle”) data without intra-candle sequencing data (tick resolution), situations arise where the execution sequence is indeterminate (SNUs). Deterministic handling is enforced by:

• Explicit Modes: The engine provides user-selectable deterministic modes: “best-case,” “worst-case,” or “ignore,” with all ambiguities resolved locally within each candle and strictly serially processed (Maier-Paape et al., 2014).
• Unique First-Hit Times: Algorithms derive algebraic first-hit times for each order level based on hypothetical tick-by-tick paths between candle endpoints, subject to strict tick-size, no intra-period gaps, and perfect liquidity (Maier-Paape et al., 2014).
• Purely Rule-Driven Execution: No random or historical tie-breaking is allowed; hence every possible combination of candle pattern and entry/exit configuration produces one unique trade list per mode.
• Testing for Determinism: Exhaustive unit and integration tests are written for every engine branch and SNU pattern to enforce strict repeatability (Maier-Paape et al., 2014).

5. Deterministic Multinomial Backtesting for ES via VaR Exception Patterns

A deterministic configuration for ES backtesting can be constructed by evaluating multinomial exception patterns across a fixed set of VaR levels:

• VaR Exception Binning: For each period, generate $N$ VaR levels $\alpha_j$ and record the vector of exception indicators $I_{t,j}$, summarizing as $X_t = \sum_{j=1}^N I_{t,j}$.
• Aggregate Count Vector: Over $T$ windows, counts $O_j$ of $X_t = j$ are computed, yielding a full multinomial summary (Kratz et al., 2016).
• Closed-form Test Statistic: Compute a statistic such as Pearson $\chi^2$, Nass adjusted, or LRT—each functionally pure, without simulation or resampling—then compare to tabulated $\chi^2$ critical values for color-coded accept/review/reject zones.
• Parameter Recommendations (“Rough Rule-of-Thumb”): For $T \geq 250$ use $N=4$; for longer samples, $N=8$ enhances power without loss of determinism.

All steps, from choice of VaR levels to critical value lookup and traffic-light assignment, are deterministic and auditable (Kratz et al., 2016).

6. Deterministic Auto-Tuned Backtesting in Execution-Constrained Environments

Recent frameworks extend determinism to the configuration-selection and auto-tuning layer of complex execution-constrained backtests, ensuring strict reproducibility even when Bayesian optimization and multiple screening stages are introduced (Deng, 27 Dec 2025):

• T+1 Execution Semantics: Signals are computed at bar close $t$; trades execute deterministically at $t+1$, strictly disallowing look-ahead bias or ambiguous fill logic.
• Transaction Cost Decomposition: All cost drivers—fees, linear/power-law slippage, funding—are computed as deterministic functions of prior states and observed market parameters.
• Bayesian Search Traceability: Although configuration candidates are generated via probabilistic search (e.g., TPE), all candidates’ performance is evaluated deterministically on fixed data windows and cost grids, and selection thresholds are fixed ex ante.
• Double-Screening Protocol: Parameter survivors are filtered by deterministic screens on out-of-sample returns, max drawdown, and trade density, evaluated across a cost-sensitivity grid (Deng, 27 Dec 2025).
• Overfitting Diagnostics: CSCV/PBO and deflated Sharpe ratio diagnostics are computed deterministically on the returned metric series, ensuring auditable false-positive risk adjustment.

Summary artifacts log all deterministic input/output, enabling strict forensic replay and regulatory compliance for each backtest run.

---

References:

• Moldenhauer, F. & Pitera, M. “Backtesting Expected Shortfall: a simple recipe?” (Moldenhauer et al., 2017)
• Kratz, Lok & McNeil. “Multinomial VaR Backtests: A simple implicit approach to backtesting expected shortfall” (Kratz et al., 2016)
• Löw, R., Maier-Paape, S., Platen, A., “Correctness of Backtest Engines” (Löw et al., 2015)
• Maier-Paape, S., Platen, A., “Backtest of Trading Systems on Candle Charts” (Maier-Paape et al., 2014)
• AutoQuant project, “An Auditable Expert-System Framework…” (Deng, 27 Dec 2025)