Refinement of Reliability Grid Codes in the Provision of Ancillary Services

Abstract: Stochastic resources such as wind farms, electric vehicle aggregators, and demand-side assets are increasingly participating as reserve providers in ancillary service markets. To manage delivery uncertainty, system operators impose minimum reliability thresholds on such providers. Energinet, the Danish transmission system operator (TSO), has pioneered this approach through the P90 requirement, requiring stochastic providers to make accepted reserve capacity bids available with at least 90% probability. Yet this threshold is set by regulatory convention, not optimization: no existing framework treats it as a design variable or characterizes the cost-reliability trade-off it governs. This paper closes that gap. We develop a bilevel optimization framework in which the TSO in the upper level sets the reliability threshold endogenously while providers in the lower levels respond through reliability-constrained bidding, with chance constraints reformulated analytically using a Weibull tail distribution. Applied to the Nordic frequency containment reserve for disturbances (FCR-D) market, the cost-optimal threshold lies below P90 in the studied cases, with cost reductions by up to 14.5% relative to the fixed standard. Dynamic hourly thresholds yield a further reduction of up to 2.4%, suggesting efficiency gains may increase in larger and more diverse reserve markets.

• The paper demonstrates that treating reliability thresholds as variables via bilevel optimization can yield up to 14.5% cost savings compared to the static P90 rule.
• It develops a tractable model using Weibull-tail distributions and McCormick relaxations to accurately capture risk and bid behaviors for stochastic reserve providers.
• The study validates optimized thresholds in the Nordic FCR-D market, highlighting trade-offs among cost, system reliability, and the reserve provision mix.

Optimization of Reliability Grid Codes for Stochastic Reserve Providers in Ancillary Service Markets

Introduction

The integration of stochastic resources such as wind farms, EV aggregators, and demand-side assets into reserve provision for ancillary services introduces significant uncertainty in delivery compared to traditional, dispatchable generators. To manage this uncertainty, system operators commonly impose exogenous reliability thresholds—like the P90 rule used by Energinet (the Danish TSO)—requiring providers to guarantee reserve availability with at least 90% probability. However, these thresholds are not the result of rigorous optimization and have lacked a formal framework for balancing reliability against economic efficiency within reserve markets.

This study formulates and analyzes the endogenous optimization of reliability grid codes, treating the reliability threshold as a design variable. Using a bilevel optimization framework, the work demonstrates that cost-reliability trade-offs can be systematically explored. The methodology is developed for and tested on the Nordic FCR-D reserve market, focusing explicitly on the Danish regulatory context.

Bilevel Optimization Framework for Reliability Thresholds

The core methodological innovation is the formulation of reserve market operation as a bilevel optimization problem. The upper level models the TSO, which sets the reliability threshold $1-\varepsilon_t$ for stochastic providers in each hour to minimize the sum of reserve procurement and expected shortfall penalty costs. The lower level consists of reserve providers—representative wind farm, EV aggregator, and a dispatchable generator—that respond by maximizing their reserve bids subject to probabilistic availability constraints (chance constraints).

Unlike typical approaches that treat reliability standards as exogenous, this model optimizes them with full anticipation of provider bidding behavior and market clearing outcomes. The lower-level chance constraints for stochastic providers are analytically reformulated for tractability, leveraging a Weibull distribution fitted only on the relevant lower tail (up to 20th percentile), thus ensuring an accurate and computationally efficient representation of high-reliability demand.

The bid formation process and the impact of different reliability thresholds on bid sizes and market prices are depicted in Figure 1.

Figure 1: Effect of reliability threshold $1-\varepsilon_t$ on wind farm reserve bids (left) and resulting reserve merit order curves (right); stricter thresholds yield smaller bids and increased clearing prices.

Data, Case Study Design, and Distributional Assumptions

Case studies are grounded in the Danish FCR-D up-regulation market using real data for wind power forecasts and EV fleet flexibility. Representative providers are parameterized using archival hourly wind forecasts (transformed via a power curve and assumed reserve fraction) and charging records from 1,400 EVs to estimate up-regulation capacity.

The foundational statistical assumption is a two-parameter Weibull CDF fitted only to the empirical tail (20th percentile and below) of provider availability data, justified by prior work indicating global fits poorly capture tail risk behavior crucial for reserve reliability. Goodness of fit is validated using Kolmogorov-Smirnov tests; representative fits for wind and EV flexibility both attain high $p$-values ($0.62$ and $0.99$, respectively), indicating accurate tail modeling.

Figure 2: Empirical data vs. Weibull fit for wind (left) and EV flexibility (right), illustrating tail fit quality for reserve bid reliability modeling.

Numerical Results and Performance Analysis

Cost-Reliability Pareto Front and Threshold Optimization

The system-wide cost-effectiveness of endogenous reliability threshold selection is evaluated via the static and dynamic optimization of $1-\varepsilon$. The static case enforces a single threshold for the day, while the dynamic case allows hourly variation. Penalty parameters for shortfalls are varied to trace the system's Pareto frontier.

Optimization reveals that the conventional static P90 threshold (i.e., $1-\varepsilon = 0.9$) is not generally cost-minimizing. Instead, the optimal threshold typically lies in the range $(0.8, 0.9)$, yielding up to 14.5% reduction in total reserve costs relative to the P90 standard. Marginal improvements (up to 2.4%) are realized with fully dynamic, time-variant thresholds, reflecting additional efficiency from adapting reliability requirements to supply and demand variations.

Figure 3: Pareto frontier of total reserve cost vs. static reliability threshold for varying shortfall penalty values, with diamonds indicating cost-minimizing thresholds below P90.

Impact on Reserve Provision Mix

Lower reliability thresholds expand the market share of stochastic (wind/EV) providers, enabling cheaper reserve procurement. At higher reliability, reliance shifts toward dispatchable generators with higher marginal costs, as visually summarized in the reserve mix figure.

Figure 4: Aggregate reserve provision mix as a function of the reliability threshold, illustrating the shift from stochastic to dispatchable capacity and tracking shortfall costs.

Out-of-Sample Reliability Compliance

Robustness and generalization are validated by holding out samples for out-of-sample assessment. Reliability is measured both by the fraction of shortfall events (violations) and the magnitude of those shortfalls as a fraction of awarded bids. Even at the optimal (lower-than-P90) threshold, violation rates remain within prescribed bounds, and shortfall magnitudes are generally small—demonstrating that structured, endogenous relaxation of reliability does not cause dramatic failures.

Figure 5: Out-of-sample reliability, by number of violations (top) and quantity (bottom), for optimal static thresholds, showing good compliance with reliability targets.

Computational Aspects

The bilevel optimization is reformulated to a computable, single-level mixed-integer linear program using McCormick relaxations and piecewise-linear approximations for all nonlinearities. Solution times are modest (seconds to minutes), supporting day-ahead operational feasibility.

Figure 6: Comparison of system cost vs. reliability for the nonlinear and McCormick-relaxed models, illustrating the small relaxation gap.

Implications and Extensions

The evidence that static, regulator-imposed reliability criteria (like P90) are suboptimal in real market contexts challenges existing grid codes for reserve markets integrating variable renewables. Dynamic, context-aware reliability standards can more efficiently harness stochastic flexibility without endangering system security, redistributing procurement toward cheaper but less firm resources—provided penalties for shortfall are properly calibrated.

Theoretical implications include:

• The tractability and suitability of bilevel Stackelberg formulations for ancillary service market design,
• The necessity of accurate, tail-oriented probabilistic modeling for reserve delivery,
• The formalization of the cost-reliability trade-off as a Pareto frontier to inform regulatory standards.

In practice, system operators can leverage this framework to:

• Calibrate reliability thresholds adaptively in real-time or day-ahead markets,
• Promote market participation of stochastic assets without overpaying for unattainable reliability,
• Quantitatively trade off reserve costs against probabilistic delivery risks within explicit regulatory tolerances.

Future research directions include generalizing the agent model to disaggregate provider heterogeneity, extending the framework to energy-intensive reserves (aFRR/mFRR) with rebound constraints, introducing co-optimization across multiple market products, and endogenizing market entry (participation) and opportunity cost modeling as the reliability bar is adjusted.

Conclusion

This work formalizes and solves the problem of reliability threshold selection for stochastic reserve providers using a bilevel optimization approach. It is demonstrated that traditional static reliability requirements (P90) are not optimal, and that both static and dynamic endogenous selection of reliability thresholds can yield substantial cost savings—up to 14.5% for static and further for dynamic thresholds—while maintaining system reliability within tolerable margins. The approach is practically tractable, robustly validated with real data, and broadly generalizable. Transitioning to optimally designed, rather than solely conventionally specified, grid codes offers meaningful efficiency gains as power systems incorporate increasing shares of stochastic resources.