Risk-Averse Learning Algorithms

• Risk-averse learning algorithms are methods that explicitly quantify tail risk using measures like CVaR.
• They employ both gradient-based and zeroth-order techniques to adapt to time-varying risk and environmental changes.
• Theoretical dynamic regret bounds and empirical evaluations confirm their robust performance in uncertain, safety-critical domains.

Risk-averse learning algorithms are a class of methods in online optimization and machine learning that explicitly account for the tail risk of losses—i.e., the probability and impact of incurring significantly high costs—rather than merely optimizing expected performance. By employing coherent risk measures such as Conditional Value-at-Risk (CVaR), these algorithms provide tools for robust decision-making in dynamic, uncertain, and safety-critical environments, especially when the level of risk aversion itself may vary over time (Wang et al., 28 Dec 2025).

1. Problem Formulation and Risk Measure Framework

Risk-averse learning algorithms operate in settings where the learner makes sequential decisions $x_t \in \mathcal X \subseteq \mathbb R^d$, after which a stochastic cost $J_t(x_t, \xi)$ is incurred, with $\xi \sim \mathcal D_t$ representing possibly nonstationary, time-varying environmental noise (Wang et al., 28 Dec 2025). The distinctive feature is the use of CVaR at time-varying confidence levels $\alpha_t$, which quantifies the expected cost in the worst-case $(1-\alpha_t)$-fraction of outcomes:

$$C_t(x) := \mathrm{CVaR}_{\alpha_t}[J_t(x, \xi)] = \inf_\nu \left\{ \nu + \frac{1}{\alpha_t} \mathbb E\left[(J_t(x,\xi)-\nu)_+\right] \right\}.$$

This risk-centric formulation contrasts with classical risk-neutral learning, which minimizes expected losses. When $J_t(x,\xi)$ is convex and Lipschitz in $x$, $C_t(x)$ inherits these properties.

2. Nonstationarity and Variation Metrics

To systematically capture the environment's nonstationarity, two variation metrics are introduced:

• Function Variation ($V_f$): Measures temporal drift in the expected cost function:

$$V_f = \sum_{t=2}^T \sup_{x \in \mathcal X} \left| \mathbb{E}_\xi [J_t(x, \xi)] - \mathbb{E}_\xi [J_{t-1}(x, \xi)] \right|.$$

• Risk-Level Variation ($V_\alpha$): Measures cumulative change in the risk-aversion parameter:

$V_\alpha = \sum_{t=2}^T |\alpha_t - \alpha_{t-1}|.$

The aggregate $V_T = V_f + V_\alpha$ quantifies overall nonstationarity, and sublinear growth (i.e., $V_T = o(T)$) indicates a mildly nonstationary scenario where adaptation is feasible (Wang et al., 28 Dec 2025).

3. Algorithmic Approaches

Risk-averse learning under time-varying objectives and risk levels is facilitated by two algorithmic frameworks, distinguished by the type of feedback available:

3.1 First-Order (Gradient-Based) Algorithm

Applicable when both function values and gradients can be sampled. At each step:

1. Collect $n_t$ i.i.d. samples $\xi_t^i$, compute $J_t^i = J_t(x_t, \xi_t^i)$, $\nabla_x J_t(x_t, \xi_t^i)$.
2. Compute empirical VaR, $\hat \nu_t$, as the minimizer in the empirical CVaR expression.
3. Form the gradient estimator:

$$\hat g_t = \frac{1}{n_t \alpha_t} \sum_{i=1}^{n_t} \mathbf{1}\{J_t^i \ge \hat \nu_t\} g_t^i.$$

1. Update the decision by projected gradient descent:

$x_{t+1} = \Pi_{\mathcal X}[x_t - \eta \hat g_t].$

This estimator leverages the CVaR gradient identity, which requires knowledge of the underlying quantile; the empirical substitute introduces statistical error controlled via sample size (Wang et al., 28 Dec 2025).

3.2 Zeroth-Order (Bandit) Algorithm

Targeted at settings where only function evaluations are accessible (bandit feedback). The algorithm performs:

1. One-point smoothing: sample a direction $u_t$, perturb $x_t$ to $x_t + \delta u_t$.
2. Query $n_t$ function evaluations at perturbed points, estimate empirical CVaR.
3. Construct the gradient estimator via:

$$\hat g_t = \frac{d}{\delta} \mathrm{CVaR}_{\alpha_t}[\hat F_t] u_t,$$

where $d$ is the problem dimension.

1. Update with projection onto the feasible set (or a shrunken version thereof).

This smoothing approach yields an unbiased estimator for the gradient of the CVaR-smoothed cost, enabling zeroth-order optimization of risk-averse objectives (Wang et al., 28 Dec 2025).

4. Regret Analysis and Theoretical Guarantees

Performance is measured by dynamic regret: $$\mathrm{DR}(T) = \sum_{t=1}^T C_t(x_t) - \sum_{t=1}^T C_t(x_t^*), \quad x_t^* = \arg\min_{x \in \mathcal X} C_t(x).$$

Regret Bounds

Letting the number of samples per round $n_t$ satisfy $\sum_t 1/\sqrt{n_t} = O(T^{1-a/2})$ for some $a>0$:

• First-Order Algorithm:

$$\mathrm{DR}(T) = \widetilde O\left( T^{2/3} (V_f + V_\alpha)^{1/3} + T^{1-a/2} \right).$$

When $a \ge 2/3$, the regret is dominated by the first term.

• Zeroth-Order Algorithm:

$$\mathrm{DR}_0(T) = \widetilde O\left( T^{1-a/4} (V_f + V_\alpha)^{1/5} \right),$$

if $a>4/5$, this simplifies to

$$\mathrm{DR}_0(T) = \widetilde O\left( T^{4/5} (V_f + V_\alpha)^{1/5} \right).$$

If $V_f + V_\alpha = o(T)$ and the sample budget is sufficiently large, both frameworks guarantee sublinear dynamic regret, meaning average per-round regret vanishes as $T\to\infty$ (Wang et al., 28 Dec 2025).

5. Empirical Evaluation and Observations

A dynamic parking-price problem with abrupt changes in both the environmental objective and risk level is used to empirically assess the algorithms. Key findings:

• Both first-order and zeroth-order methods successfully track the time-varying optimal solution; the first-order method exhibits faster convergence and greater stability.
• Regret increases as $V_f$ or $V_\alpha$ grows, validating theoretical dependence on the nonstationarity budget.
• Increasing per-round sample count reduces the CVaR estimation error and regret, consistent with the sample-complexity term in the theoretical bounds.
• Benchmarks that ignore either form of variation (function or risk-level) incur much larger regret, demonstrating the necessity of dual adaptation for dynamic, risk-sensitive settings (Wang et al., 28 Dec 2025).

6. Assumptions, Limitations, and Extensions

The algorithms and bounds are derived under:

• Convexity and Lipschitz continuity of $J_t(x, \xi)$ in $x$ (uniform in $\xi$),
• Bounded gradients,
• Uniformly positive density of $J_t(x, \xi)$ around relevant quantiles.

Potential extensions and open problems include:

• Generalizing to non-convex CVaR objectives, or relaxing smoothness constraints,
• Studying online games with agent-specific, time-varying risk preferences and analyzing the tracking of dynamic Nash equilibria,
• Extending to distributionally robust risk-averse learning where the ambiguity set over the cost distribution itself evolves,
• Leveraging variance-reduced CVaR gradient estimators or employing accelerated smoothing strategies for tighter theoretical guarantees, especially in the bandit regime.

7. Summary Table of Core Quantities and Algorithms

Quantity / Step	First-Order Algorithm	Zeroth-Order (Bandit) Algorithm
Feedback	$J_t(x_t,\xi)$, $\nabla_x J_t(x_t,\xi)$	$J_t(x_t+\delta u_t, \xi)$
CVaR Gradient Estimation	Empirical CVaR plug-in with empirical quantile	One-point finite-difference with isotropic random direction
Regret Bound ($V_f+V_\alpha=o(T)$)	$$\widetilde O\left(T^{2/3}(V_f+V_\alpha)^{1/3}\right)$$	$$\widetilde O\left(T^{4/5}(V_f+V_\alpha)^{1/5}\right)$$
Adaptation to Nonstationarity	Both $V_f$ and $V_\alpha$	Both $V_f$ and $V_\alpha$

All formal claims, design steps, and numerical patterns above are directly present in (Wang et al., 28 Dec 2025).

---

In summary, risk-averse learning algorithms with time-varying risk levels deliver provable robustness and adaptability in nonstationary environments by quantifying and tracking both functional and risk-level drift. These approaches leverage empirical CVaR gradient estimators within online convex optimization, and, under sublinear environment drift and sufficient sampling, yield dynamic regret bounds assuring that adaptation to both environmental and risk-preference changes remains theoretically sound and practically viable (Wang et al., 28 Dec 2025).