Adaptive LinUCB Selector

• Adaptive LinUCB Selector is a class of algorithms that dynamically adjusts exploration bonuses and model parameters in linear stochastic contextual bandits.
• These methods employ opportunistic exploration, meta-selection, discounting, and randomized strategies to manage cost variation, model uncertainty, and nonstationarity.
• They achieve improved theoretical regret bounds (e.g., O((log T)^2)) and robust practical performance in applications such as online LLM routing and adaptive model selection.

An Adaptive LinUCB Selector refers to any algorithmic scheme in the linear stochastic contextual bandit setting that systematically adjusts its exploration-exploitation policy or structural assumptions in response to observed data, nonstationarity, or model uncertainty. Unlike classical LinUCB—which uses a fixed elliptical bonus and fixed-confidence radius for all rounds—adaptive versions modulate exploration bonuses, dimension, discounting, or structural constraints, thereby yielding improved regret guarantees or broader applicability under complex or drifting environments. Multiple independent lines of recent research provide rigorous algorithms and analysis frameworks for adaptivity, each tailored to different adaptation targets (cost, model class, support, regularization, time-variation, computational budget, etc.).

1. Adaptive LinUCB via Opportunistic Exploration

AdaLinUCB (Guo et al., 2019) formalizes adaptivity to exogenous “exploration cost” signals $L_t$ in contextual bandits. At each round $t$, an observed cost factor $L_t$ is used to transform LinUCB’s bonus by a data-driven coefficient: $$p_{t,a} = \theta_{t-1}^T x_{t,a} + \alpha \sqrt{(1-\tilde L_t) \, x_{t,a}^T A_{t-1}^{-1} x_{t,a}},$$ where $\tilde L_t\in[0,1]$ maps $L_t$ to a standardized scale using thresholds $(l^{(-)},l^{(+)})$. When the cost is low ($\tilde L_t\to0$), the algorithm maximizes exploration; when cost is high ($\tilde L_t\to1$), it exploits using the current estimate. This opportunism leads to a problem-dependent regret bound of $O((\log T)^2)$ in the actual reward metric, improving significantly over standard LinUCB in environments where exploration cost fluctuates, including both real-world and synthetic benchmarks.

2. Meta-Selector Adaptation: Regret Balancing among LinUCBs

Adaptive LinUCB can be realized as a meta-algorithm running a pool of LinUCB/OFUL copies parameterized by a set of confidence multipliers $\{\kappa_i\}$, as developed by Pacchiano et al. (Pacchiano et al., 2020). At each round, the selector balances the (candidate) regret bounds $\tilde R_i(n)$ of all sub-learners and eliminates any that empirically violate these bounds according to a precise statistical test. Action selection always comes from the currently “least-regret” sub-learner. The proved regret scaling is, up to logarithmic factors, within $O(M)$ multiplicative overhead of the best base LinUCB’s regret, where $M$ is the number of base learners. This guarantees robust, essentially optimal performance even if the correct exploration width is not known a priori, under both stochastic and adversarial context sequences.

Adaptivity target	Technique	Regret guarantee
Exploration cost ($L_t$)	AdaLinUCB (Guo et al., 2019)	$O((\log T)^2)$, improved actual regret
Confidence width	Regret-balanced meta-selector	Within $O(M)$ of best LinUCB (Pacchiano et al., 2020)
Model uncertainty	Successive elimination	Additive overhead + $\widetilde O(d^*\sqrt T)$ (Ghosh et al., 2021)

3. Adaptive LinUCB for Non-Stationarity

In non-stationary environments, where the parameter vector $\theta_t^\star$ drifts over time, adaptive LinUCB selectors employ discounting, randomized bonuses, or both. D-LinUCB (Russac et al., 2019) uses exponentially discounted least-squares estimates, modifying the Gram matrix and confidence radii via a discount factor $\gamma$ to optimize for dynamic regret: $$A_t = \gamma A_{t-1} + x_{t} x_{t}^T + (1-\gamma)\lambda I, \qquad b_t = \gamma b_{t-1} + x_{t}y_t.$$ The UCB radius is then computed with respect to a weighted norm. Tuning $\gamma$ yields minimax-optimal dynamic regret of $O(d^{2/3}B_T^{1/3}T^{2/3})$, where $B_T$ is the variation budget. D-RandLinUCB (Kim et al., 2019) further introduces randomized confidence perturbations atop discounting, which empirically and theoretically overcome conservatism, achieving $\widetilde O(d^{7/8}B_T^{1/4}T^{3/4})$ regret and superior empirical adaptation to regime changes.

4. Dimension, Model, and Confidence Adaptation

4.1 Support/Dimension Adaptivity

Adaptive LinUCB selectors for support/model dimension, as in the ALB-Dim algorithm (Ghosh et al., 2021), phase between standard LinUCB (OFUL) and exploratory subroutines dedicated to support estimation. After each pure-exploration phase, the estimated active coordinates are updated, and regret-minimization proceeds restricted to the support estimate. This ensures additive model-selection cost and leading term matching the optimal $\widetilde O(d^*\sqrt T)$ as soon as the true dimension $d^*$ is discovered.

4.2 Data-driven Confidence Adaptivity

A differentiable LinUCB (SoftUCB) (Yang et al., 2020) leverages offline or online gradient ascent over the bonus parameter $\beta$ (the width of the confidence set), using expected reward gradients computed through softmax-based differentiable surrogates. This approach directly learns the optimal $\beta$ for the given data distribution, yielding regret bounds of $\widetilde O(\hat\beta\sqrt{dT})$, with $\hat\beta$ empirically far smaller than the pessimistic theoretical bound.

5. Algorithmic Mechanisms: Discounting, Randomization, and Truncation

Adaptive LinUCB variants further comprise mechanisms such as:

• Truncation: Tr-LinUCB (Song et al., 2022) runs LinUCB for $S=Cd\log T$ rounds and then switches to pure exploitation. This achieves the minimax rate $O(d\log T)$, being robust to overshooting $S$.
• Randomized UCB: Randomization in the UCB bonus accelerates adaptation in nonstationary settings, as in D-RandLinUCB (Kim et al., 2019).
• Optimal Allocation Matching: Adaptive selectors (OAM) (Hao et al., 2019) continuously check for information-theoretic sufficiency in confidence widths and sample allocation, interpolating between LinUCB and greedy play, thus recovering asymptotically optimal allocation and sub-logarithmic regret in benign regimes.

6. Computational and Structural Adaptivity

Adaptive LinUCB selectors have been extended to:

• Low-rank approximations: Scalable LinUCB (Shustova et al., 22 Oct 2025) replaces full-rank inverse design matrices with dynamically parametrized low-rank factorizations, using projector-splitting integrators to bound computational and memory costs at $O(dr)$ per update for approximation rank $r$.
• Action/feasible set structure: For ellipsoidal action sets, efficient optimistic maximization schemes—MaxNorm and Newton subroutines—enable high-dimensional LinUCB variants to remain tractable and regret-optimal (Zhang et al., 10 Nov 2025).

7. Practical Implications, Model Selection, and Empirical Performance

Adaptive LinUCB selectors underpin robust performance in real-world settings marked by cost fluctuation, unknown structure, high feature dimension, and non-stationarity. Their meta-algorithmic or internally adaptive character allows for simultaneous learning and model selection (as in universal/data-adaptive model selection (Muthukumar et al., 2021)), yielding rates interpolating between $O(\sqrt{T})$ and $O(d^{1/6}T^{5/6})$ depending on observed context diversity and regime. Adaptive LinUCB also empowers large-scale applications such as online LLM routing under unstructured context evolution, using budget-aware and positionally-aware extensions (Poon et al., 21 Jun 2025), and enables statistically principled inference for adaptively collected data (Fan et al., 28 Nov 2025).

In summary, the Adaptive LinUCB Selector encompasses a diverse and rapidly expanding class of algorithms in linear contextual bandits unified by their real-time adaptation to cost, model, structural, or environmental variation, with clear regret and scalability benefits across theoretical and applied domains.