QA-in-the-loop Kernel Learning Framework

• The paper presents a QA-in-the-loop framework that employs quantum annealing to actively sample RBM-based spectral distributions for adaptive kernel construction.
• It integrates QA hardware into regression models, leveraging non-Gaussian, multimodal spectral parameterization to overcome fixed-kernel limitations.
• Empirical results show improved stability and performance in NW and local linear regression through hardware-accelerated, data-adaptive training cycles.

A QA-in-the-loop kernel learning framework is a paradigm wherein quantum annealing (QA) is integrated not solely as an auxiliary optimization or sampling engine, but as a trainable, active component within the data-adaptive kernel construction for regression tasks. In this setting, QA hardware is embedded in the kernel learning loop—rather than outside it—to perform tractable, hardware-accelerated sampling from a parametrically learned spectral distribution, enabling the direct modulation of the spectral content of random Fourier features (RFF) used for kernel estimation. This approach leverages the expressiveness of quantum-assisted sampling, particularly via restricted Boltzmann machine (RBM) architectures, to surpass the limitations of fixed-kernel structures and latent multimodality in traditional randomized feature approximations (Hasegawa et al., 13 Jan 2026).

1. Theoretical Foundation and Motivation

Kernel regression methods such as the Nadaraya–Watson (NW) estimator are sensitive to the specification of the positive-definite kernel $k(\mathbf{x}, \mathbf{x}')$, especially in the shift-invariant case $k(\Delta\mathbf{x}) = k(\mathbf{x} - \mathbf{x}')$. Random Fourier features (RFF) provide finite-dimensional approximations to such kernels via

$$k(\Delta\mathbf{x}) = \int_{\mathbb{R}^d} p(\boldsymbol{\omega}) e^{i\boldsymbol{\omega}^\top \Delta\mathbf{x}} d\boldsymbol{\omega} \approx \mathbb{E}_{\boldsymbol{\omega} \sim p}\left[\cos(\boldsymbol{\omega}^\top \Delta\mathbf{x})\right],$$

where Bochner's theorem guarantees that any continuous, shift-invariant positive-definite kernel admits such an integral representation. Classical RFF employs a fixed (usually Gaussian) spectral distribution $p(\boldsymbol{\omega})$, which limits the adaptability of the kernel to complex, nonlinear data structure, and—when the number of features is limited—admits negative contributions leading to possible denominator instability in NW regression (Hasegawa et al., 13 Jan 2026).

A QA-in-the-loop approach seeks to overcome these limitations by parameterizing $p_\theta(\boldsymbol{\omega})$ via an RBM and drawing Boltzmann samples leveraging quantum annealing. This permits non-Gaussian, potentially multimodal, and highly adaptive kernel constructions with increased contrast and data alignment, inaccessible to conventional RFF methods.

2. Spectral Distribution Parameterization and Quantum Annealing

The QA-in-the-loop framework models the spectral distribution of the kernel, $p_\theta(\boldsymbol{\omega})$, using an RBM energy function: $$E(\mathbf{v}, \mathbf{h} \mid \theta) = -\sum_{i} b_i v_i - \sum_{j} c_j h_j - \sum_{i,j} W_{ij} v_i h_j,$$ where $\mathbf{v} \in \{\pm1\}^{N_v}$ and $\mathbf{h} \in \{\pm1\}^{N_h}$ are visible and hidden spins, and $\theta$ collectively denotes the RBM parameters. The joint Boltzmann probability is

$$P(\mathbf{v}, \mathbf{h}) = \frac{\exp[-E(\mathbf{v}, \mathbf{h})]}{Z}.$$

Sampling from this RBM is performed via a quantum annealer, which maps the energy landscape to an Ising Hamiltonian. Finite temperature and device noise ensure that the annealer outputs samples distributed according to an approximate Boltzmann distribution. This configuration enables hardware-accelerated, nonlocal exploration of the RBM's state space, increasing sampling diversity and efficiency relative to classical MCMC approaches (Hasegawa et al., 13 Jan 2026).

3. Gaussian–Bernoulli Mapping and Random Fourier Feature Construction

To translate discrete RBM samples into continuous spectral frequencies required for the construction of RFFs, a Gaussian–Bernoulli conditional model is employed: $$P(\boldsymbol{\omega}\mid \mathbf{v}) = \prod_{j} \mathcal{N}\left(\omega_j \mid a_j + \sum_{i} U_{ji} v_i, \sigma_j^2 \right).$$ Given discrete QA+RBM samples $\{\mathbf{v}^{(s)}\}_{s=1}^S$, continuous frequencies $\boldsymbol{\omega}_s$ are generated. RFF vectors for each datum are constructed as: $$\boldsymbol{\phi}_i = \frac{1}{\sqrt{S}} (\cos(\boldsymbol{\omega}_1^\top \mathbf{x}_i), \ldots, \sin(\boldsymbol{\omega}_S^\top \mathbf{x}_i))^\top.$$ The resultant kernel estimate is: $$k_{ij} \approx \frac{1}{S} \sum_{s=1}^S \cos(\boldsymbol{\omega}_s^\top (\mathbf{x}_i-\mathbf{x}_j)).$$ This mechanism, in contrast to fixed $p(\boldsymbol{\omega})$ strategies, endows the kernel with nonparametric adaptivity, as the RBM parameters can be learned from data to reflect the underlying geometry and structure.

4. Regression Estimators and Nonnegative Squared-Kernel Weights

In NW regression, finite-sample kernel estimates can admit negative or cancelling contributions, endangering the stability of predictions through vanishing denominators. The QA-in-the-loop framework addresses this by using squared kernel entries as regression weights: $w_{ij} = k_{ij}^2 \geq 0.$ This construction ensures strictly positive weights, magnifies contrast between neighbor relationships, and stabilizes the estimator. The leave-one-out NW predictor is: $$\hat{y}_i^{(-i)} = \frac{\sum_{j \neq i} w_{ij} y_j}{\sum_{j \neq i} w_{ij}}.$$ At prediction, local linear regression (LLR) with the same weights is also introduced for further bias reduction, especially at the boundary: $$(\hat{\beta}_0, \hat{\boldsymbol{\beta}}) = \arg\min_{\beta_0, \boldsymbol{\beta}} \sum_j w_{*j} (y_j - \beta_0 - \boldsymbol{\beta}^\top (\mathbf{x}_j - \mathbf{x}_*))^2,$$ with the LLR prediction set to $\hat{y}_{\mathrm{LLR}} = \hat{\beta}_0$ (Hasegawa et al., 13 Jan 2026).

5. Training Objective, Differentiability, and Learning Procedure

Parameter learning proceeds via minimization of the leave-one-out mean squared error: $$L(\theta) = \frac{1}{N} \sum_{i=1}^N (y_i - \hat{y}_i^{(-i)}(\theta))^2.$$ Gradients are tractable via the chain rule: $$\frac{\partial w_{ij}}{\partial \theta} = 2k_{ij} \frac{\partial k_{ij}}{\partial \theta},$$ with

$$\frac{\partial k_{ij}}{\partial \theta} \approx \frac{1}{S} \sum_{s=1}^S \cos(\boldsymbol{\omega}_s^\top (\mathbf{x}_i-\mathbf{x}_j)) \frac{\partial \log p_\theta(\boldsymbol{\omega}_s)}{\partial \theta}.$$

Given $$p_\theta(\boldsymbol{\omega}) \propto \exp(-E_\theta(\boldsymbol{\omega}))$$, $$\partial \log p_\theta = -\partial E_\theta - \partial \log Z$$, both expectations being accessible through the same QA+RBM samples. In practice, alternation between quantum-annealed sampling, feature computation, leave-one-out regression, and gradient-based parameter updates yields an end-to-end trainable, hardware-assisted learning cycle (Hasegawa et al., 13 Jan 2026).

6. Empirical Performance and Structural Insights

Benchmark experiments (bodyfat, Mackey–Glass, energy efficiency, concrete compressive strength) demonstrate steady decreases in training loss upon learning, accompanied by structural evolution of the kernel matrix—manifested by block or cluster patterns aligning with inherent data groupings. Out-of-sample performance, as measured by $R^2$ and RMSE, exceeds that of Gaussian-kernel NW regression, particularly as the number of Monte Carlo features increases. Endpoint-corrected LLR provides further improvements, especially in boundary regions. Empirical histograms of spectral samples post-training exhibit pronounced deviations from Gaussianity, confirming the capacity to learn and exploit multimodal, data-structured spectral representations (Hasegawa et al., 13 Jan 2026).

7. Future Directions and Broader Context

Potential extensions include systematic studies of QA hardware noise and effective temperature impacts on kernel quality, scaling to high-dimensional settings with sparse or deep Boltzmann models, and incorporating flexible generative mappings (e.g., normalizing flows) for further adaptability. Broader deployment to problems such as classification, Bayesian GP regression, and robust regression via alternative loss functions (e.g., Huber, quantile) is viable. Modulations of annealing schedules and quantum control parameters may further enhance the diversity and utility of generated samples. This suggests a fertile avenue for further integration of quantum resources in statistical kernel learning and nonlinear regression (Hasegawa et al., 13 Jan 2026).