Minimax optimality of RaT beyond kernel-based settings

Determine whether the residual-as-teacher (RaT) estimator achieves minimax-optimal convergence rates outside the kernel ridge regression student–teacher framework analyzed in the paper, particularly for more general student–teacher pairs and non-kernel model classes under covariate shift.

Background

The paper proves that, for kernel ridge regression (KRR) student–teacher pairs under covariate shift with a biased teacher, RaT achieves the minimax-optimal rate while standard student soft-matching is inconsistent. This is established via exact mean-squared error formulas and a separation theorem.

The authors explicitly raise the question of whether this optimality persists when moving beyond the specific kernel-based construction to broader student–teacher interactions and non-kernel settings.