Alternative Likelihood Approximations for High-Dimensional Intervals for Lasso

Abstract: Classical frequentist approaches to inference for the lasso emphasize exact coverage for each feature, which requires debiasing and severs the connection between confidence intervals and the original lasso estimates. To address this, in earlier work we introduced the idea of average coverage, allowing for biased intervals that align with the lasso point estimates, and proposed the Relaxed Lasso Posterior (RL-P) intervals, which leverage the Bayesian interpretation of the lasso penalty as a Laplace prior together with a Normal likelihood conditional on the selected features. While RL-P achieves approximate average coverage, its intervals need not contain the lasso estimates. In this work, we propose alternative constructions based on different likelihood approximations to the full high-dimensional likelihood, yielding intervals that remain centered on the lasso estimates while still achieving average coverage. Our results continue to demonstrate that intentionally biased intervals provide a principled and practically useful framework for inference in high-dimensional regression.