Debiasing the Debiased Lasso with Bootstrap

Abstract: We consider statistical inference for a single coordinate of regression coefficients in high-dimensional linear models. Recently, the debiased estimators are popularly used for constructing confidence intervals and hypothesis testing in high-dimensional models. However, some representative numerical experiments show that they tend to be biased for large coefficients, especially when the number of large coefficients dominates the number of small coefficients. In this paper, we propose a modified debiased Lasso estimator based on bootstrap. Let us denote the proposed estimator BS-DB for short. We show that, under the irrepresentable condition and other mild technical conditions, the BS-DB has smaller order of bias than the debiased Lasso in existence of a large proportion of strong signals. If the irrepresentable condition does not hold, the BS-DB is guaranteed to perform no worse than the debiased Lasso asymptotically. Confidence intervals based on the BS-DB are proposed and proved to be asymptotically valid under mild conditions. Our study on the inference problems integrates the properties of the Lasso on variable selection and estimation novelly. The superior performance of the BS-DB over the debiased Lasso is demonstrated via extensive numerical studies.