Bootstrap confidence sets for spectral projectors of sample covariance

Abstract: Let $X_{1},\ldots,X_{n}$ be i.i.d. sample in $\mathbb{R}^{p}$ with zero mean and the covariance matrix $\mathbf{\Sigma}$. The problem of recovering the projector onto an eigenspace of $\mathbf{\Sigma}$ from these observations naturally arises in many applications. Recent technique from [Koltchinskii, Lounici, 2015] helps to study the asymptotic distribution of the distance in the Frobenius norm $| \mathbf{P}r - \widehat{\mathbf{P}}_r |{2}$ between the true projector $\mathbf{P}r$ on the subspace of the $r$-th eigenvalue and its empirical counterpart $\widehat{\mathbf{P}}_r$ in terms of the effective rank of $\mathbf{\Sigma}$. This paper offers a bootstrap procedure for building sharp confidence sets for the true projector $\mathbf{P}_r$ from the given data. This procedure does not rely on the asymptotic distribution of $| \mathbf{P}_r - \widehat{\mathbf{P}}_r |{2}$ and its moments. It could be applied for small or moderate sample size $n$ and large dimension $p$. The main result states the validity of the proposed procedure for finite samples with an explicit error bound for the error of bootstrap approximation. This bound involves some new sharp results on Gaussian comparison and Gaussian anti-concentration in high-dimensional spaces. Numeric results confirm a good performance of the method in realistic examples.