Detection of Signals in Colored Noise: Leading Eigenvalue Test for Non-central $F$-matrices

Abstract: This paper investigates the signal detection problem in colored noise with an unknown covariance matrix. In particular, we focus on detecting an unknown non-random signal by capitalizing on the leading eigenvalue of the whitened sample covariance matrix as the test statistic (a.k.a. Roy's largest root test). Since the unknown signal is non-random, the whitened sample covariance matrix turns out to have a non-central $F$-distribution. This distribution assumes a singular or non-singular form depending on whether the number of observations $p\lessgtr$ the system dimensionality $m$. Therefore, we statistically characterize the leading eigenvalue of the singular and non-singular $F$-matrices by deriving their cumulative distribution functions (c.d.f.). Subsequently, they have been utilized in deriving the corresponding receiver operating characteristic (ROC) profiles. We also extend our analysis into the high dimensional domain. It turns out that, when the signal is sufficiently strong, the maximum eigenvalue can reliably detect it in this regime. Nevertheless, weak signals cannot be detected in the high dimensional regime with the leading eigenvalue.