Roy's largest root under rank-one alternatives:The complex valued case and applications

Abstract: The largest eigenvalue of a Wishart matrix, known as Roy's largest root (RLR), plays an important role in a variety of applications. Most works to date derived approximations to its distribution under various asymptotic regimes, such as degrees of freedom, dimension, or both tending to infinity. However, several applications involve finite and relative small parameters, for which the above approximations may be inaccurate. Recently, via a small noise perturbation approach with fixed dimension and degrees of freedom, Johnstone and Nadler derived simple yet accurate stochastic approximations to the distribution of Roy's largest root in the real valued case, under a rank-one alternative. In this paper, we extend their results to the complex valued case. Furthermore, we analyze the behavior of the leading eigenvector by developing new stochastic approximations. Specifically, we derive simple stochastic approximations to the distribution of the largest eigenvalue under five common complex single-matrix and double-matrix scenarios. We then apply these results to investigate several problems in signal detection and communications. In particular, we analyze the performance of RLR detector in cognitive radio spectrum sensing and constant modulus signal detection in the high signal-to-noise ratio (SNR) regime. Moreover, we address the problem of determining the optimal transmit-receive antenna configuration (here optimality is in the sense of outage minimization) for rank-one multiple-input multiple-output Rician Fading channels at high SNR.