Asymptotic optimality of sparse linear discriminant analysis with arbitrary number of classes

Abstract: Many sparse linear discriminant analysis (LDA) methods have been proposed to overcome the major problems of the classic LDA in high-dimensional settings. However, the asymptotic optimality results are limited to the case that there are only two classes, which is due to the fact that the classification boundary of LDA is a hyperplane and explicit formulas exist for the classification error in this case. In the situation where there are more than two classes, the classification boundary is usually complicated and no explicit formulas for the classification errors exist. In this paper, we consider the asymptotic optimality in the high-dimensional settings for a large family of linear classification rules with arbitrary number of classes under the situation of multivariate normal distribution. Our main theorem provides easy-to-check criteria for the asymptotic optimality of a general classification rule in this family as dimensionality and sample size both go to infinity and the number of classes is arbitrary. We establish the corresponding convergence rates. The general theory is applied to the classic LDA and the extensions of two recently proposed sparse LDA methods to obtain the asymptotic optimality. We conduct simulation studies on the extended methods in various settings.