Two-Sample Test Based on Classification Probability

Abstract: Robust classification algorithms have been developed in recent years with great success. We take advantage of this development and recast the classical two-sample test problem in the framework of classification. Based on the estimates of classification probabilities from a classifier trained from the samples, a test statistic is proposed. We explain why such a test can be a powerful test and compare its performance in terms of the power and efficiency with those of some other recently proposed tests with simulation and real-life data. The test proposed is nonparametric and can be applied to complex and high dimensional data wherever there is a classifier that provides consistent estimate of the classification probability for such data.