The Kolmogorov-Smirnov Statistic Revisited

Abstract: The Kolmogorov-Smirnov (KS) statistic is a classical nonparametric test widely used for comparing an empirical distribution function with a reference distribution or for comparing two empirical distributions. Despite its broad applicability in statistical hypothesis testing and model validation, certain aspects of the KS statistic remain under-explored among the young generation, particularly under finite sample conditions. This paper revisits the KS statistic in both one-sample and two-sample scenarios, considering one-sided and two-sided variants. We derive exact probabilities for the supremum of the empirical process and present a unified treatment of the KS statistic under diverse settings. Additionally, we explore the discrete nature of the hitting times of the normalized empirical process, providing practical insights into the computation of KS test p-values. The study also discusses the Dvoretzky-Kiefer-Wolfowitz-Massart (DKWM) inequality, highlighting its role in constructing confidence bands for distribution functions. Using empirical process theory, we establish the limit distribution of the KS statistic when the true distribution includes unknown parameters. Our findings extend existing results, offering improved methodologies for statistical analysis and hypothesis testing using the KS statistic, particularly in finite sample scenarios.