Simultaneous Control of All False Discovery Proportions in Large-Scale Multiple Hypothesis Testing

Abstract: Closed testing procedures are classically used for familywise error rate (FWER) control, but they can also be used to obtain simultaneous confidence bounds for the false discovery proportion (FDP) in all subsets of the hypotheses. In this paper we investigate the special case of closed testing with Simes local tests. We construct a novel fast and exact shortcut which we use to investigate the power of this method when the number of hypotheses goes to infinity. We show that, if a minimal amount of signal is present, the average power to detect false hypotheses at any desired FDP level does not vanish. Additionally, we show that the confidence bounds for FDP are consistent estimators for the true FDP for every non-vanishing subset. For the case of a finite number of hypotheses, we show connections between Simes-based closed testing and the procedure of Benjamini and Hochberg.