The Power of Tests for Detecting $p$-Hacking

Abstract: $p$-Hacking undermines the validity of empirical studies. A flourishing empirical literature investigates the prevalence of $p$-hacking based on the distribution of $p$-values across studies. Interpreting results in this literature requires a careful understanding of the power of methods for detecting $p$-hacking. We theoretically study the implications of likely forms of $p$-hacking on the distribution of $p$-values to understand the power of tests for detecting it. Power depends crucially on the $p$-hacking strategy and the distribution of true effects. Publication bias can enhance the power for testing the joint null of no $p$-hacking and no publication bias.