Comment on: "Low significance of evidence for black hole echoes in gravitational wave data"

Abstract: In a recent publication (1612.00266), we demonstrated that the events in the first observing run of the Advanced LIGO gravitational wave observatory (aLIGO O1) showed tentative evidence for repeating "echoes from the abyss" caused by Planck-scale structure near black hole horizons. By considering a phenomenological echo model, we showed that the pure noise hypothesis is disfavored with a p-value of 1%, i.e. higher amplitude for echoes than those in aLIGO O1 events are only recovered in 1% of random noise realizations. A recent preprint by Westerweck, et al. (1712.09966) provides a careful re-evaluation of our analysis which claims "a reduced statistical significance ... entirely consistent with noise". It is a mystery to us why the authors make such a statement, while they also find a p-value of 2 $\pm$ 1% (given the Poisson error in their estimate) for the same model and dataset. This is p-erfectly consistent with our results, which would be commonly considered as disfavoring the null hypothesis, or "moderate to significant" evidence for "echoes". Westerweck, et al. also point to diversity of the observed echo properties as evidence for statistical fluke, but such a diversity is neither unique nor surprising for complex physical phenomena in nature.