A study of two periodogram algorithms for improving the detection of small transiting planets

Abstract: The sensitivities of two periodograms are compared for weak signal planet detection in transit surveys: the widely used Box-Least Squares (BLS) algorithm following light curve detrending and the Transit Comb Filter (TCF) algorithm following autoregressive ARIMA modeling. Small depth transits are injected into light curves with different simulated noise characteristics. Two measures of spectral peak significance are examined: the periodogram signal-to-noise ratio (SNR) and a False Alarm Probability (FAP) based on the generalized extreme value distribution. The relative performance of the BLS and TCF algorithms for small planet detection is examined for a range of light curve characteristics, including orbital period, transit duration, depth, number of transits, and type of noise. We find that the TCF periodogram applied to ARIMA fit residuals with the SNR detection metric is preferred when short-memory autocorrelation is present in the detrended light curve and even when the light curve noise had white Gaussian noise. BLS is more sensitive to small planets only under limited circumstances with the FAP metric. BLS periodogram characteristics are inferior when autocorrelated noise is present due to heteroscedastic noise and false period detection. Application of these methods to TESS light curves with known small exoplanets confirms our simulation results. The study ends with a decision tree that advises transit survey scientists on procedures to detect small planets most efficiently. The use of ARIMA detrending and TCF periodograms can significantly improve the sensitivity of any transit survey with regularly spaced cadence.