Free Anytime Validity by Sequentializing a Test and Optional Continuation with Tests as Future Significance Levels

Abstract: Anytime valid sequential tests permit us to stop and continue testing based on the current data, without invalidating the inference. Given a maximum number of observations $N$, one may believe this must come at the cost of power when compared to a conventional test that waits until all $N$ observations have arrived. Our first contribution is to show that this is false: for any valid test based on $N$ observations, we derive an anytime valid sequential test that matches it after $N$ observations. Our second contribution is that the outcome of a continuously-interpreted test can be used as a significance level in subsequent testing, leading to an overall procedure that is valid at the original significance level. This shows anytime validity and optional continuation are readily available in traditional testing, without requiring explicit use of e-values. We illustrate this by deriving the anytime valid sequentialized $z$-test and $t$-test, which at time $N$ coincide with the traditional $z$-test and $t$-test. Lastly, we show the popular log-optimal sequential $z$-test can be interpreted as desiring a rejection by the traditional $z$-test at some tiny significance level in the distant future.