2000 character limit reached
An information-theoretic lower bound in time-uniform estimation
Published 13 Feb 2024 in cs.IT, math.IT, math.ST, and stat.TH | (2402.08794v2)
Abstract: We present an information-theoretic lower bound for the problem of parameter estimation with time-uniform coverage guarantees. Via a new a reduction to sequential testing, we obtain stronger lower bounds that capture the hardness of the time-uniform setting. In the case of location model estimation, logistic regression, and exponential family models, our $\Omega(\sqrt{n{-1}\log \log n})$ lower bound is sharp to within constant factors in typical settings.
- L. D. Brown. Fundamentals of Statistical Exponential Families. Institute of Mathematical Statistics, Hayward, California, 1986.
- F. Chyzak and F. Nielsen. A closed-form formula for the Kullback-Leibler divergence between Cauchy distributions. arXiv:1905.10965 [cs.IT], 2019.
- Elements of Information Theory, Second Edition. Wiley, 2006.
- D. A. Darling and H. Robbins. Confidence sequences for mean, variance, and median. Proceedings of the National Academy of Sciences, 58(1):66, 1967.
- J. C. Duchi. Stats311/EE377: Information theory and statistics. Course at Stanford University, Fall 2023. URL http://web.stanford.edu/class/stats311.
- R. H. Farrell. Asymptotic behavior of expected sample size in certain one sided tests. Annals of Mathematical Statistics, 35(1):36–72, 1964.
- Time-uniform, nonparametric, nonasymptotic confidence sequences. Annals of Statistics, 49(2):1055–1080, 2021.
- lil’UCB: An optimal exploration algorithm for multi-armed bandits. In Proceedings of the Twenty Seventh Annual Conference on Computational Learning Theory, pages 423–439. PMLR, 2014.
- C. Jennison and B. W. Turnbull. Interim analyses: the repeated confidence interval approach. Journal of the Royal Statistical Society, Series B, 51(3):305–334, 1989.
- Always valid inference: Continuous monitoring of A/B tests. Operations Research, pages 1806–1821, 2022.
- On the complexity of best arm identification in multi-armed bandit models. Journal of Machine Learning Research, 17(1):1–42, 2016.
- S. Kullback. Information Theory and Statistics. Dover Publications, 1968.
- T.-L. Lai. On confidence sequences. The Annals of Statistics, 4(2):265–280, 1976.
- T.-L. Lai. Incorporating scientific, ethical and economic considerations into the design of clinical trials in the pharmaceutical industry: a sequential approach. Communications in Statistics – Theory and Methods, 13(19):2355–2368, 1984.
- E. L. Lehmann and G. Casella. Theory of Point Estimation, Second Edition. Springer, 1998.
- H. Robbins. Statistical methods related to the law of the iterated logarithm. Annals of Mathematical Statistics, 41(5):1397–1409, 1970.
- A. B. Tsybakov. Introduction to Nonparametric Estimation. Springer, 2009.
- M. J. Wainwright. High-Dimensional Statistics: A Non-Asymptotic Viewpoint. Cambridge University Press, 2019.
- B. Yu. Assouad, Fano, and Le Cam. In Festschrift for Lucien Le Cam, pages 423–435. Springer-Verlag, 1997.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.