2000 character limit reached
Which statistical hypotheses are afflicted with false confidence?
Published 24 Apr 2024 in math.ST and stat.TH | (2404.16228v1)
Abstract: The false confidence theorem establishes that, for any data-driven, precise-probabilistic method for uncertainty quantification, there exists (non-trivial) false hypotheses to which the method tends to assign high confidence. This raises concerns about the reliability of these widely-used methods, and shines new light on the consonant belief function-based methods that are provably immune to false confidence. But an existence result alone is insufficient. Towards a partial answer to the title question, I show that, roughly, complements of convex hypotheses are afflicted by false confidence.
- Balch, M. S. (2012). Mathematical foundations for a theory of confidence structures. Internat. J. Approx. Reason., 53(7):1003–1019.
- Satellite conjunction analysis and the false confidence theorem. Proc. Royal Soc. A, 475(2227):2018.0565.
- An exposition of the false confidence theorem. Stat, 7(1):e201.
- Confidence in confidence distributions! Proc. Roy. Soc. A, 476:20190781.
- Dawid, A. P. (2020). Fiducial inference then and now. arXiv:2012.10689.
- Dempster, A. P. (1966). New methods for reasoning towards posterior distributions based on sample data. Ann. Math. Statist., 37:355–374.
- Dempster, A. P. (1968). A generalization of Bayesian inference. (With discussion). J. Roy. Statist. Soc. Ser. B, 30:205–247.
- Dempster, A. P. (2008). The Dempster–Shafer calculus for statisticians. Internat. J. Approx. Reason., 48(2):365–377.
- Denœux, T. (2014). Likelihood-based belief function: justification and some extensions to low-quality data. Internat. J. Approx. Reason., 55(7):1535–1547.
- Frequency-calibrated belief functions: review and new insights. Internat. J. Approx. Reason., 92:232–254.
- Possibility Theory. Plenum Press, New York.
- Fisher, R. A. (1930). Inverse probability. Proceedings of the Cambridge Philosophical Society, 26:528–535.
- Fisher, R. A. (1935). The fiducial argument in statistical inference. Ann. Eugenics, 6:391–398.
- Fraser, D. A. S. (2011). Is Bayes posterior just quick and dirty confidence? Statist. Sci., 26(3):299–316.
- Fraser, D. A. S. (2014). Why does statistics have two theories? In Lin, X., Genest, C., Banks, D. L., Molenberghs, G., Scott, D. W., and Wang, J.-L., editors, Past, Present, and Future of Statistical Science, chapter 22. Chapman & Hall/CRC Press.
- Hacking, I. (1976). Logic of Statistical Inference. Cambridge University Press, Cambridge-New York-Melbourne.
- Generalized fiducial inference: a review and new results. J. Amer. Statist. Assoc., 111(515):1346–1361.
- Hose, D. (2022). Possibilistic Reasoning with Imprecise Probabilities: Statistical Inference and Dynamic Filtering. PhD thesis, University of Stuttgart.
- Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proc. Roy. Soc. London Ser. A, 186:453–461.
- Martin, R. (2015). Plausibility functions and exact frequentist inference. J. Amer. Statist. Assoc., 110(512):1552–1561.
- Martin, R. (2018). On an inferential model construction using generalized associations. J. Statist. Plann. Inference, 195:105–115.
- Martin, R. (2019). False confidence, non-additive beliefs, and valid statistical inference. Internat. J. Approx. Reason., 113:39–73.
- Martin, R. (2021). An imprecise-probabilistic characterization of frequentist statistical inference. arXiv:2112.10904.
- Martin, R. (2022a). Valid and efficient imprecise-probabilistic inference with partial priors, I. First results. arXiv:2203.06703.
- Martin, R. (2022b). Valid and efficient imprecise-probabilistic inference with partial priors, II. General framework. arXiv:2211.14567.
- Martin, R. (2023a). Fiducial inference viewed through a possibility-theoretic inferential model lens. In Miranda, E., Montes, I., Quaeghebeur, E., and Vantaggi, B., editors, Proceedings of the Thirteenth International Symposium on Imprecise Probability: Theories and Applications, volume 215 of Proceedings of Machine Learning Research, pages 299–310. PMLR.
- Martin, R. (2023b). Fisher’s underworld and the behavioral–statistical reliability balance in scientific inference. arXiv:2312.14912.
- Martin, R. (2023c). A possibility-theoretic solution to Basu’s Bayesian–frequentist via media. Sankhya A, to appear, arXiv:2303.17425.
- Response to the comment ‘Confidence in confidence distributions!’. Proc. R. Soc. A., 477:20200579.
- Inferential Models, volume 147 of Monographs on Statistics and Applied Probability. CRC Press, Boca Raton, FL.
- Shafer, G. (1976). A Mathematical Theory of Evidence. Princeton University Press, Princeton, N.J.
- Shafer, G. (1982). Belief functions and parametric models. J. Roy. Statist. Soc. Ser. B, 44(3):322–352. With discussion.
- Stein, C. (1959). An example of wide discrepancy between fiducial and confidence intervals. Ann. Math. Statist., 30:877–880.
- van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge University Press, Cambridge.
- Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities, volume 42 of Monographs on Statistics and Applied Probability. Chapman & Hall Ltd., London.
- Wasserman, L. A. (1990). Belief functions and statistical inference. Canad. J. Statist., 18(3):183–196.
- Confidence distribution, the frequentist distribution estimator of a parameter: a review. Int. Stat. Rev., 81(1):3–39.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.