Optimal nonparametric estimation of the expected shortfall risk
Abstract: We address the problem of estimating the expected shortfall risk of a financial loss using a finite number of i.i.d. data. It is well known that the classical plug-in estimator suffers from poor statistical performance when faced with (heavy-tailed) distributions that are commonly used in financial contexts. Further, it lacks robustness, as the modification of even a single data point can cause a significant distortion. We propose a novel procedure for the estimation of the expected shortfall and prove that it recovers the best possible statistical properties (dictated by the central limit theorem) under minimal assumptions and for all finite numbers of data. Further, this estimator is adversarially robust: even if a (small) proportion of the data is maliciously modified, the procedure continuous to optimally estimate the true expected shortfall risk. We demonstrate that our estimator outperforms the classical plug-in estimator through a variety of numerical experiments across a range of standard loss distributions.
- C. Acerbi. Coherent measures of risk in everyday market practice. 2007.
- The space complexity of approximating the frequency moments. In Proceedings of the twenty-eighth annual ACM symposium on Theory of computing, pages 20–29, 1996.
- D. W. Andrews. Stability comparison of estimators. Econometrica: Journal of the Econometric Society, pages 1207–1235, 1986.
- Coherent measures of risk. Math. Finance, 9:203–228, 1999.
- J. Beirlant and J. L. Teugels. Modeling large claims in non-life insurance. Insurance: Mathematics and Economics, 11(1):17–29, 1992.
- Concentration inequalities. In Summer school on machine learning, pages 208–240. Springer, 2003.
- Estimating conditional tail expectation with actuarial applications in view. Journal of Statistical Planning and Inference, 138(11):3590–3604, 2008.
- On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705, 2019.
- S. X. Chen. Nonparametric estimation of expected shortfall. Journal of financial econometrics, 6(1):87–107, 2008.
- Total stability of kernel methods. Neurocomputing, 289:101–118, 2018.
- Robustness and sensitivity analysis of risk measurement procedures. Quantitative finance, 10(6):593–606, 2010.
- Risk measures and comonotonicity: a review. Stochastic models, 22(4):573–606, 2006.
- D. Duffie and J. Pan. An overview of value at risk. Journal of derivatives, 4(3):7–49, 1997.
- Dimensionality reduction and wasserstein stability for kernel regression. Journal of Machine Learning Research, 24(334):1–35, 2023.
- An academic response to basel 3.5. Risks, 2(1):25–48, 2014.
- W. Feller. Generalization of a probability limit theorem of cramér. Transactions of the American Mathematical Society, 54(3):361–372, 1943.
- Making machine learning robust against adversarial inputs. Communications of the ACM, 61(7):56–66, 2018.
- J. B. Hill. Expected shortfall estimation and gaussian inference for infinite variance time series. Journal of Financial Econometrics, 13(1):1–44, 2015.
- J. L. Horowitz. Bootstrap methods in econometrics. Annual Review of Economics, 11:193–224, 2019.
- Y. Hyndman, Rob J.; Fan. Sample quantiles in statistical packages. The American Statistician 1996-nov vol. 50 iss. 4, 50, nov 1996.
- Random generation of combinatorial structures from a uniform distribution. Theoretical computer science, 43:169–188, 1986.
- B. L. Jones and R. Zitikis. Empirical estimation of risk measures and related quantities. North American Actuarial Journal, 7(4):44–54, 2003.
- J. Kim and C. D. Scott. Robust kernel density estimation. The Journal of Machine Learning Research, 13(1):2529–2565, 2012.
- Quantitative risk management: concepts, techniques and tools-revised edition. Princeton university press, 2015.
- S. Minsker. U-statistics of growing order and sub-gaussian mean estimators with sharp constants. arXiv preprint arXiv:2202.11842, 2022.
- S. Minsker. Efficient median of means estimator. arXiv preprint arXiv:2305.18681, 2023.
- Estimation methods for expected shortfall. Quantitative Finance, 14(2):271–291, 2014.
- Estimating the conditional tail expectation in the case of heavy-tailed losses. Journal of Probability and Statistics, 2010, 2010.
- Problem complexity and method efficiency in optimization. 1983.
- Adversarial robustness toolbox v1.2.0. CoRR, 1807.01069, 2018.
- Calculating cvar and bpoe for common probability distributions with application to portfolio optimization and density estimation. Annals of Operations Research, 299:1281–1315, 2021.
- G. Pflug and N. Wozabal. Asymptotic distribution of law-invariant risk functionals. Finance and Stochastics, 14(3):397–418, 2010.
- R. T. Rockafellar and S. Uryasev. Optimization of conditional value-at-risk. Jounal of Risk, 2:21–42, 2000.
- R. T. Rockafellar and S. Uryasev. Conditional value-at-risk for general loss distributions. Journal of banking & finance, 26(7):1443–1471, 2002.
- P. J. Rousseeuw and M. Hubert. Robust statistics for outlier detection. Wiley interdisciplinary reviews: Data mining and knowledge discovery, 1(1):73–79, 2011.
- Robust regression and outlier detection. John wiley & sons, 2005.
- Distortion risk measures in portfolio optimization. Handbook of portfolio construction, pages 649–673, 2010.
- I. Steinwart and A. Christmann. Support vector machines. Springer Science & Business Media, 2008.
- H. Tsukahara. Estimation of distortion risk measures. Journal of Financial Econometrics, 12(1):213–235, 2014.
- M. V. Wüthrich and M. Merz. Statistical foundations of actuarial learning and its applications. Springer Nature, 2023.
- Y. Yamai and T. Yoshiba. Value-at-risk versus expected shortfall: A practical perspective. Journal of Banking & Finance, 29(4):997–1015, 2005.
- Comparative analyses of expected shortfall and value-at-risk: their estimation error, decomposition, and optimization. Monetary and economic studies, 20(1):87–121, 2002.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.