Metric Entropy of Ellipsoids in Banach Spaces: Techniques and Precise Asymptotics

Abstract: We develop new techniques for computing the metric entropy of ellipsoids -- with polynomially decaying semi-axes -- in Banach spaces. Besides leading to a unified and comprehensive framework, these tools deliver numerous novel results as well as substantial improvements and generalizations of classical results. Specifically, we characterize the constant in the leading term in the asymptotic expansion of the metric entropy of $p$-ellipsoids with respect to $q$-norm, for arbitrary $p,q \in [1, \infty]$, to date known only in the case $p=q=2$. Moreover, for $p=q=2$, we improve upon classical results by specifying the second-order term in the asymptotic expansion. In the case $p=q=\infty$, we obtain a complete, as opposed to asymptotic, characterization of metric entropy and explicitly construct optimal coverings. To the best of our knowledge, this is the first exact characterization of the metric entropy of an infinite-dimensional body. Application of our general results to function classes yields an improvement of the asymptotic expansion of the metric entropy of unit balls in Sobolev spaces and identifies the dependency of the metric entropy of unit balls in Besov spaces on the domain of the functions in the class. Sharp results on the metric entropy of function classes find application, e.g., in machine learning, where they allow to specify the minimum required size of deep neural networks for function approximation, nonparametric regression, and classification over these function classes.