Sampling Theory for Function Approximation with Numerical Redundancy

Abstract: We explore the interaction between numerical rounding errors and discretization errors involved in linear function approximation schemes. To precisely identify when numerical effects become significant, we introduce the concept of numerical redundancy: a set of functions is numerically redundant if, due to finite-precision arithmetic, it spans a lower-dimensional space numerically than it does analytically. This phenomenon arises in a variety of settings, though it is not always explicitly identified as such. When such a set is used as a basis for numerical approximations, recovering the expansion coefficients of the best approximation is generally impossible. More specifically, we show that approximations produced by practical algorithms-affected by rounding errors-are implicitly subject to $\ell^2$-regularization. The computed solutions, therefore, exhibit reduced accuracy compared to the best approximation. On the other hand, there is also a benefit: regularization reduces the amount of data needed for accurate and stable discrete approximation. The primary aim of this paper is to develop the tools needed to fully understand this effect. Additionally, we present a constructive method for optimally selecting random data points for $L^2$-approximations, explicitly accounting for the influence of rounding errors. We illustrate our results in two typical scenarios that lead to numerical redundancy: (1) approximation on irregular domains and (2) approximation that incorporate specific features of the function to be approximated. In doing so, we obtain new results on random sampling for Fourier extension frames.