Approximation rates for invariant approximation using generators, separating sets, and generic separators

Determine the quantitative approximation rates for uniform function approximation in the Stone–Weierstrass framework when learning invariant functions using different invariant feature sets—specifically, sets of algebra generators of the invariant algebra, orbit-separating sets of at most 2D+1 functions (where D is the dimension of the orbit space), and generic separating sets such as field generators—and compare how these rates differ in practice.

Background

In the discussion of constructing small sets of invariants for learning, the paper contrasts algebra generators of the invariant algebra with smaller separating sets (bounded by 2D+1, with D the orbit-space dimension) and with generic separating sets (e.g., field generators). Through Stone–Weierstrass, these sets are said to have the same expressive power with respect to universality of approximation.

However, the authors point out that this equivalence is coarse because it ignores potentially different approximation rates. They explicitly state that these approximation rates are generally not known, highlighting an unresolved quantitative question about how fast approximation improves as model complexity increases when using different types of invariant feature sets.