Bounds on minimum depth for manifold simplification to linear separability

Determine tight upper and lower bounds on the minimum number of layers required in a feedforward network with threshold/ReLU activations to transform low-dimensional data manifolds embedded in high-dimensional ambient space into representations that are linearly separable by a single hyperplane at the output layer.

Background

Section 4 argues that depth operates by folding and simplifying data manifolds via threshold/ReLU operations so that a final linear classifier suffices. While qualitative mechanisms are described, quantitative depth requirements are not established.

The conclusion identifies the need for bounding the minimum depth necessary to achieve such manifold simplification as an open question.

References

Open questions remain. The formal derivation of perceptron freedom from the four geometric properties of high-dimensional space; the bounds on minimum depth required for manifold simplification; the connections between the semiotic interpretation and philosophical debates about understanding in AI; and the practical implications for architecture design - all invite further investigation.

— Understanding the Nature of Generative AI as Threshold Logic in High-Dimensional Space  (2604.02476 - Levin, 2 Apr 2026) in Conclusion (Section 6), final paragraph