Extensions beyond the single-neuron and two-latent setting

Extend the theoretical analysis developed for single-neuron autoencoders and the two-latent spiked cumulant model to multi-neuron architectures, to richer latent-dependence structures, and to other self-supervised objectives such as contrastive or masked predictive losses.

Background

The current work analyzes a minimal single-hidden-unit autoencoder and a specific class of higher-order dependencies that render one spike invisible to PCA but learnable by nonlinear methods.

Generalizing the solvable framework to wider architectures, more complex latent dependencies, and different self-supervised paradigms would broaden the scope of the theoretical insights and connect more directly to modern practice.