Geometric State-Space Neural Network

• Geometric state-space neural networks are models that embed neural dynamics on non-Euclidean manifolds, ensuring that state evolution respects curvature and symmetry.
• They leverage tools from Riemannian, Hessian, and Lie group geometries to generalize classical state-space models with metric-preserving updates and dual coordinate formulations.
• Applications span dynamical systems, neuroscience, quantum tomography, and reinforcement learning, showcasing improved robustness, interpretability, and efficiency.

A geometric state-space neural network is a neural architecture and modeling paradigm in which neural dynamics, transformations, latent states, or learning algorithms are constructed intrinsically on a non-Euclidean geometric manifold, typically with explicit control of the network's geometry via differential, Riemannian, or information-geometric structure. This approach generalizes classical state-space models (SSMs) and neural networks by allowing state evolution, parameterization, and optimization to respect manifold constraints, curvature, and symmetries rather than operating in ambient Euclidean space. Prominent theoretical and applied developments have established geometric state-space neural networks as powerful tools across dynamical systems, physical modeling, manifold learning, neuroscientific time series, quantum state analysis, and sparse geometric data processing.

1. Geometric Foundations and State-Space Formulation

The geometric state-space neural network framework begins by endowing the set of neural states, latent variables, or system parameters with a manifold structure. For an $n$-dimensional system, one selects a smooth manifold $M$ of states $x=(x^1,...,x^n)$, and equips it with one (or more) geometric tools:

• Riemannian or Hessian geometry: Many instances use a Hessian manifold $(M,\psi)$, where a strictly convex potential function $\psi:M\rightarrow\mathbb{R}$ yields a metric $g_{ij}(x) = \partial_i\partial_j\psi(x)$ that is positive definite. This induces a natural geometry on $M$, used to compare and interpolate states and gradients (Goto, 2019).
• SPD (symmetric positive definite) manifold: For networks whose state is a covariance or connectivity matrix, the state space is the manifold $\mathcal{S}^n_{++}$ of $n\times n$ SPD matrices. Here, the affine-invariant metric $$g_P(\Delta_1,\Delta_2)=\mathrm{Tr}(P^{-1}\Delta_1P^{-1}\Delta_2)$$ ensures operations remain on $\mathcal{S}^n_{++}$ (Conti et al., 19 Feb 2025, Dan et al., 20 Jan 2026).
• Lie groups and symplectic/Poisson geometry: Neural networks can also be constructed directly on homogeneous spaces, e.g., the Poincaré disk, using Hamiltonian mechanics. The dynamics are defined by flows invariant under a Lie group action, as in networks on the lognormal statistical manifold (Assandje et al., 30 Sep 2025).

Within these structures, the state-space model is typically generalized as:

$\dot{x} = X(x) = -g^{ij}(x)\,\partial_j U(x)$

with $U(x)$ a smooth “loss”, “energy”, or “potential” function, and $g^{ij}$ the metric tensor's inverse (Goto, 2019, Halder et al., 2019). This differential equation is the natural (Riemannian) gradient flow of $U$, defining the neural update/transition law.

2. Dual Coordinates, Mirror Descent, and Information Geometry

The geometric structure induces canonical dual coordinates via the Legendre transform, connecting the theory to information geometry:

• Dual affine coordinates $(\theta,\eta)$: From a convex potential $\psi(x)$, define $\theta_i = \partial_i\psi(x)$, and the dual potential $\varphi(\theta) = x^i\theta_i - \psi(x)$. Then $\eta^i=\partial^i\varphi(\theta)$ gives primal coordinates. In information geometry, $(x^i, \theta_i)$ are dual flat coordinates (Goto, 2019).
• Mirror descent equivalence: Many geometric state-space flows—including Hopfield neural networks—can be equivalently described as mirror descent on the dual (Hessian) manifold. The iteration $x^+ = x - h\,G(x)^{-1}\nabla f(x)$ in the primal pulls back to a mirror step in dual coordinates, with the Bregman divergence $D_\psi$ induced by $\psi$ (Halder et al., 2019).

The selection of the activation function (e.g., sigmoid) determines the underlying geometry: for instance, using $\psi(u) = \log(1+e^u)$ yields a Fisher-like metric $g(u) = \phi(u)[1-\phi(u)]$ where $\phi$ is the sigmoid, and makes the network dynamics a gradient flow on the induced Hessian manifold.

3. Manifold-aware Neural Dynamics and Learning

The architecture and learning dynamics in geometric state-space neural networks explicitly respect the manifold structure:

• Riemannian natural-gradient flow: Updates are performed along the steepest direction measured by the manifold metric, rather than the Euclidean gradient. For instance, in SPD parameterizations, gradient steps are projected back onto the manifold via the exponential map (Conti et al., 19 Feb 2025).
• Discretization and high-order state-space embeddings: Modern networks (e.g., ResNets, DenseNets) can be viewed as time/multistep discretizations of geometric flows on the data manifold, with higher-order skip-connections effectively embedding the data in a $k$-jet bundle, increasing effective phase-space dimension to $k\cdot d$ for $k$-order models (Hauser et al., 2018).
• Geometric loss and metric preservation: Loss functions often enforce the geometry, e.g., by penalizing deviations in geodesic (Riemannian) distance between predicted and true states, or by metric-preserving loss terms that align latent Euclidean and manifold geodesic distances (Tomal et al., 16 Dec 2025, Conti et al., 19 Feb 2025).

Manifold-aware updates are essential when the state is constrained to live in SPD, Lie group, or more general non-Euclidean spaces, ensuring stability, symmetry, and proper system identification.

4. Network Architectures and Explicit Geometric Parameterizations

Construction of geometric state-space neural networks proceeds by parameterizing key operators or transitions in a geometry-aware manner:

• SPD parameterizations: Mapping unconstrained neural parameters to elements of $\mathcal{S}^n_{++}$ through matrix exponentiation (e.g., $S(\theta)=\frac{1}{2}[M(\theta)+M(\theta)^\top]$, then $\Phi_A(\theta)=\exp(S(\theta))$), or via Cholesky factorization, to ensure SPD structure (Conti et al., 19 Feb 2025, Dan et al., 20 Jan 2026).
• Hamiltonian/Lie-group dynamics: On homogeneous spaces, transition operators and activations are constructed as group exponentials and translations, e.g., rotation matrices from $\mathrm{SU}(1,1)$ action and symplectic exponential nonlinearities (Assandje et al., 30 Sep 2025).
• State-space models for geometric data: In the handling of sparse geometric data, the geometric structure is injected into the SSM via explicit use of relative coordinate differences as “step sizes” in state transitions (e.g., $\Delta_i = t_i - t_{i-1}$ in the STREAM model) (Schöne et al., 2024).

Manifold-valued architectures extend to spatiotemporal operator learning by factorizing spatial and temporal dimensions, with each factor represented as a structured state-space model, as in the ST-SSM (Koren et al., 31 Jul 2025).

5. Applications in Dynamical Systems, Physics, and Neuroscience

Geometric state-space neural networks have found application across a broad set of domains requiring manifold-valued modeling:

• Learning dynamics on $\mathcal{S}^n_{++}$: Identification of linear time-invariant (LTI) systems and brain functional connectivity is accomplished by parameterizing and learning SPD-valued discrete state transitions, enabling generalizable system reconstructions and interpretable low-dimensional embeddings (Conti et al., 19 Feb 2025, Dan et al., 20 Jan 2026).
• Neural operator learning: Spatiotemporal state-space operators with factorized spatial/temporal SSMs achieve parameter-efficient, theoretically universal operator regression on PDE tasks, outperforming classical neural operators under fixed parameter budgets (Koren et al., 31 Jul 2025).
• Quantum state tomography: Networks combining classical autoencoders and quantum circuit decoders, trained with metric-preserving losses, yield latent representations aligning Euclidean and Bures geodesic distances, enabling polynomial-scaling quantum tomography and geometric error estimation (Tomal et al., 16 Dec 2025).
• Reinforcement learning and manifold induced state compression: The space of attainable neural-policy-induced states in continuous RL can be shown to concentrate on a manifold of dimension $O(d_a)$, where $d_a$ is the action dimension, leading to improved compression and sample efficiency when using manifold learning layers (Tiwari et al., 28 Jul 2025).

These architectures can be combined with geometric loss terms, Riemannian optimization, or manifold-specific regularizers to maximize structural generalization, enforce invariance, and yield interpretable latent dynamics.

6. Geometric Structures, Optimization, and Interpretability

Preserving geometric structure imparts several favorable properties:

• Symmetry and invariance: Manifold operations, such as congruence actions on SPD, guarantee that model predictions are invariant to isometries of the underlying physical or system domain (Conti et al., 19 Feb 2025, Dan et al., 20 Jan 2026).
• Stable and globally valid system identification: Manifold-aware parameterizations avoid spurious folding or singularities of Euclidean embeddings, yielding robust identification of system matrices (Conti et al., 19 Feb 2025).
• Optimization in non-Euclidean spaces: Losses and gradients must be computed with respect to the appropriate Riemannian metric. For SPD outputs, the affine-invariant Riemannian gradient is $$\nabla_R\ell = P\,\mathrm{sym}(P^{-1}\nabla_E\ell\,P^{-1})\,P$$ (Conti et al., 19 Feb 2025).
• Explicitly interpretable transitions and weights: When constructed from geometric/Hamiltonian/Lie group principles, every parameter (rotation generator, translation, exponential map) acquires intrinsic meaning tied to the geometry (Assandje et al., 30 Sep 2025).

A plausible implication is that these properties support superior generalization, robustness under covariate shift, and interpretability compared to unconstrained, Euclidean, model-free neural networks.

7. Future Directions and Limitations

Geometric state-space neural networks continue to evolve in several directions:

• Generalization to arbitrary manifolds and graphs: Techniques such as geodesic kernel operators or graph-based SSMs aim to extend geometric SSMs to irregular meshes and complex topologies (Koren et al., 31 Jul 2025).
• Nonlinear and stochastic manifold dynamics: Beyond deterministic flows, stochastic processes and nonlinear interactions on manifolds remain challenging and active topics.
• Hardware and scalability: Efficient implementation of manifold-valued recurrences, e.g., via scan-optimized kernels for sparse geometric data, is necessary to scale to large datasets and high dimensions (Schöne et al., 2024).
• Explicit geometric bias in learning: Recent results confirm that hard-injecting geometric structure (e.g., true step sizes in SSMs) yields measurable improvements over purely learned structures for point clouds, events, and audio (Schöne et al., 2024).

Critical analysis points to resolving open challenges in handling extreme nonlinearities, optimizing on highly irregular manifolds, and efficiently encoding domain knowledge through geometric priors (Koren et al., 31 Jul 2025, Dan et al., 20 Jan 2026).

---

References:

• (Goto, 2019) Hessian-information geometric formulation of a class of deterministic neural network models
• (Halder et al., 2019) Hopfield Neural Network Flow: A Geometric Viewpoint
• (Conti et al., 19 Feb 2025) Geometric Principles for Machine Learning of Dynamical Systems
• (Hauser et al., 2018) State Space Representations of Deep Neural Networks
• (Schöne et al., 2024) STREAM: A Universal State-Space Model for Sparse Geometric Data
• (Dan et al., 20 Jan 2026) GeoDynamics: A Geometric State-Space Neural Network for Understanding Brain Dynamics on Riemannian Manifolds
• (Assandje et al., 30 Sep 2025) A Hamiltonian driven Geometric Construction of Neural Networks on the Lognormal Statistical Manifold
• (Tomal et al., 16 Dec 2025) Geometric Latent Space Tomography with Metric-Preserving Autoencoders
• (Koren et al., 31 Jul 2025) Merging Memory and Space: A Spatiotemporal State Space Neural Operator
• (Tiwari et al., 28 Jul 2025) Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces