Generalised Synchronisations, Embeddings, and Approximations for Continuous Time Reservoir Computers

Abstract: We establish conditions under which a continuous time reservoir computer, such as a leaky integrator echo state network, admits a generalised synchronisation $f$ between between the source dynamics and reservoir dynamics. We show that multiple generalised synchronisations can exist simultaneously, and connect this to the multi-Echo-State-Property (multi-ESP). In the special case of a linear reservoir computer, we derive a closed form expression for the generalised synchronisation $f$. Furthermore, we establish conditions under which $f$ is of class $C^1$, and conditions under which $f$ is a topological embedding on the fixed points of the source system. This embedding result is closely related to Takens' embedding Theorem. We also prove that the embedding of fixed points occurs almost surely for randomly generated linear reservoir systems. With an embedding achieved, we discuss how the universal approximation theorem makes it possible to forecast the future dynamics of the source system and replicate its topological properties. We illustrate the theory by embedding a fixed point of the Lorenz-63 system into the reservoir space using numerical methods. Finally, we show that if the observations are perturbed by white noise, the GS is preserved up to a perturbation by an Ornstein-Uhlenbeck process.