Brain-inspired interpretable reservoir computing with resonant recurrent neural networks

Abstract: Traditional artificial neural networks consist of nodes with non-oscillatory dynamics. Biological neural networks, on the other hand, consist of oscillatory components embedded in an oscillatory environment. Motivated by this feature of biological neurons, we describe a reservoir computing framework with explicit damped, oscillatory node dynamics. We express the oscillatory dynamics using two history dependent terms to connect these dynamics with existing artificial neural network approaches and apply physical and stationary constraints to reduce the number of free parameters. We then optimize and illustrate reservoir performance by classifying different brain rhythms associated with epilepsy and show that reservoir elements support classification by resonating with features of the input signals. Applying the same reservoir network to visual and auditory signal types, we show the reservoir generalizes for accurate classification with few trainable parameters. Compared to existing artificial neural network approaches, the proposed resonant reservoir network (RRN) utilizes oscillatory dynamics expressed as a straightforward extension of traditional artificial neural networks, produces interpretable features for classification, avoids computationally expensive training (e.g., backpropagation), and performs well with few parameters in different classification scenarios. We propose that RRNs may serve as efficient, biologically implemented building blocks to achieve complex goals in biological and artificial neural networks.

• The paper introduces a novel reservoir computing framework using damped, driven oscillators to create interpretable features from input signals.
• The method leverages AR(2) oscillatory dynamics to capture resonant features for high accuracy in tasks such as epilepsy detection and multi-modal classification.
• Experimental results demonstrate robust performance across datasets like MNIST and speech recognition while using significantly fewer trainable parameters.

Introduction

This paper presents a novel reservoir computing framework called Resonant Reservoir Network (RRN), characterized by nodes with intrinsic oscillatory dynamics, inspired by biological oscillations in neural environments. Unlike existing artificial neural networks with non-oscillatory dynamics, the RRN's damped oscillator-based node dynamics are specifically formulated to produce interpretable features crucial for signal classification tasks, such as epilepsy diagnosis, while circumventing traditional, computationally intensive training procedures like backpropagation. The framework seamlessly connects statistical methods and physical principles with artificial intelligence paradigms to enhance versatility in diverse classification contexts.

Oscillatory Dynamics in Reservoir Networks

The RRN leverages damped, driven oscillators at each node, defined by two history-dependent terms, facilitating intrinsic oscillations. Nodes interact based on a sparse, random connectivity matrix, with dynamics governed by spectral constraints ensuring stability and computation efficiency. The oscillatory model resonates with features of input signals, optimizing classification tasks by transforming these signals into sinusoidal resonances. Such dynamics are modeled as second-order autoregressive (AR(2)) systems, akin to physical harmonic oscillators, facilitating compartmental signal processing and improved interpretability.

Application to Classification Tasks

Initially, the RRN was optimized to classify synthetic brain rhythms, specifically targeting epileptic markers like spike ripples. With minimal preprocessing, the RRN achieved high accuracy through resonant node dynamics matching specific frequency components characteristic of pathological signals. The performance metrics demonstrate notable advantages in classification accuracy compared to standard power spectral analysis, showcasing the efficacy of oscillatory features for neural rhythm identification. Furthermore, the RRN extends its application to visual and auditory tasks, maintaining competitive performance with fewer parameters in scenarios such as handwritten digit and spoken word recognition.

Performance and Generalization

The RRN exhibits robust classification performance across diverse datasets with minimal parameter tuning, leveraging its generality and simplicity. For example, the RRN classified MNIST handwritten digits and Speech Commands Dataset spoken digits, achieving competitive accuracy despite using significantly fewer trainable parameters compared to state-of-the-art models like LSTMs or HORNs. This generality across modalities indicates that the RRN's oscillatory nodes efficiently extract essential features from various temporal and spatial data types, ensuring its potential for widespread applicability in computational tasks requiring reduced computational overhead and enhanced interpretability.

Implications and Future Directions

The development of RRNs highlights the potential of biologically motivated oscillatory dynamics in computational models, suggesting avenues for integrating biological processes in artificial systems to exploit inherent frequency-driven responses. The RRN's structural simplicity may pave the way for new neurocomputational models that emulate natural resonance phenomena found in biological systems. Future research could explore alternative network architectures, potentially integrating physical systems like nanoscale oscillators or memristors, and further align artificial intelligence models with in vivo biological networks to refine signal processing and interpretation.

Conclusion

The Resonant Reservoir Network establishes a principled, accessible framework for integrating oscillatory dynamics into artificial neural networks, offering computational efficiency alongside interpretability and generalization across classification tasks. By revisiting the fundamental rhythmic nature of biological neuronal systems, the RRN proposes an intriguing paradigm for future artificial intelligence development. As biological understanding progresses, the RRNs may serve as pivotal computational constructs bridging artificial models and biological intelligence through rhythm-inspired designs that predictably interact with temporal and spectral features of real-world signals.