Dendritic Localized Learning: Toward Biologically Plausible Algorithm

Abstract: Backpropagation is the foundational algorithm for training neural networks and a key driver of deep learning's success. However, its biological plausibility has been challenged due to three primary limitations: weight symmetry, reliance on global error signals, and the dual-phase nature of training, as highlighted by the existing literature. Although various alternative learning approaches have been proposed to address these issues, most either fail to satisfy all three criteria simultaneously or yield suboptimal results. Inspired by the dynamics and plasticity of pyramidal neurons, we propose Dendritic Localized Learning (DLL), a novel learning algorithm designed to overcome these challenges. Extensive empirical experiments demonstrate that DLL satisfies all three criteria of biological plausibility while achieving state-of-the-art performance among algorithms that meet these requirements. Furthermore, DLL exhibits strong generalization across a range of architectures, including MLPs, CNNs, and RNNs. These results, benchmarked against existing biologically plausible learning algorithms, offer valuable empirical insights for future research. We hope this study can inspire the development of new biologically plausible algorithms for training multilayer networks and advancing progress in both neuroscience and machine learning. Our code is available at https://github.com/Lvchangze/Dendritic-Localized-Learning.

• The paper introduces DLL, addressing backpropagation's limitations by using separate trainable backward weights for asymmetry.
• It employs local error computation and simultaneous training phases to more closely mirror biological neural processing.
• Empirical results demonstrate competitive performance on datasets like MNIST and CIFAR-10 and effective application to sequential RNN tasks.

Dendritic Localized Learning: Toward Biologically Plausible Algorithms

The paper "Dendritic Localized Learning: Toward Biologically Plausible Algorithms" addresses the fundamental challenge of developing biologically plausible learning mechanisms for artificial neural networks. The work critically examines the limitations of backpropagation—namely, its lack of biological plausibility due to the requirement for symmetrical weight updates, global error signals, and a dual-phase training process. To bridge the gap between biological neural processes and artificial models, the authors propose Dendritic Localized Learning (DLL), an algorithm inspired by the dynamic and plastic nature of pyramidal neurons.

Overview and Methodology

DLL is a novel learning algorithm designed to overcome the aforementioned limitations of backpropagation by meeting three key criteria for biological plausibility:

1. Asymmetry of Weights: Unlike traditional backpropagation, which necessitates symmetric forward and backward weight updates, DLL employs separate trainable backward weights, addressing the asymmetrical nature of real synaptic connections.
2. Local Error Representation: DLL computes errors locally within each neuron, eliminating the need for a global error signal. This approach is aligned with the localized processing observed in biological systems.
3. Non-Two-Stage Training: DLL allows inference and training phases to occur simultaneously, contrasting with the distinct forward and backward phases required in conventional training methods.

Experimental Validation

The authors conduct extensive empirical evaluations to benchmark DLL against existing biologically plausible algorithms across a variety of network architectures—MLPs, CNNs, and RNNs—and real-world datasets, including MNIST, FashionMNIST, SVHN, and CIFAR-10. The findings reveal that DLL achieves state-of-the-art performance among algorithms satisfying biological plausibility criteria, demonstrating strong generalization capabilities across diverse architectures and tasks.

Models trained with DLL successfully converge, yielding results competitive with backpropagation, particularly in simple datasets. On complex datasets like CIFAR-10, DLL demonstrates robustness by achieving comparable performance without relying on biologically implausible mechanisms. Furthermore, the adaptability of DLL is highlighted through its application to RNNs for sequential tasks, such as text-character prediction and time-series forecasting.

Implications and Future Directions

The implications of DLL are significant, contributing to the development of biologically plausible machine learning algorithms that maintain high performance. By aligning more closely with the neural architecture and functionality found in biological systems, DLL not only advances the understanding of synaptic learning processes but also presents practical benefits for AI applications requiring energy-efficient and scalable models.

Future research avenues might include extending DLL to spiking neural networks (SNNs), exploring hybrid architectures that integrate DLL with traditional learning methods, and addressing challenges posed by the algorithm's reliance on unsigned error signals. Additionally, there is potential for applying DLL to complex problem domains such as robotics and neuroscience, validating its practical impact.

In conclusion, this study makes substantial strides in the intersection of neuroscience and artificial intelligence, providing a framework for integrating biologically inspired learning mechanisms into neural network models. By addressing the core limitations of traditional learning algorithms, DLL sets a foundation for future work in the development of efficient, scalable, and naturally inspired AI systems.