Brain-like Functional Organization within Large Language Models

Abstract: The human brain has long inspired the pursuit of AI. Recently, neuroimaging studies provide compelling evidence of alignment between the computational representation of artificial neural networks (ANNs) and the neural responses of the human brain to stimuli, suggesting that ANNs may employ brain-like information processing strategies. While such alignment has been observed across sensory modalities--visual, auditory, and linguistic--much of the focus has been on the behaviors of artificial neurons (ANs) at the population level, leaving the functional organization of individual ANs that facilitates such brain-like processes largely unexplored. In this study, we bridge this gap by directly coupling sub-groups of artificial neurons with functional brain networks (FBNs), the foundational organizational structure of the human brain. Specifically, we extract representative patterns from temporal responses of ANs in LLMs, and use them as fixed regressors to construct voxel-wise encoding models to predict brain activity recorded by functional magnetic resonance imaging (fMRI). This framework links the AN sub-groups to FBNs, enabling the delineation of brain-like functional organization within LLMs. Our findings reveal that LLMs (BERT and Llama 1-3) exhibit brain-like functional architecture, with sub-groups of artificial neurons mirroring the organizational patterns of well-established FBNs. Notably, the brain-like functional organization of LLMs evolves with the increased sophistication and capability, achieving an improved balance between the diversity of computational behaviors and the consistency of functional specializations. This research represents the first exploration of brain-like functional organization within LLMs, offering novel insights to inform the development of artificial general intelligence (AGI) with human brain principles.

• The paper introduces a framework that maps temporal responses of artificial neurons in LLMs to fMRI data, revealing a brain-like functional organization.
• It employs sparse representation and voxel-wise encoding models to statistically link AN sub-groups with specific functional brain networks.
• Results indicate that advanced models, particularly Llama3, demonstrate hierarchical organization and enhanced representational capabilities akin to human neural architectures.

Brain-like Functional Organization within LLMs

Introduction

The investigation into the alignment of artificial neural networks (ANNs) with neural responses of the human brain to external stimuli forms a crucial intersection between AI and neuroscience. The paper "Brain-like Functional Organization within LLMs" explores this alignment by focusing on LLMs and their potential to mirror the functional organization of the human brain's functional brain networks (FBNs). This paper presents methodologies for linking artificial neurons (ANs) in LLMs, such as BERT and Llama models, with FBNs using functional magnetic resonance imaging (fMRI), revealing compelling insights into the brain-like architecture of these models.

Methodology

The authors propose a comprehensive framework to study LLMs by mapping AN sub-groups to FBNs.

1. Artificial Neurons in LLMs: By focusing on BERT and Llama1-3, ANs are defined in the fully connected layers of the feed-forward network within each transformer block. Their temporal responses to stimuli are aligned with fMRI data using the Narratives fMRI dataset.
2. Sparse Representation: Representative temporal response patterns of ANs are identified using a sparse representation technique. These patterns form a dictionary used to predict fMRI brain activity, establishing a voxel-wise encoding model that allows linking of AN sub-groups to brain activity.

   Figure 1: The study overview. We learn representative patterns $\mathbf{D_AN}$ from the temporal responses of ANs in LLMs and use them to reconstruct fMRI brain activity.

3. Voxel-wise Encoding Models: The encoding models showcase the importance of representative patterns in reconstructing brain activity. These models are statistically validated to associate AN responses with specific brain regions.

Results

1. Sparse Representation: The $R^2$ values indicate that temporal responses of ANs are well-represented by the learned dictionary, with distinct variations between BERT and the Llama models. The distribution of these values also suggests that Llama models show better alignment with brain activity compared to BERT.

   Figure 2: The $R^2$ values in the sparse representation of temporal responses of ANs (a), and the sparse reconstruction of fMRI activity (b) demonstrate close alignment with fMRI data.

2. Brain Map Analysis: The brain maps derived from voxel-wise encoding models reveal complex functional interactions among FBNs. Significant FBNs such as the language network, default mode network (DMN), and visual cortex show unique activation patterns across different models (Figure 3).

   Figure 4: Example brain maps show relevant FBN activation and deactivation patterns illustrating detailed functional interaction within LLMs.

   

   Figure 3: The distribution of brain maps showcases the involvement and complexity of different FBNs.

3. Evolution of Functional Organization: It is evident that brain-like functional architecture evolves with increasing sophistication in LLMs, emphasizing hierarchical organization and enhanced representational capabilities in advanced models like Llama3.

   Figure 5: FBN involvement depicted by Dice coefficients highlights active and deactivated regions within brain maps, demonstrating varied LLM functionalities.

   

   Figure 6: Temporal consistency and distribution of ANs across LLM layers in functional interactions, showing evolving patterns from BERT to Llama3.

Conclusion

This study provides a robust framework for examining the brain-like functional organization within LLMs, furthering our understanding of how these models emulate neural architectures. While more advanced LLMs achieve a better balance between computational diversity and functional specialization, future research should address limitations such as fixed dictionary sizes and broader fMRI session evaluations to improve the mapping precision between AN sub-groups and FBNs. The findings offer a pathway for developing brain-inspired AI models, potentially advancing the pursuit of artificial general intelligence (AGI) based on human brain principles.