Representational Drift in Intermediate Layers

• Representational drift in intermediate layers is the gradual, sustained change in neural population codes despite stable outputs in both biological and artificial systems.
• It arises from mechanisms like synaptic turnover, learning noise, and plasticity, leading to rotational shifts in intermediate representations over time.
• Quantification methods such as SVCCA, CKA, and principal angle analysis provide insights into drift dynamics, impacting continual learning and model interpretability.

Representational drift in intermediate layers refers to the gradual, often non-reverting, evolution of a neural system’s population code at intermediate computational stages—whether in biological or artificial networks—observed even under stable behavioral or output performance. Although the term originates from neuroscience, where longitudinal recordings show pronounced changes in the responses of mid-cortical regions such as hippocampus or association cortex, the phenomenon is equally robust in artificial neural networks: intermediate-layer activations exhibit slow but measurable changes under continued online learning, continual fine-tuning, architectural redundancy, spontaneous synaptic fluctuations, or background noise, often without impacting task-level accuracy. This drift is shaped by multiple interacting mechanisms—plasticity, synaptic turnover, learning noise, and intrinsic critical dynamics—and can have profound implications for the stability, transferability, and interpretability of deep representational systems.

1. Formal Definitions and Quantification Across Frameworks

Representational drift in intermediate layers is rigorously quantified as the cumulative, often non-reverting, change in a layer’s activity vector under fixed input and preserved output performance. Formally, for a map $h_\ell(x; t)$ from input $x$ to the $\ell$-th layer at time $t$, the drift over interval $[t_0, t_1]$ is

$$\Delta h_\ell := \mathbb{E}_{x \sim D} \|h_\ell(x;t_1) - h_\ell(x;t_0)\|_2,$$

where $D$ is a fixed dataset and task error remains constant, $\text{Err}_D(t_1)=\text{Err}_D(t_0)$ (Veldt et al., 26 Dec 2025).

To analyze drift geometry, one compares subspaces via principal angles, $\theta_{\ell,i}$, defined by the singular values of $U_0^\top U_1$ for orthonormal bases $U_0, U_1$ of the two timepoints:

$$\cos \theta_{\ell,i} = \sigma_i, \qquad i=1, \ldots, k.$$

Alternatively, representational similarity matrices (RSMs), $$RSM_\ell(t)_{ij} = h_\ell(x_i; t)^\top h_\ell(x_j; t)$$, provide a drift summary via correlation $\rho_\ell(t_0, t_1)$ between vectorized RSMs (Veldt et al., 26 Dec 2025, Klabunde et al., 2023).

Common metrics in the ANN literature further include:

• SVCCA, PWCCA: CCA-based similarity between representation subspaces (Klabunde et al., 2023).
• Linear CKA: HSIC-normalized kernel alignment, invariant to orthogonal transforms (Klabunde et al., 2023).
• Information imbalance $\Delta$: average rank-change of neighborhood structure between representations (Mahaut et al., 29 Jan 2026).
• Entropy, curvature, augmentation-invariance: quantifying representational diversity and stability in LLMs (Skean et al., 2024).

2. Experimental and Theoretical Evidence for Drift

Biologically, multi-session recording in mouse V1, piriform cortex, hippocampus, and other association areas finds that intermediate-layer population codes display low per-neuron overlap (e.g., 50% across V1 sessions), low cross-session vector correlation (e.g., $\sim$0.15), and steadily increasing principal angles ($\sim$20°/$\text{wk}$ in associative cortex) while maintaining stable outputs (Yang et al., 15 Sep 2025, Morales et al., 2024, Veldt et al., 26 Dec 2025).

In artificial networks, empirical studies show:

• Early layers of CNNs or Transformers rapidly stabilize (high CKA/SVCCA, $\sim$0.9) during initial training; drift then localizes to intermediate/deep layers, which continue to evolve after accuracy saturates (Klabunde et al., 2023, Kapoor et al., 26 Feb 2025, Mahaut et al., 29 Jan 2026).
• Layer-wise alignment (Procrustes/CKA) between independently seeded networks converges sharply in the first epoch for intermediate layers, then plateaus to within $1\%$-level drift per epoch (Kapoor et al., 26 Feb 2025).
• In continual fine-tuning setups, intermediate layers (IRS) undergo systematic shifts upon arrival of new tasks, directly disrupting test-time normalization in architectures with BN (Jie et al., 2022).
• Task-irrelevant input variance and dimension increase drift rate—quantitatively, $D \propto \eta^3 \sigma_{irrel}^2 d_{irrel}$ in several canonical architectures (Pashakhanloo, 24 Oct 2025, Pashakhanloo et al., 2023).

3. Mechanistic Origins: Plasticity, Synaptic Turnover, and Learning Noise

The mechanisms generating drift in intermediate layers can be stratified as follows:

• Spontaneous Synaptic Fluctuations: Biological net synapses (e.g., in piriform cortex) undergo slow, log-normally distributed multiplicative noise (mean-reverting SDE), leading to continuous drift of population vectors despite stable mean statistics (Morales et al., 2024). In networks with homeostatic plasticity, such as E–I models with iSTDP, synaptic weights fluctuate around an attractor, with criticality (SOC) minimizing drift (Yang et al., 15 Sep 2025).
• Learning-Induced Plasticity: Upon learning new tasks, parameter updates along task-orthogonal flat directions or on degenerate loss manifolds produce drift in hidden-layer representations, manifesting as rotational diffusion in overcomplete layers, but rarely altering output (Veldt et al., 26 Dec 2025, Pashakhanloo, 24 Oct 2025, Pashakhanloo et al., 2023).
• Learning Noise from Task-Irrelevant Inputs: Intrinsic SGD noise from non-task (irrelevant) subspaces causes rotational drift in the intermediate-layer subspace. Analytical expressions show scaling of drift rate with variance and dimension of the ignored subspace, and purely rotational geometry, contrasting with isotropic Gaussian synaptic noise (Pashakhanloo, 24 Oct 2025, Pashakhanloo et al., 2023).
• Homeostatic Turnover: Synaptic weights exhibit natural turnover toward a baseline, adding a low-frequency, non-task-driven drift component (Veldt et al., 26 Dec 2025).

4. Architectural, Objective, and Training Regime Dependence

The manifestation and quantitative strength of drift depend critically on network architecture and learning regime:

• CNNs vs. Transformers: CNNs exhibit erratic, block-wise jumps in intermediate-layer drift (standard deviation of adjacent-layer drift $0.1$); transformers show smoother, distributed drift (standard deviation $0.02$–$0.04$), reflecting structural differences in feature recomposition (Mahaut et al., 29 Jan 2026).
• Universal Mid-Level Geometry: Intermediate layers in diverse models (DINOv2, ConvNeXt, ViT) align closely in representation structure, especially under shared input statistics—so-called “canonical mid-level basis”—in contrast to highly variable final layers (Kapoor et al., 26 Feb 2025).
• Objective Dependence: Classification-trained networks discard low-level cues in final layers, while self-supervised/generative models (DINOv2, iGPT) preserve both low-level and semantic information, resulting in distinct late-layer drift patters (Mahaut et al., 29 Jan 2026).
• Impact of Task Distribution and Frequency: Frequent or recently presented stimuli induce lower drift rates in their representations, both in biological and artificial circuits, indicating drift is anchored by online learning pressure toward familiar signals (Pashakhanloo et al., 2023, Morales et al., 2024).
• Batch Normalization Coupling: In CNNs with BN layers, intermediate-layer drift (IRS) misaligns true vs. running-mean statistics, directly degrading accuracy on old tasks after continual fine-tuning, unless compensated by correction schemes (e.g., Xconv BN) (Jie et al., 2022).

5. Drift Geometry and Low-Dimensional Manifold Structure

Drift in intermediate layers frequently manifests as rotations of the learned representation subspace rather than global translation or loss of function:

• Rotational Drift: Analytical treatment under Oja’s rule, similarity matching, linear autoencoders, and two-layer supervised models shows all possess a high-dimensional tangent space of rotational degrees of freedom, along which drift accumulates at rates set by learning rate, irrelevant-subspace variance, and input dimension (Pashakhanloo, 24 Oct 2025, Pashakhanloo et al., 2023).
• Manifold Stability: Critical regimes (e.g., self-organized criticality in E–I networks) restrict synaptic and representational drift, preserving the low-dimensional manifold geometry of population codes even as individual neuron identities drift. Cross-session decoding accuracy and angle preservation quantify this phenomenon (Yang et al., 15 Sep 2025).
• Partial Universality: Layer-matching studies indicate that intermediate layers form a partially universal representational scaffold, tightly aligned under distribution shift, in both artificial and biological systems (Kapoor et al., 26 Feb 2025).

6. Functional and Algorithmic Implications

Although traditionally considered a liability, representational drift in intermediate layers does not necessarily degrade functional performance and may reflect an intrinsic property of distributed learning:

• Continual Learning: Drift arises unless CL algorithms enforce parameter freezing. Orthogonal-updates, replay, and functional-regularization regimes allow controlled drift in intermediate layers without task performance loss, quantifiable as principal angle or RSM correlation increases over time (Veldt et al., 26 Dec 2025).
• Plasticity–Stability Tradeoff: Tuning parameter regularization, replay, and synaptic turnover terms can interpolate from stable (minimal drift) to plastic (rapidly drifting) regimes; optimal tradeoffs induce moderate drift and best retention/adaptability (Veldt et al., 26 Dec 2025).
• Practical Diagnoses and Control: For LLMs, layerwise entropy, curvature, and invariance profiling localize maximal drift to intermediate layers—offering strategies for optimal embedding extraction and architectural tuning (Skean et al., 2024). In CNNs, correction of BN statistics or hierarchical fine-tuning can minimize the destructive impacts of IRS (Jie et al., 2022).

7. Open Challenges and Future Directions

Open questions highlighted in the literature include:

• Interpretability: Relationship between drift magnitude (e.g., CKA, Procrustes) and functional change or task performance is nontrivial and context-dependent (Klabunde et al., 2023).
• Input/Task Dependence: How best to select stimuli for drift quantification and how input statistics structure the geometry and rate of drift remain active areas (Kapoor et al., 26 Feb 2025, Pashakhanloo, 24 Oct 2025).
• Scaling and Computation: For very deep/wide models, computational cost of standard drift metrics (SVCCA, PWCCA) demands more efficient alternatives (Klabunde et al., 2023).
• Theoretical Underpinning: Recent evidence challenges strict task-constrained convergence, advocating for an architecture- and data-imposed inductive bias as the prime driver of ‘canonical’ mid-level representations (Kapoor et al., 26 Feb 2025).
• Extension to Hierarchical and Structured Inputs: Elucidating how drift evolves with hierarchical, hyperbolic, or behaviorally complex tasks, and across interconnected brain regions, is a frontier for both systems neuroscience and foundation model research (Morales et al., 2024).

Table: Drift Metrics, Mechanisms, and Contexts

Drift Metric / Context	Main Mechanism	Key Quantitative Dependencies
SVCCA, PWCCA, CKA, RSM	Training noise, plasticity	Drift rate $\uparrow$ with layer depth, task-irrelevant variance, learning rate
Principal angles, Procrustes	Homeostatic turnover, input statistics	Drift $O(\text{\# tasks}, \eta)$; saturates post early epochs
IRS in CNNs w/ BN	Convolutional weight updates, BN stats	IRS induces catastrophic forgetting unless corrected
Rotational diffusion (SGD, Oja)	Learning noise, initialization	Drift purely rotational, $D \propto \eta^3 d_{irrel}$
SOC in E–I networks (bio)	Critical synaptic dynamics	SOC restricts drift, preserves geometry
Curvature, entropy (LLMs)	Internal compression, training	Bimodality in Transformer mid-layers, drift peaks at mid-depth

Empirical and theoretical studies across neuroscience and machine learning thus converge on the conclusion that representational drift in intermediate layers is not only prevalent but is governed by a principled interplay of architectural constraints, learning-induced rotations, criticality, and plasticity-stability tradeoffs. Understanding and leveraging this drift—especially in shaping generalizable low-dimensional manifold structure—remains a central problem for robust representation learning in both natural and artificial systems (Yang et al., 15 Sep 2025, Morales et al., 2024, Klabunde et al., 2023, Pashakhanloo, 24 Oct 2025, Veldt et al., 26 Dec 2025, Pashakhanloo et al., 2023, Kapoor et al., 26 Feb 2025, Jie et al., 2022, Mahaut et al., 29 Jan 2026, Skean et al., 2024).