Neuron Attribution Strategies

• Neuron Attribution Strategies are mathematically grounded methods that assign explicit importance scores to individual neurons using techniques like integrated gradients, log-probability shifts, and semantic activations.
• They enable practical applications such as model pruning, adversarial robustness, knowledge editing, and structural interpretability by guiding targeted modifications within deep networks.
• Efficient computation methods, including Riemann approximations, sparse masking, and concept vector pooling, reduce overhead while maintaining high attribution fidelity.

Neuron attribution strategies refer to a family of principled, mathematically-grounded methods for quantifying the contribution of individual neurons or channels in deep neural networks (DNNs) to model outputs. These strategies go beyond classic saliency analysis by assigning explicit importance scores to intermediate- and output-layer activations, often leveraging path-integrated gradients, log-probability shifts, causal interventions, or semantic associations. Attribution scores are central in interpretability, pruning, adversarial robustness, knowledge editing, and mechanistic analyses of neural computation. Within contemporary research, neuron attribution plays a critical role in feature-level attacks (Zhang et al., 2022), localized knowledge interventions (Yang et al., 9 Oct 2025), time-series identifiability (Schneider et al., 17 Feb 2025), task-level control (Li et al., 8 Jan 2026), and model compression (Ding et al., 3 Mar 2025, Yvinec et al., 2022).

1. Mathematical Foundations of Neuron Attribution

The formal core of neuron attribution methods is the construction of scalar scores quantifying each neuron's influence on the model's output, prediction, or loss. The most prevalent mechanism is path-integrated gradients (Sundararajan et al. 2017), applied to hidden-layer activations or network weights:

• Integrated Gradients (IG): For a scalar output $F(x)$ and neuron activation $y_j$, the IG attribution is:

$$\mathrm{IG}_{y_j}(x) = (y_j(x)-y_j(x'))\int_{0}^1 \frac{\partial F(x'+\alpha(x-x'))}{\partial y_j} d\alpha$$

ensuring completeness (sum of attributions equals output difference) (Zhang et al., 2022, Juneja et al., 2022).

• Static log-probability shifts: Certain transformer analyses compute per-neuron scores by the change in output log-probability after adding a subvalue to the residual stream:

$\mathcal{I}(v) = \log p(w|x+v)-\log p(w|x)$

directly measuring causal impact on token prediction (Yu et al., 2023, Yang et al., 9 Oct 2025).

• Semantic activation-based attribution: Some strategies use average neuron activations over tokens for specific words or classes to associate neurons with human-interpretable concepts (Ding et al., 3 Mar 2025).
• Range-based attribution: Modern approaches such as NeuronLens argue that individual neurons are polysemantic and their activations for different concepts form non-overlapping Gaussian-like ranges, thus proposing attributions over activation intervals rather than whole neurons (Haider et al., 4 Feb 2025).

2. Algorithmic Implementation: Efficient Computation and Approximations

Direct computation of path-integrated neuron attributions is usually computationally prohibitive. Key advances include:

• Riemann Approximation: For both input features and internal neurons, the integral in IG is efficiently estimated via a finite sum over interpolated inputs or weights (Zhang et al., 2022, Juneja et al., 2022, Kavuri et al., 21 Aug 2025).
• Zero-covariance factorization: In feature-level adversarial attacks (NAA), factors in the chain rule are separated under the zero-covariance assumption, reducing the computation from $O(n\,HWC)$ to $O(n)$ per example (Zhang et al., 2022).
• Non-linear adversarial path integration: DANAA employs an adversarially-steered trajectory (rather than a straight path) for collecting attribution gradients, yielding more transferability in black-box attacks (Jin et al., 2023).
• Sparse and dynamic masking: Editing frameworks achieve parameter-efficient updates by constructing entropy-guided neuron masks based on the distribution of attribution scores over prompt sets (Liu et al., 25 Oct 2025).
• Concept vector construction: NEAT collapses large sets of examples into pooled concept vectors, allowing $O(n)$ inference passes to localize concept neurons—orders of magnitude cheaper than per-example ablations (Kavuri et al., 21 Aug 2025).

3. Structural and Functional Interpretability

Neuron attribution enables the structural interpretability of DNNs and LLMs at multiple resolutions:

• Query vs. value neurons: Contemporary transformer analyses distinguish “value neurons” (direct contributors to final prediction) from “query neurons” (upstream activators) via importance metrics and subkey-projection analysis (Yang et al., 9 Oct 2025, Yu et al., 2023).
• Layer-resolved mapping: Empirical studies identify a division of labor—mid-layers encode relational and factual knowledge, final layers concentrate answer refinement, while low-level syntax is dispersed over initial layers (Juneja et al., 2022).
• Good and bad neurons: Task-control frameworks such as NeuronLLM define “good” (facilitative) and “bad” (inhibitive) neurons at the task level, using contrastive metrics and augmentation (AQUA) to differentiate true contributors from spurious co-activators (Li et al., 8 Jan 2026).
• Range-based polysemantic encoding: Attribution ranges per concept reduce collateral impact when intervening on polysemantic neurons, which otherwise encode multiple competing concepts at different activation levels (Haider et al., 4 Feb 2025).

4. Attribution-Guided Model Modification: Attacks, Pruning, Editing, Fusion

Neuron attribution directly informs practical interventions in models:

• Feature-level adversarial attacks: Attribution-weighted perturbations (NAA, DANAA) yield more transferable adversarial samples compared to activation-only heuristics, outperforming benchmarks by up to 10% (Zhang et al., 2022, Jin et al., 2023).
• Model pruning: Integrated-gradient based neuron relevance (SInGE) achieves superior accuracy vs. sparsity trade-offs for both structured and unstructured channel/weight removal (Yvinec et al., 2022). NSA exposes calibration-set sensitivity and explains why sentiment channels are disproportionately damaged (Ding et al., 3 Mar 2025).
• Knowledge editing: Attribution-controlled strategies (AcE) enable multi-hop factual recalls in LLMs by joint editing of query-value pathways, dramatically increasing multi-hop answer rates over baselines (Yang et al., 9 Oct 2025). NMKE introduces sparse entropy-guided masks for fine-grained edit isolation (Liu et al., 25 Oct 2025).
• Model fusion: Neuron-centric fusion objectives weight neurons by their IG or conductance scores, resulting in improved zero-shot learning and non-IID integration across diverse architectures (Luenam et al., 18 Jun 2025).

5. Evaluation, Faithfulness, and Theoretical Guarantees

Rigorous evaluation of attribution involves faithfulness and identifiability:

• Sufficiency and comprehensiveness interventions: Faithfulness tests quantify whether the most critical neurons alone reproduce predictions and whether their removal flips decisions, demonstrating that NA-identified neurons are more decision-critical than IA-identified ones (Yu et al., 2024).
• Identifiability guarantees: In time-series settings, regularized contrastive learning (xCEBRA) paired with the Inverted Neuron Gradient reads out the true underlying Jacobian connectivity up to a block-diagonal indeterminacy, with empirical recovery rates up to 98% auROC (Schneider et al., 17 Feb 2025).
• Behavioral alignment: MAPS converts attribution maps into explanation-masked images, simultaneously validating attribution methods against human and primate object recognition with minimal experimental overhead. Among methods, smoothed gradient-based saliency scores (e.g. Noise Tunnel Saliency) exhibit highest behavioral and neural alignment (Muzellec et al., 14 Oct 2025).

6. Extensions, Limitations, and Best Practices

Several limitations and ongoing refinements are evident:

• Polysemanticity remains a fundamental obstacle; range-based interventions partially mitigate interference but Gaussian ranges may overlap (Haider et al., 4 Feb 2025).
• Computational cost scales with model width and prompt diversity, though concept pooling, clustering, and efficient factorization are effective optimizations (Kavuri et al., 21 Aug 2025, Zhang et al., 2022).
• Layer selection and prompt diversity are critical; for knowledge editing and attribution, middle-to-high FFN layers and multiple syntactically diverse prompts yield the most interpretable and transferable neuron sets (Juneja et al., 2022, Yang et al., 9 Oct 2025, Liu et al., 25 Oct 2025).
• Contrastive augmentation combats spurious correctness; joint augmentation and scoring ensure only consistently facilitative/inhibitive neurons are uncovered for robust control (Li et al., 8 Jan 2026).
• Synergistic combination of instance and neuron attributions; cross-modal wrappers such as NA-Instances and IA-Neurons facilitate a holistic understanding of parametric knowledge and dataset biases (Yu et al., 2024).

In summary, neuron attribution strategies encompass a diverse suite of mathematically rigorous techniques for decomposing model decisions and guiding interventions across a spectrum of architectures, tasks, and domains. Their methodological evolution continues to refine both the interpretability and controllability of neural networks, and recent advances ensure not only empirical superiority but also theoretical identifiability and faithfulness.