Function-Aware Neuron Grouping

• Function-Aware Neuron Grouping (FANG) is a method that groups neurons based on shared functional roles rather than activation magnitude.
• It employs statistical clustering and importance attribution techniques to identify specialized neuron groups for optimized pruning, conversion, and analysis.
• Applications in ANN-SNN conversion, Transformer sparsity, and CNN bias analysis demonstrate FANG’s effectiveness in enhancing model performance and interpretability.

Function-Aware Neuron Grouping (FANG) encompasses a spectrum of principled mechanisms for grouping neurons based on shared functional characteristics rather than superficial connectivity or activation magnitude. Across neural network domains, FANG frameworks facilitate improved performance, interpretability, and resource efficiency by leveraging neuron specialization and collaborative interactions. The central concept is identifying groups of neurons that process similar semantic contexts, visual concepts, or activation nonlinearities, and using these groupings to optimize conversion, pruning, or analysis tasks.

1. Foundational Principles of Function-Aware Neuron Grouping

Function-Aware Neuron Grouping is predicated on the observation that neurons within deep networks often acquire specialized roles, processing distinct semantic, logical, or visual contexts. Rather than treating all neurons as independent units, FANG methods group neurons by the type of function they serve, such as processing factual information, logical reasoning, co-reference resolution, or distinct visual concepts. This functional taxonomy enables context-sensitive operations—pruning, conversion, and interpretability—while preserving critical model capacities.

A neuron’s “function” is operationalized as a mapping from input contexts (tokens, patches, activations) to semantic groups wherein neurons activate preferentially. Groupings are then established through statistical clustering (e.g., K-means, PCA, agglomerative algorithms) using relevant internal representations. FANG approaches generalize beyond simple structural heuristics, employing importance scores derived from Taylor sensitivity, Integrated Gradients (IG), or activation-centric objective functions (Yu et al., 28 Dec 2025, &&&1&&&).

2. Methodologies for Discovering Functional Neuron Groups

Structured Clustering

In Transformer architectures, neuron grouping commences by partitioning the input token embeddings (post-multi-head attention and residual addition) into context clusters via PCA followed by K-means. For $L$ tokens, input activations $X^\ell \in \mathbb{R}^{d \times L}$ in layer $\ell$ are dimensionality-reduced and clustered into $\{C_1^\ell, \ldots, C_K^\ell\}$.

Cluster-Neuron Assignment

Functional group membership is established by constructing a cluster–neuron score matrix using first-order Taylor sensitivity:

$$s_{k,j}^\ell = \frac{1}{|C_k|} \sum_{i \in C_k} |h_{i, j}^\ell \cdot \partial \mathcal{L} / \partial h_{i, j}^\ell|$$

where $h_{i,j}^\ell$ denotes the activation of neuron $j$ on token $i$ and $\mathcal{L}$ is the language modeling loss. The Hungarian method solves the assignment to maximize the functional association between neurons and semantic clusters, resulting in disjoint functional groups $G_k^\ell$ (Yu et al., 28 Dec 2025).

Multi-Function and Shared Neuron Detection

FANG identifies neurons that have significant contributions across multiple context types, assembling an additional “shared” group per layer by frequency analysis of top-scoring neurons in each cluster. These generalists are exempt from pruning or perturbation, safeguarding cross-functional capacities (Yu et al., 28 Dec 2025).

Importance Attribution in Vision Models

In convolutional networks, core concept neurons are identified by evaluating the perturbation in top-$k$ concept patches after neuron knockout:

$$\mathbb{S}_a = \arg\min_{S \subset \text{layer } l-1, |S| \leq \tau} |\bar{\mathcal{V}}_a^{S} \cap \mathcal{V}_a|$$

Effectively, core neurons are those whose disruption most alters the visual concept encoded by a target neuron. IG-based importance scores further rank candidate neurons for functional grouping (Cao et al., 22 Feb 2025).

3. Architectures and Dynamics in Function-Aware Grouping

Group Neurons (GNs) in ANN-SNN Conversion

In spiking networks, each Group Neuron (GN) replaces a standard Integrate-and-Fire unit with $\tau$ parallel members sharing a membrane potential but having distinct thresholds:

$$\theta_i^l = \frac{i}{\tau} \theta^l, \quad \theta_{GN}^l = \frac{\theta^l}{\tau}$$

GN neural dynamics combine parallel member spiking and lateral inhibition: \begin{align*} p^l(t) &= v^l(t-1) + W^l x^l(t) \ s_i^l(t) &= \mathrm{Heaviside}(p^l(t) - \theta_i^l) \ v^l(t) &= p^l(t) - \theta_{GN}^l \sum_{i=1}^\tau s_i^l(t) \ s_{GN}^l(t) &= \sum_{i=1}^\tau s_i^l(t) \end{align*} This piecewise mapping enables finer-grained rate approximation to the original ANN activation, mitigating conversion error and latency (Lv et al., 2024).

Adaptive Sparsity Allocation in Transformers

Block-wise sparsity is allocated in accordance with functional complexity:

$$FC_\ell = 1 - \mathbb{E}_{X, t}\left[ \frac{(X_{:, t}^\ell)^T X_{:, t}^{\ell+1}}{\|X_{:, t}^\ell\|_2 \|X_{:, t}^{\ell+1}\|_2} \right]$$

Higher functional complexity blocks receive lower sparsity during pruning, ensuring that critical transformation capacity is preserved (Yu et al., 28 Dec 2025).

Hierarchical Circuit Construction in Vision Models

Hierarchical circuits of neuron groups are constructed by recursively linking core neurons (and their semantic groups) across layers. Weighted edges represent IG-derived functional dependencies, assembling hypertrees and concept-group circuits that clarify internal logic and compositional pathways (Cao et al., 22 Feb 2025).

4. Empirical Results and Performance Impact

Conversion and Latency in SNNs

GN-based SNNs achieve near-ANN accuracy at minimal time-steps:

• CIFAR-10 (ResNet-18): At $T = 1$–4, accuracy is $96.01$–$96.57\%$ (ANN $96.48\%$).
• CIFAR-100 (ResNet-20): At $T = 2$–8, accuracy is $67.60$–$69.38\%$ (ANN $68.41\%$).
• ImageNet (ResNet-34, $\tau=6$): At $T = 2$–8, accuracy is $73.61$–$73.73\%$ (ANN $74.35\%$).

Mean-squared conversion errors decrease from $3.72\%$ (IF, $T=4$) to $0.37\%$ (GN, $\tau=4$) (Lv et al., 2024).

Post-Training Pruning in LLMs

FANG combined with FLAP or OBC achieves SOTA zero-shot accuracy and perplexity under extreme sparsity:

• At $30\%$ sparsity (LLaMA-2-7B), O-FANG raises average accuracy by $+1.95\%$ and reduces perplexity from $7.34$ to $7.23$.
• At $40\%$ sparsity, accuracy improves by $+2.49\%$, perplexity falls from $9.13$ to $8.67$.

Ablation studies confirm additive contributions: adaptive sparsity ($\sim 1.3\%$), shared neuron retention ($\sim 0.5\%$), and function-aware pruning ($0.2$–$0.6\%$) (Yu et al., 28 Dec 2025).

Interpretability and Debugging

Masking or retaining NeurFlow-generated neuron groups in vision models produces pronounced changes in top-1 accuracy and logit responses, enabling identification of causal biases (e.g., “flower petals” spurious association with “bee” class). Empirical measures verify the near-optimality and fidelity of core neuron groups, outperforming weight-magnitude heuristics and other attribution methods (Cao et al., 22 Feb 2025).

5. Application Domains and Use Cases

Structured Pruning for Efficiency

FANG is deployed for post-training structured pruning in LLMs, identifying and preserving functionally critical neurons to minimize calibration bias and retain downstream generalization. The framework ensures that calibration set representation does not inadvertently discard context-specialized units, and dynamically adjusts resource allocation per functional complexity (Yu et al., 28 Dec 2025).

ANN-SNN Conversion for Latency Reduction

GN-based function-aware grouping in spiking networks yields high accuracy with few time-steps, achieving efficient inference without sacrificing performance. The method generalizes to more complex nonlinear activation function approximations, potentially extending FANG into architectures requiring Swish or GELU-like responses (Lv et al., 2024).

Neural Network Interpretability

NeurFlow operationalizes FANG principles for enhanced network explainability, shifting focus to neuron group hierarchies. This facilitates visual concept debugging and automatic semantic group labeling by multimodal LLMs, providing layer-by-layer explanations without manual annotation (Cao et al., 22 Feb 2025).

6. Technical and Methodological Comparisons

Domain	FANG Mechanism	Performance Benefit
Spiking NN	Piecewise GN activation	Accuracy ~ANN, fast inference
Transformers	Context-based neuron grouping, sparsity allocation	SOTA pruning, low calibration bias
Vision CNNs	Core neuron/semantic group discovery	Interpretable circuits, bias analysis

FANG frameworks diverge from traditional approaches such as random assignment, Taylor-based allocation, or weight-magnitude heuristics. Empirical ablations demonstrate performance degradation when semantic grouping or token reweighting is omitted. IG-based attribution methods deliver higher fidelity and runtime efficiency over alternatives such as Saliency, LRP, or Knockoffs (Cao et al., 22 Feb 2025).

7. Prospective Directions and Generalizations

Function-Aware Neuron Grouping is extensible to broader modeling scenarios. GN mechanisms represent a specific instantiation for piecewise linearity; FANG principles could further generalize to individuated spike dynamics or more complex nonlinearities, supporting adaptive quantization and context-sensitive neuron specialization (Lv et al., 2024). In LLM pruning, adaptive calibration and dynamic block-wise resource allocation based on functional complexity remain active fields of inquiry, promising advances in both performance and robustness (Yu et al., 28 Dec 2025).

In sum, FANG provides a principled foundation for neuron-level specialization, guiding conversion, pruning, and interpretability methodologies that preserve functional integrity within deep networks. The integration of context-sensitive clustering, importance attribution, and hierarchical circuit assembly positions FANG as instrumental in high-performance, interpretable, and resource-efficient neural modeling.