Analytic Dynamic Imbalance Rectifier (ADIR)

• ADIR is a closed-form algorithm that rebalances class contributions using weighted ridge regression to address intra- and inter-task imbalances.
• It employs a dynamic regularization strategy based on the Gini coefficient to adaptively mitigate catastrophic forgetting.
• ADIR eliminates raw exemplar storage, enabling memory-efficient and privacy-compliant incremental learning in applications like sound source localization.

The Analytic Dynamic Imbalance Rectifier (ADIR) is a closed-form, exemplar-free algorithmic module designed to address dynamic class-imbalance and catastrophic forgetting in generalized class-incremental learning (GCIL) settings, particularly for sound source localization (SSL) and related continual learning domains. ADIR operates by analytically re-weighting class contributions and adaptively regularizing classifier updates based on evolving cross-task class distributions, eliminating the need for raw exemplar storage and enabling privacy-compliant, memory-efficient incremental adaptation (Fan et al., 26 Jan 2026, Fang et al., 2024).

1. Motivation and Context

Many real-world class-incremental and generalized class-incremental learning (GCIL) problems exhibit two pervasive forms of imbalance: intra-task (long-tailed distribution of labels within a task) and inter-task (distribution skews and overlaps across tasks). In GCIL, classes (e.g., discrete directions-of-arrival for SSL) arrive in a non-monotonic sequence, may reappear or disappear, and are often distributed unevenly across time. Uncorrected, this leads to two major issues:

• Majority bias: Classifiers become biased toward classes with abundant recent samples, neglecting minority and previously encountered classes.
• Catastrophic forgetting: The classifier's accuracy on earlier classes deteriorates as it is updated for new classes or tasks.

ADIR is developed to explicitly correct these phenomena in an analytic, computationally efficient, memory-light, and privacy-preserving manner—critical in scenarios where storing raw data for replay is infeasible or prohibited (Fan et al., 26 Jan 2026, Fang et al., 2024).

2. Theoretical Formulation

ADIR frames incremental classifier learning as a weighted ridge regression problem, in which class contributions are rebalanced inversely to their total sample counts, and the regularization coefficient is dynamically adapted to class-imbalance severity. Let $T$ be the total number of tasks, $t$ the current task, and $C$ the number of classes observed so far.

Given accumulated per-class statistics:

• $N_c$: total samples of class $c$
• $A^{(c)} = \sum_{i: y_i = c} h_i h_i^T$ (auto-correlation)
• $C^{(c)} = \sum_{i: y_i = c} h_i z_i^T$ (cross-correlation)

where $h_i$ is the feature representation (from a frozen extractor) and $z_i$ is a smoothed one-hot target, the ADIR loss for weight matrix $W \in \mathbb{R}^{d \times C}$ is: $$L(W) = \sum_{c=1}^C \pi_c \|Z_t^{(c)} - H_t^{(c)} W\|_F^2 + \gamma_t \|W\|_F^2$$ Here, $\pi_c = 1/N_c$ (per-class inverse frequency); $\gamma_t > 0$ is the adaptive regularization.

The analytic optimal solution is: $W_t^* = (A_{1:t} + \gamma_t I_d)^{-1} C_{1:t}$ with $A_{1:t} = \sum_{c=1}^C \pi_c A^{(c)}$, $C_{1:t} = \sum_{c=1}^C \pi_c C^{(c)}$ (Fan et al., 26 Jan 2026, Fang et al., 2024).

3. Dynamic Regularization and Class-Imbalance Quantification

ADIR incorporates a novel dynamic regularization strategy responsive to the instantaneous skew of the class distribution:

• Gini coefficient: For class prior $p_c = N_c / \sum_c N_c$,

$$\mathrm{Gini}_t = \frac{\sum_{i=1}^C \sum_{j=1}^C |p_i - p_j|}{2C \sum_{i=1}^C p_i}$$

• Adaptive regularization: The regularization parameter is set as

$$\gamma_t = \gamma_0 \exp[\alpha(\mathrm{Gini}_t - 0.5)]$$

where $\gamma_0$ and $\alpha$ are hyperparameters (e.g., $\gamma_0 = 100$, $\alpha = 2$) (Fan et al., 26 Jan 2026). Higher skew (Gini $> 0.5$) results in exponentially increased regularization, which suppresses overfitting to head classes.

4. Incremental Update Algorithm

ADIR operates solely via incremental maintenance of per-class auto- and cross-correlation matrices and per-class sample counts. The update algorithm, executed at each new phase or task, comprises:

1. For each new sample $(x_i, y_i, z_i)$:
   • Extract feature $h_i$
   • Update $A^{(y_i)} \gets A^{(y_i)} + h_i h_i^{\top}$
   • Update $C^{(y_i)} \gets C^{(y_i)} + h_i z_i^{\top}$
   • Increment $N_{y_i}$
2. Compute $\pi_c = 1/N_c$ for all classes with $N_c > 0$
3. Compute class proportions $p_c$, then Gini coefficient and set $\gamma_t$ adaptively
4. Form weighted global sums $A_{1:t}$, $C_{1:t}$
5. Solve for $W_t^* = (A_{1:t} + \gamma_t I)^{-1} C_{1:t}$
6. Use $W_t^*$ for classification in subsequent inference (Fan et al., 26 Jan 2026, Fang et al., 2024).

5. Memory and Computational Complexity

ADIR does not retain raw exemplars, relying exclusively on first- and second-order class statistics. The memory footprint is thus $O(C d^2 + C d C)$ for $d$-dimensional features and $C$ classes—tractable for common dimensionalities (e.g., $d = 1000$, $C = 360$ in SSL (Fan et al., 26 Jan 2026)). Each phase update requires $O(d^3)$ for a single matrix inversion and $O(C d^2 + C d C)$ for accumulations, which is comparable or superior to gradient-based approaches that require repeated backpropagation across epochs (Fang et al., 2024).

6. Empirical Validation and Comparative Analysis

Ablation experiments on the SSLR benchmark (Fan et al., 26 Jan 2026) and standard CIL datasets (Fang et al., 2024) demonstrate the effectiveness of ADIR for mitigating catastrophic forgetting and long-tailed imbalance. In SSL, core metrics include accuracy (ACC), mean absolute error (MAE), and backward transfer (BWT):

Setting	ACC (%)	MAE (°)	BWT
Baseline (no GDA, no ADIR)	72.0	—	–17.7
+ ADIR only	82.4	—	+1.4
+ GDA only	75.0	—	–15.8
Full model (+ADIR & +GDA)	89.0	5.3	+1.6

ADIR alone yields a $\sim$10% increase in accuracy and reverses catastrophic forgetting (BWT becomes positive). When combined with intra-task augmentation (GDA), state-of-the-art results are achieved under both clean and noisy conditions. Similarly, AIR achieves last-phase accuracy on CIFAR-100 above 79.7% with invariance to class order and matches or exceeds the performance of replay-based approaches with large memory.

Both studies confirm that ADIR eliminates head-class bias, enhances tail-class representation, and is robust to dynamic, nonstationary class distributions (Fan et al., 26 Jan 2026, Fang et al., 2024).

7. Connections and Extensions

ADIR/AIR generalizes to various continual learning modalities beyond SSL, including computer vision and other domains where non-stationary, imbalanced incremental learning is essential. The subclass of methods characterized by:

• Closed-form, weighted ridge regression reflection of per-class sample distributions
• Memoryless, privacy-compliant operation (via sufficient statistics)
• Task-adaptive analytic control of regularization

forms a scalable foundation for exemplar-free continual learning. A plausible implication is that further extensions could integrate per-sample weighting or adaptive feature extraction, although this is not covered in the referenced works.

ADIR stands as a rigorously validated, theoretically principled, and empirically effective approach for dynamically rectifying class imbalance and preventing catastrophic forgetting in generalized class-incremental learning (Fan et al., 26 Jan 2026, Fang et al., 2024).