Gradient-Based Corrective Penalty

• Gradient-Based Corrective Penalty is a method that uses surrogate model confidence volatility to assess class and sample difficulty for adaptive buffer allocation in continual learning.
• It replaces static uniform quotas with empirical, data-driven allocations that prioritize high-vulnerability classes and samples, effectively mitigating catastrophic forgetting.
• Empirical evaluations on benchmarks like Split CIFAR100 and Mini-ImageNet demonstrate improved end accuracy and reduced forgetting compared to traditional rehearsal methods.

A gradient-based corrective penalty, as formalized in the Class-Adaptive Sampling Policy (CASP) for continual learning, denotes a dynamic mechanism for allocating memory resources within buffer-based continual learning protocols. The CASP framework leverages surrogate model-derived gradient information—specifically, confidence fluctuations (vulnerability)—to measure both class-level and sample-level learning difficulty. These measurements subsequently inform adaptive buffer partitions, prioritizing storage for those classes and samples that are most susceptible to forgetting. The approach replaces static uniform quotas with empirical, data-driven allocations and sample selection policies, thereby improving memory utilization and reducing catastrophic forgetting across diverse continual learning benchmarks (Rezaei et al., 2023).

1. Partitioned Buffer Sampling in Continual Learning

In buffer-based continual learning, a fixed-size memory buffer $\mathbf{B}$ stores exemplars from previously encountered tasks or classes. Partitioned buffer sampling organizes $\mathbf{B}$ into disjoint partitions or quotas per class (or task), with sampling performed independently within each partition during rehearsal. Typically, if $\mathbf{B}$ can accommodate $M$ exemplars and there are $K$ classes, the standard fixed-partition strategy allocates $M/K$ slots per class, followed by random or uniform sampling within each class partition.

Fixed partitioning presents key limitations:

• It ignores the variable "forgettability" across classes; difficult classes are more likely to be forgotten but may not be sufficiently represented.
• Class difficulty and its contribution to learning can change during non-stationary data streams, so static splits lead to suboptimal buffer use and exacerbate catastrophic forgetting.
• Uniform partitioning can cause overrepresentation of easy or outlier classes, undercutting buffer efficiency.

2. Gradient-Based Corrective Penalty: Formulation and CASP Framework

CASP introduces a gradient-based corrective penalty by using vulnerability measures based on surrogate model confidence trajectories:

Class “Difficulty” (Vulnerability) Measure:

For each class $\mathcal{C}_j$ in new task $\mathcal{D}_t$, a lightweight surrogate model $\hat f_{\hat \theta}$ is trained for $E$ epochs. At each epoch $e$, the average SoftMax confidence for $\mathcal{C}_j$ is calculated:

$$\Gamma(\mathcal{C}_j, e) = \mathbb{E}_{X\sim\mathcal{C}_j}[P_{\hat\theta_e}(Y=j|X)]$$

The mean confidence $\bar\Gamma(\mathcal{C}_j)$ and its standard deviation $\mathcal{V}(\mathcal{C}_j)$ (the vulnerability) are computed as:

$$\bar\Gamma(\mathcal{C}_j) = \frac{1}{E}\sum_{e=1}^{E} \Gamma(\mathcal{C}_j, e)$$

$$\mathcal{V}(\mathcal{C}_j) = \sqrt{\frac{1}{E}\sum_{e=1}^{E}\left(\Gamma(\mathcal{C}_j, e) - \bar\Gamma(\mathcal{C}_j)\right)^2}$$

A higher $\mathcal{V}(\mathcal{C}_j)$ denotes greater difficulty and forgettability.

Sample “Contribution” (Difficulty) Measure:

For each sample $(X_i, y_i) \in \mathcal{D}_t$:

$$\bar\Gamma(X_i) = \frac{1}{E}\sum_{e=1}^E P_{\hat{\theta}_e}(Y=y_i|X_i)$$

$$\mathcal{V}(X_i) = \sqrt{\frac{1}{E}\sum_{e=1}^E\left(P_{\hat{\theta}_e}(Y=y_i|X_i) - \bar\Gamma(X_i)\right)^2}$$

Samples with high $\mathcal{V}(X_i)$ are concentrated near decision boundaries and are prioritized for rehearsal.

Buffer Allocation:

Let total buffer quota for task $t$ be $\mathcal{M}_t$. Per-class slots are assigned as:

$$\mathcal{S}_j = \frac{\mathcal{V}(\mathcal{C}_j)}{\sum_{k=1}^K \mathcal{V}(\mathcal{C}_k)}\;\mathcal{M}_t$$

Within each class, the top $\mathcal{S}_j$ samples, as ranked by $\mathcal{V}(X_i)$, are reserved.

3. Algorithmic Workflow

CASP is applied at the end of each task in continual learning. The key steps, as implemented in the cited work, are:

1. Train a surrogate $\hat{f}$ on $\mathcal{D}_t$ for $E$ epochs, saving $P_{\hat{\theta}_e}(Y|X)$ for each $X$.
2. For every class $j$, compute $\Gamma(\mathcal{C}_j,e)$, $\bar\Gamma(\mathcal{C}_j)$, and $\mathcal{V}(\mathcal{C}_j)$.
3. Set per-class quotas $\mathcal{S}_j$ proportional to $\mathcal{V}(\mathcal{C}_j)$.
4. For each sample $(X_i, y_i)$ in $\mathcal{D}_t$, calculate $\mathcal{V}(X_i)$.
5. Within each class, select the top $\mathcal{S}_j$ samples by $\mathcal{V}(X_i)$ for memory.
6. Update buffer: remove data from the old task $t$, combine the new selected samples with the retained buffer.

The following table summarizes CASP’s buffer quota logic for comparison:

Partition Strategy	Quota Rule for Class $\mathcal{C}_j$	Within-Class Sampling
Uniform Partition	$\mathcal{S}_j = \mathcal{M}_t / K$	Random/Uniform
CASP	$\mathcal{S}_j \propto \mathcal{V}(\mathcal{C}_j)$	Rank by $\mathcal{V}(X_i)$

Editor's term: "Surrogate confidence volatility" denotes the empirical proxy for forgetting risk in this context.

4. Impact on Memory Utilization and Forgetting

Empirical evaluation on Split CIFAR100 and Split Mini-ImageNet demonstrates the quantitative benefits of CASP. CASP concentrates buffer capacity on high-vulnerability classes, thereby preserving critical decision boundaries. This corrective allocation yields marked improvements:

• On Split CIFAR100 (buffer size 1000), uniform ER achieves 12.07% average end accuracy and 54.66% average forgetting, whereas ER+CASP yields 16.14% (+4.07 pp) accuracy and 51.96% (-2.70 pp) forgetting.
• Across buffer sizes 200–5000 on Split CIFAR100, average end accuracy increases for representative methods:
  • ER: 13.26% → 16.81%
  • MIR: 13.07% → 16.65%
  • SCR: 31.43% → 32.73%
  • DVC: 30.45% → 31.79%
  • PCR: 31.03% → 32.95%
• Forgetting is also reduced across these methods by 0.34 to 2.46 percentage points.

CASP likewise outperforms policy-driven baselines such as GSS and ASER, improving end accuracy and reducing forgetting with minimal computational overhead (Rezaei et al., 2023).

5. Generality and Integration with Continual Learning Algorithms

The CASP mechanism, predicated on confidence-based vulnerability, is compatible with a broad spectrum of buffer-based continual learning algorithms. Its principles can be extended to:

• Task-agnostic continual streams and hierarchical class structures
• Multi-modal data
• Algorithms using generative replay or explicit parameter isolation

This suggests broad applicability, as the adaptive quota policy is agnostic to the main learning backbone.

6. Future Research Directions

Advances can be pursued along several axes:

• Online Estimation of Vulnerability: Eliminating the need for a separate surrogate model by integrating vulnerability estimation into the primary learning loop.
• Theoretical Analysis: Formally characterizing the trade-offs between buffer allocation strategies and their impact on catastrophic forgetting.
• Integration with Other Stability–Plasticity Mechanisms: Incorporating generative replay or parameter isolation to further enhance continual learning robustness.
• Dynamic Buffer Sizing: Adjusting the overall buffer size $\sum_t \mathcal{M}_t$ in response to empirical stream complexity or class dynamics.

A plausible implication is that such dynamic, gradient-based corrective penalties will continue to play a central role in optimizing rehearsal buffer utilization and mitigating catastrophic forgetting in diverse continual learning settings (Rezaei et al., 2023).