BP-TTA: Balanced and Prototype-Guided Test-Time Adaptation in Dynamic Scenarios

Abstract: Test-Time Adaptation (TTA) enables models trained on a source domain to adapt online to unlabeled test data under distribution shifts. While recent TTA methods have moved beyond static settings and begun to consider continual domain shifts, they primarily address distribution drift and fail to account for class imbalance in dynamic scenarios. In real-world test-time streams, class imbalance and continual domain shifts often occur at the same time and interact with each other. In this paper, we propose a novel Balanced and Prototype-Guided Test-Time Adaptation (BP-TTA) method, which combines batch-balanced sampling with prototype-guided adaptation to handle the class imbalance and continual domain shift problems. BP-TTA constructs balanced adaptation batches by integrating current samples with high-confidence historical instances, effectively mitigating bias toward dominant classes and stabilizing online updates. Meanwhile, BP-TTA maintains evolving class prototypes during inference and leverages prototype similarity as a constraint for model adaptation, thereby improving the reliability of pseudo-labels and enhancing the stability of online updates under persistent domain shifts. Extensive experiments demonstrate that BP-TTA consistently outperforms state-of-the-art TTA methods in dynamic test-time streaming settings.

• The paper introduces BP-TTA that simultaneously tackles class imbalance and domain shift using batch-balanced sampling and prototype-guided adaptation.
• It utilizes a class-aware memory bank and momentum-based prototype updates to stabilize feature extraction and enhance robustness in streaming data.
• Experimental results on CIFAR10-C, CIFAR100-C, and ImageNet-C demonstrate significant error rate reductions compared to prior state-of-the-art TTA methods.

Balanced and Prototype-Guided Test-Time Adaptation in Dynamic Scenarios: An Expert Analysis

Introduction and Problem Definition

The "BP-TTA: Balanced and Prototype-Guided Test-Time Adaptation in Dynamic Scenarios" (2606.31420) paper addresses critical deficiencies in current Test-Time Adaptation (TTA) frameworks, focusing on the dual, often-compounded challenges of class imbalance and continual domain shift in streaming data. Under practical deployment, distributional nonstationarities arise from environmental dynamics and temporal correlations, rendering static and i.i.d. test set assumptions invalid. Prior TTA methods have predominantly optimized adaptation for domain drift but are fundamentally limited by their neglect of dynamic class imbalance, which leads to adaptation dominated by majority classes, misguidance from noisy pseudo-labels, and unstable performance.

BP-TTA directly tackles these issues through two complementary and novel mechanisms: Batch-Balanced Sampling (BBS) and Category Prototype-Guided Adaptation (CPGA). The framework integrates a class-aware memory bank to enforce batch-level class homogeneity and prototype-driven constraints to stabilize online updates under shift, thereby enhancing model reliability for streaming real-world deployment.

Methodological Innovations: BBS and CPGA

Batch-Balanced Sampling (BBS)

BBS leverages a dynamically updated, class-structured memory bank to augment incoming batches, selectively down-sampling majority classes and supplementing underrepresented classes with high-confidence, low-uncertainty, temporally relevant instances. Each sample’s viability is scored by an exponential combination of its temporal recency and predictive uncertainty. BBS ensures each adaptation batch maintains class equilibrium, counteracting the temporal overrepresentation biases characteristic of streaming settings. Compared to prior memory-centric strategies (e.g., CSTU in RoTTA), BBS is more responsive to domain shifts owing to its batch-centric design and directly reflects the current incoming distribution.

Figure 1: (a) BBS constructs a balanced batch for adaptation by filtering majority-class samples and supplementing minorities from the memory bank; the bank is continually updated with new samples based on recency and uncertainty.

Category Prototype-Guided Adaptation (CPGA)

CPGA continually maintains class prototypes—essentially, streaming class centroids in feature space—via a momentum-based strategy, updating only with high-confidence pseudo-labeled samples. Unlike classifier recalibration techniques such as T3A, CPGA directly regularizes the student feature extractor by minimizing the cosine distance between each feature and its matched prototype, thus jointly promoting intra-class compactness and inter-class segregation. The prototype initialization harnesses classifier weights from the pre-trained model, with adaptivity governed by a smoothing coefficient and a confidence threshold for update filtering.

Figure 1: (b) CPGA iteratively updates class prototypes with high-confidence features and uses a prototype loss to align features, fostering robust clustering even under domain shift.

Overall Framework and Training Objective

BP-TTA incorporates a mean teacher paradigm for consistent pseudo-label propagation and prediction smoothing. For each balanced batch, the adaptation objective sums a consistency loss (between student and teacher predictions under perturbation) and the prototype alignment loss, jointly promoting fine-grained pseudo-label reliability and stable class-level transitions.

Figure 2: The BP-TTA framework addresses class imbalance through BBS and mitigates domain shift via CPGA, with the overall optimization guided by a combination of consistency and prototype alignment losses.

Experimental Evaluation

BP-TTA is evaluated on CIFAR10-C, CIFAR100-C, ImageNet-C, and DomainNet-126, employing challenging PTTA settings with strong domain shift and different levels of temporal class imbalance simulated via Dirichlet-controlled sampling. All results use state-of-the-art, robust vision architectures as backbones and are benchmarked against leading TTA approaches (TENT, CoTTA, EATA, SAR, PALM, SURGEON, LAME, RoTTA, DA-TTA).

Tabular Performance Highlights and Comparative Analysis

BP-TTA exhibits consistently superior average error rates compared to all baselines across diverse corruption types and datasets. On CIFAR10-C, for example, BP-TTA achieves an average error rate of 21.80%, surpassing the second-best RoTTA by 3.08%. Similarly, notable improvements are observed for CIFAR100-C and ImageNet-C, even as class imbalance becomes increasingly severe.

Robustness and Adaptivity to Shift Dynamics

The ablation study isolates the gains from each module, demonstrating that while BBS alone effectively reduces error via class balancing, CPGA's standalone application is ineffective under extreme imbalance due to prototype drift; combined, the modules synergize to provide strong generalization. Sensitivity analyses reveal optimal prototype smoothing with lower momentum and show resilience to $\lambda$ perturbations (loss weighting factor) across datasets.

Domain generalization results on DomainNet-126 substantiate BP-TTA's applicability beyond corruption-induced shift, indicating its framework generalizes to semantic and appearance-based domain variants.

Figure 3: Distribution of class imbalance as controlled by the Dirichlet parameter; lower values yield stronger temporal dependencies and less class diversity.

Figure 4: Impact of batch size and key hyperparameters $\tau$ (confidence threshold) and $\alpha$ (prototype smoothing) on error rates, indicating stable optima and robust performance.

Feature Structure and TTA Mechanisms

t-SNE projection (Figure 5) provides empirical evidence that CPGA yields more coherent and segregated class clusters in local feature space versus T3A, supporting the claim of enhanced adaptation stability and reduced label noise impact under continuous drift.

Figure 5: t-SNE of CIFAR10-C "Brightness" domain—CPGA induces more distinctive and compact class clusters compared to T3A-derived prototypes.

Sequence Robustness and Frequency of Domain Changes

BP-TTA demonstrates lower performance sensitivity to the sequence of domain shifts and the frequency of distributional transitions (Figures 6, Table \ref{tab:frequency}), a property lacking in existing memory-based methods (e.g., RoTTA), which degrade rapidly under more frequent regime changes.

Figure 6: Error rates for various TTA methods across 10 sequence orderings of domain shifts on ImageNet-C; BP-TTA consistently maintains lower error and variance.

Implications and Future Directions

BP-TTA sets a new baseline for PTTA scenarios through balanced adaptation in both the class-distribution and feature-space granularity, enabling robust operation across temporally evolving, corrupted, and semantically shifting domains. Its composability—with mean teacher schemes, memory banks, and prototype-driven feature alignment—paves the way for extensions leveraging richer memory management policies, confidence-aware pseudo-label propagation, and scalable prototypes for large-category vocabularies.

Practically, BP-TTA is particularly relevant for autonomous systems, online vision services, and streaming analytics, where both distributional shift and class skew are systemic. Theoretically, the method underscores the importance of adapting both the sampling process and the discriminative structure simultaneously—a paradigm that could inform semi-supervised, continual, or federated adaptation schemes.

Potential future work may include scalable BBS for ultra-large class sets, online model selection under compound shift regimes, or hybrid meta-learned adaptation modules that further bridge TTA with robust out-of-distribution generalization.

Conclusion

BP-TTA introduces a dual-mechanism framework to jointly mitigate class imbalance and continual domain shift for online test-time adaptation. By uniting batch-balanced sampling with feature-space prototype guidance, this approach achieves robust performance improvements over state-of-the-art baselines, supports a diverse range of streaming adaptation scenarios, and demonstrates enhanced stability under severe imbalance, frequent shift, and varied sequence orderings. The methodology advances the robustness and applicability of online adaptation methods in dynamic, real-world deployments.