CADE: Continual Anomaly Detection Ensembles

• CADE is an ensemble-based framework for continual anomaly detection that integrates robust scoring, active learning, and drift adaptation to handle evolving data streams.
• It employs diverse base detectors and principled query selection to efficiently update weights and reduce labeling efforts in dynamic environments.
• CADE enhances performance in applications like fraud prevention and video anomaly detection through adaptive drift handling and generative replay techniques.

Continual Anomaly Detection with Ensembles (CADE) methods address the problem of detecting anomalies in streaming or sequentially arriving data under evolving distributions, leveraging ensemble models to improve robustness, adaptability, and diversity. These frameworks feature prominently in critical tasks such as fraud prevention, large-scale monitoring, and continual weakly-supervised video anomaly detection (WVAD), where handling dynamic environments and preventing catastrophic forgetting are essential. CADE architectures universally exploit ensemble diversity, active learning, drift detection, and principled feedback integration, with recent advances incorporating continual learning and deep generative replay to maintain performance across domain shifts (Das et al., 2018, Xu et al., 2021, Hashimoto et al., 7 Dec 2025).

1. Model Structure and Ensemble Scoring

At the core of CADE systems is an ensemble $\mathcal{E}$ of $m$ base anomaly detectors, each producing a normalized anomaly score $z_j(x)$ for a data instance $x \in \mathbb{R}^d$. These are aggregated into a score vector:

$z(x) = [z_1(x), z_2(x), ..., z_m(x)]^T.$

The ensemble assigns a score via a learned weight vector $w\in \mathbb{R}^m$, constrained by $\|w\|_2=1$:

$$\mathrm{Score}(x) = w \cdot z(x) = \sum_{j=1}^m w_j z_j(x).$$

In tree-based ensembles (e.g., Isolation Forest), each leaf $\ell$ represents a subspace; $z_\ell(x)$ is set to $-d_\ell$ if $x$ falls into leaf $\ell$ at depth $d_\ell$, else $0$. This produces a sparse, normalized $z(x)$ of dimension equal to the number of leaves. The initial $w$ is typically uniform: $w_\text{unif} = (1/\sqrt{m}, ..., 1/\sqrt{m})$ (Das et al., 2018).

In continual weakly-supervised video anomaly detection (WVAD), each video segment is encoded by a frozen feature extractor (e.g., I3D/C3D), with anomaly scores subsequently ensembled across multiple discriminator heads (Hashimoto et al., 7 Dec 2025).

2. Active Learning and Query Selection

CADE systems employ an active learning loop optimized for label efficiency:

• Query Selection: At each step, the instance $x$ with the largest $\mathrm{Score}(x)$ is selected and presented to an analyst for labeling. This greedy selection aligns with uncertainty sampling, optimizing for both rapid anomaly identification and efficient refinement of the scoring hyperplane.
• Weight Update: Following analyst feedback, the ensemble weights $w$ are updated by minimizing a regularized hinge loss over the current labeled sets of positives ($H_\text{pos}$) and negatives ($H_\text{neg}$):

$$\min_w \frac{1}{|H_\text{pos}|}\sum_{z \in H_\text{pos}} \ell(q, w; z, +1) + \frac{1}{|H_\text{neg}|}\sum_{z \in H_\text{neg}} \ell(q, w; z, -1) + \lambda \|w-w_{\text{unif}}\|^2,$$

where $\ell$ is a hinge loss, $q$ is an adaptively chosen threshold, and $w_{\text{unif}}$ is the initialization (Das et al., 2018).

This process operates in both batch and streaming regimes. In streaming active learning (SAL), the process merges new and historic high-scoring instances, manages a query budget, and adapts in response to feedback.

3. Drift Detection and Continual Adaptation

CADE introduces a robust, ensemble-driven method for maintaining performance under changing data distributions:

• Per-tree KL Divergence Test: For each tree, leaf histograms estimated from recent windows are compared to historical baselines using Kullback–Leibler (KL) divergence. Trees exhibiting $D_{KL} > q_{KL}$ are considered "drifting." If the proportion of drifting trees exceeds a threshold ($\geq2\alpha_{KL}\cdot T$), drift is declared.
• Tree Replacement and Adaptation: Drifting trees are replaced with new ones trained on current data. Ensemble weights for new leaves are initialized uniformly and renormalized. This mechanism ensures that model adaptation is localized and robust, avoiding catastrophic performance losses (Das et al., 2018).

Replay-based adaptation, as implemented in continual WVAD, employs dual deep variational autoencoder GANs ($\mathcal{G}_n$ for normals, $\mathcal{G}_a$ for anomalies) to regenerate past domain features, preventing forgetting without storing real data. Multi-head discriminators ($\mathcal{D}$, $\mathcal{D}_n$, $\mathcal{D}_a$) capture distinct anomaly modes across evolving domains (Hashimoto et al., 7 Dec 2025).

4. Diversity and Compact Description

CADE systems employ formalized descriptions of anomalous groups to improve diversity and interpretability of anomaly discovery:

• Compact Description Formalism: For a set $Z$ of instances, the union of their $\delta$ most-relevant leaves yields candidate subspaces. A binary selection vector $x\in\{0,1\}^k$ is optimized via integer programming to cover all instances with the minimum total subspace volume:

$$\min_{x \in \{0,1\}^k} v^T x \quad \text{s.t.} \quad Ux \geq \mathbf{1}_p,$$

where $v$ contains subspace volumes and $U$ is a $p\times k$ coverage matrix. The resulting compact cover supports batch selection with minimal overlap, increasing the variety of anomaly classes per batch by 20–50% without degrading overall detection rate (Das et al., 2018).

• Diversity Regularization in Ensembles: In continual WVAD, an explicit diversity term penalizes discriminator heads that converge to similar solutions, sustaining model robustness across multiple tasks and data modalities (Hashimoto et al., 7 Dec 2025).

5. Algorithms and Pseudocode

CADE methodology is precisely specified through procedural pseudocode in both batch and streaming settings.

Batch Active Learning (BAL):

Initialization with $w_\text{unif}$; at each iteration, score all candidates, query the top instance, update the labeled pools and ensemble weights, normalize, and repeat for $B$ queries.

Streaming Active Learning (SAL) with Drift Adaptation:

At each window, update per-tree histograms, detect drift via KL-testing, replace drifting trees, merge and rank top-scoring instances, query for labels, and update weights.

Continual WVAD (Generative Replay):

For each new domain, sample real and replay (generator-based) features to construct bags, update discriminator and generator via joint loss including MIL ranking, GAN, and diversity regularization, and ensemble multi-head scores for final decisions (Das et al., 2018, Hashimoto et al., 7 Dec 2025).

6. Empirical Results and Comparative Performance

CADE variants demonstrate superior label-efficiency and adaptability over static and non-ensemble baselines:

• Batch Setting: On public datasets (e.g., Abalone, Thyroid, Cardio), BAL discovers 2–3$\times$ more anomalies within a given query budget than unsupervised Isolation Forest, and consistently outperforms High-Scoring Trees (HST) and Random Subspace Forest (RSF) regardless of feedback (Das et al., 2018).
• Streaming Setting: SAL with adaptive drift detection matches batch performance and achieves stable detection under non-stationarity. On concept-drifted electricity and Covtype data, drift adaptation maintains high label-efficiency with minor degradation (Das et al., 2018).
• Continual WVAD: On ShanghaiTech and Charlotte Anomaly, applying CADE to standard WVAD backbones (MIST, RTFM, UR-DMU) provides AUC gains of $+0.08$–$+0.23$ over baselines. Ablation confirms additive gains from dual generator, multi-discriminator heads, and latent separation (Hashimoto et al., 7 Dec 2025).

Setting/Dataset	CADE Method	Main Result
Batch anomaly bench	BAL	2–3$\times$ anomalies found (vs. IFOR)
Streaming w/ drift	SAL (KL-Adaptive)	Matches batch; robust to drift
Multi-scene VAD (SHT)	MIST w/ CADE Ensemble	$\mathrm{AUC}=0.8490$ (vs. $0.5640$ FT)

7. Theoretical Guarantees and Practical Considerations

CADE frameworks incorporate the following guarantees and practical strategies:

• Type-I Error and Coverage: When paired with conformal methods and appropriate bootstrapped ensembles, approximate marginal error control is achievable under mild dependence assumptions (Xu et al., 2021).
• Parameter Guidelines: Practical tuning includes normalizing score vectors, setting drift sensitivity (e.g., $\alpha_{KL}=0.05$), anomaly rate prior ($\tau$), number of top leaves ($\delta=5$), and batch size ($b=3$–$5$) (Das et al., 2018).
• Limitations: CADE performance is contingent on the generative replay quality (for generative-ensemble variants). There is residual performance gap relative to multi-task learning (oracle joint training), and tuning of replay or batch ratios remains dataset-dependent (Hashimoto et al., 7 Dec 2025). Handling continuously drifting, unlabeled data and integrating online feedback represent ongoing challenges.

---

CADE stands as a general framework unifying ensemble-based anomaly detection, active label-efficient querying, robust handling of non-stationarity (via explicit drift checks or continual generative replay), and principled diversity promotion, with domain-specific specialization for spatio-temporal and video anomaly detection (Das et al., 2018, Xu et al., 2021, Hashimoto et al., 7 Dec 2025).