Weakly Supervised Anomaly Detection

• WSAD is a framework that leverages limited, noisy anomaly labels alongside abundant normal data to enhance detection accuracy.
• The approach integrates density estimation, ranking-based surrogate losses, and score distribution modeling to separate anomalies from normal instances.
• Empirical studies show that WSAD methods boost AUC performance, maintain robustness under label scarcity, and offer efficient, scalable solutions for diverse data types.

Weakly Supervised Anomaly Detection (WSAD) encompasses @@@@1@@@@ where only limited, noisy, or partially labeled anomaly information is available during training. The paradigm is motivated by real-world settings in which precise anomaly annotation is costly or infeasible, yet some supervisory signal can be leveraged to enhance detection beyond purely unsupervised approaches. Techniques subsumed under WSAD include methods using scarce anomaly labels, inexact or groupwise labels, auxiliary information, and semi-supervised regularization with dense normal data. Recent advances in WSAD integrate density estimation, explicit score distribution modeling, augmented representation learning, and loss functions designed to optimize discriminatory power with minimal supervision.

1. Core Principles of Weak Supervision in Anomaly Detection

The foundation of WSAD is the exploitation of limited or imprecise anomaly information together with (often abundant) unlabeled or normal data. This includes:

• Scarce Anomaly Labels: Many approaches improve upon unsupervised models by incorporating a handful of labeled anomalies, using smooth ranking constraints or margin-based objectives to push up anomaly scores for labeled examples while retaining normality structure in the score space (Iwata et al., 2019). For instance, density models can be regularized so that normal instances are assigned higher likelihood than labeled anomalies.
• Bundle or Inexact Labeling: In settings where group labels (e.g., a bag containing at least one anomaly) are provided, WSAD employs surrogate objectives such as inexact AUC, optimizing the anomaly score so that the maximum score within a group is higher than that for independently drawn normal examples (Iwata et al., 2019).
• Score-Distribution Discrimination: Overlap-loss frameworks move beyond pairwise or margin-based discrimination, directly minimizing the overlap between score distributions of labeled anomalies and large pools of (potentially contaminated) unlabeled points (Jiang et al., 2023).
• Semi-Supervised Regularization: Joint losses comprising unsupervised objectives (e.g., autoencoder reconstruction) and supervised criteria ensure that models are not overfitted to the few available anomalous labels, but preserve the structure of normality (Iwata et al., 2019, Iwata et al., 2019).

2. Mathematical Formulations and Optimization Objectives

The distinguishing feature of WSAD is its loss design; representative mathematical formulations include:

• Sigmoid-Relaxed Ranking for Labeled Anomalies:

$$L(\theta) = L'(\theta) + \frac{\lambda}{|\mathcal{A}| |\bar{\mathcal{A}|}} \sum_{n \in \mathcal{A}} \sum_{n' \in \bar{\mathcal{A}}} f\left(\log \frac{p(x_{n'}|\theta)}{p(x_n|\theta)}\right)$$

where $f(s)$ is the sigmoid, $L'$ maximizes log-likelihood for normals, and $\lambda$ regularizes the anomaly ranking term (Iwata et al., 2019).

• Smooth Inexact AUC Maximization:

$$\mathcal{L}(\theta) = \frac{1}{|\mathcal{N}|}\sum_{x_j^N} a(x_j^N; \theta) - \frac{\lambda}{|\mathcal{S}||\mathcal{N}|} \sum_{\mathcal{B}_k \in \mathcal{S}} \sum_{x_j^N} \sigma\left( \max_{x_i \in \mathcal{B}_k} a(x_i; \theta) - a(x_j^N; \theta) \right)$$

using the $\max$ within bundles and a sigmoid surrogate for differentiable ranking (Iwata et al., 2019).

• Overlap Loss on Score Distributions:

$$\mathcal{L}_{Overlap}(\Theta) = 1 - \hat F_n(c) + \hat F_a(c)$$

with $c$ as the intersection of empirical anomaly/normal score PDFs (via batchwise KDE), distributing loss to minimize misclassification error by shifting score distributions apart (Jiang et al., 2023).

These formulations are universally designed for end-to-end gradient optimization, and can be integrated into any network architecture producing scalar anomaly scores.

3. Architectures and Training Procedures

WSAD methods typically build atop unsupervised representation learners (autoencoders, density models, discriminative networks), augmented with supervisory heads and loss modifications:

• Neural Density Estimators: Autoregressive models (e.g., MADE with masked feedforward nets and mixture outputs) permit tractable likelihood assignment and exact density evaluation required for supervised likelihood regularization (Iwata et al., 2019).
• Autoencoder-Based Rankers: Encoder-decoder architectures output reconstruction errors, which are converted to anomaly scores and shaped using inexact or overlap-based loss terms (Iwata et al., 2019, Jiang et al., 2023).
• Score-Distribution Estimator: Training leverages KDE modules for score distributions (with bandwidth selection and differentiable implementation for gradient flow) in the overlap loss setting (Jiang et al., 2023).

Hyperparameter selection (e.g., $\lambda$ regularization weights, KDE bandwidth, number of neighbors in kNN-graph-based methods) is consistently performed on held-out validation sets or via cross-validated performance criteria, with early stopping typically used for stability.

4. Statistical and Detection Properties

WSAD frameworks achieve several theoretical and empirical properties:

• AUC Alignment and Monotonicity: Supervised ranking and overlap losses demonstrate explicit maximization of area-under-ROC and minimal overlap between anomaly and normal score distributions, often converging to the optimal Neyman-Pearson detector in the large-sample limit (Iwata et al., 2019, Jiang et al., 2023).
• Distribution Robustness: KDE-based score estimation and minimization of overlap area retain the diversity and fine-grained information of input data, providing robustness to contamination in unlabeled pools (Jiang et al., 2023).
• Handling of Hard Negatives: In inexact-label settings, only the maximal score in a bundle is encouraged to be anomalous, avoiding penalization of hard negatives and preventing excessive false positive rates (Iwata et al., 2019).
• Global Density Structure Preservation: Joint unsupervised–supervised losses avoid degenerate solutions where only the labeled anomalies are separated, instead retaining the global normality manifold (Iwata et al., 2019).
• False-Alarm Control and Statistical Optimality: kNN graph-based approaches and score ranking schemes achieve asymptotic guarantees—test points can be declared anomalous at a specified false-alarm level $\alpha$, with decision regions converging to minimum-volume sets under the underlying density (Qian et al., 2015).

5. Experimental Results and Quantitative Analysis

Empirical studies consistently confirm the efficacy of WSAD approaches:

• Performance with Scarce Labels:
  • On 16 benchmarks, increasing labeled anomalies from 0 to 5 in the supervised density model improved average AUC (from 0.807 → 0.859), surpassing both unsupervised and margin-based supervised baselines (Iwata et al., 2019).
  • Inexact-label autoencoder methods outperform fully supervised and multiple instance learning baselines, with improved standard AUC on test sets and robustness to label scarcity (Iwata et al., 2019).
• Overlap Loss Generalization:
  • Across 25 tabular benchmarks, MLP/AE/ResNet-Overlap consistently outperformed DevNet, FEAWAD, DeepSAD, REPEN, and other semi-supervised methods in AUC-PR and AUC-ROC, especially at low anomaly-label ratios (γℓ = 5%–20%) (Jiang et al., 2023).
• Robustness to Label Contamination:
  • Models using overlap loss or bundle-based ranking retained performance even when unlabeled sets included moderate contamination by hidden anomalies (Jiang et al., 2023, Iwata et al., 2019).
• Computational Efficiency:
  • RKHS rankers imitating kNN anomaly scores reduced test-time complexity by factors of 2–10 relative to classical kNN algorithms (Qian et al., 2015).
  • Overlap loss models trained in comparable or fewer epochs relative to classic alternatives, owing to efficient bounded loss and absence of manual score-target tuning (Jiang et al., 2023).

6. Limitations, Extensions, and Future Directions

WSAD remains an active area characterized by practical considerations and open questions:

• Label Quality and Scarcity: Methods rely on the assumption that available anomaly labels, though sparse or noisy, are not grossly misrepresentative; model performance degrades with label contamination approaching random levels.
• Modalities: Although tabular and structured data dominate current benchmarks, applications to time series, vision, and high-dimensional scientific data are increasingly prevalent. Adaptation via self-supervised rankings, contrastive augmentation, and representation regularization is effective (Iwata et al., 2019, Jiang et al., 2023).
• Hyperparameter-Free Losses: Overlap-based WSAD frameworks minimize reliance on hand-tuned constants and score-targets, improving portability across datasets (Jiang et al., 2023).
• Scalable Label Integration: Future work targets functional losses that can directly incorporate subgroup, ordinal, or severity-based supervision, permitting granular control of anomaly scoring under multi-level or hierarchical conditions.

7. Representative WSAD Methods

Method (Paper)	Loss Principle	Label Requirement	Score Definition
Supervised MADE (Iwata et al., 2019)	Density + ranking sigmoid	Few labeled anomalies	$s(x) = -\log p(x)$ via autoregressive NN
Inexact-AUC AE (Iwata et al., 2019)	Smooth surrogate for bundle labels	Inexact/Groupwise	$a(x)$: AE error, $\max$ within groups
Overlap Loss (Jiang et al., 2023)	Score-distribution intersection	Few anomalies, many unlabelled	$s(x)$ via end-to-end KDE separation

These methodologies represent the current state of the art, demonstrating the versatility and empirical superiority of loss designs tailored to limited, noisy, or grouped labels in real-world WSAD scenarios.