ELSTMVAE-DAF-GMM: Anomaly Detection Framework

• The paper introduces a novel hybrid framework (ELSTMVAE-DAF-GMM) that integrates LSTM-VAE temporal encoding, advanced feature fusion, and GMM for superior unsupervised anomaly detection.
• It employs a DAE-LOF based sample filtering mechanism to remove intrinsic anomalies, resulting in a breakthrough performance of 94.6% accuracy and a 5.43% false alarm rate.
• The system demonstrates robust detection across varying sequence lengths and batch sizes, consistently outperforming traditional clustering and deep anomaly detection methods in industrial settings.

The Enhanced Long Short-Term Memory Variational Autoencoder using Deep Advanced Features and Gaussian Mixture Model (ELSTMVAE-DAF-GMM) is an unsupervised anomaly detection framework developed for high-dimensional, multivariate time-series data in critical industrial systems such as steam turbines. By integrating deep learning-based temporal encoding with advanced feature engineering and probabilistic modeling, ELSTMVAE-DAF-GMM addresses the essential challenges of detecting both subtle and intrinsic anomalies in operational sensor readings, offering significant improvements over classical and contemporary deep anomaly detection approaches (Xu et al., 2024).

1. LSTM-VAE Architecture in ELSTMVAE

The core of ELSTMVAE-DAF-GMM is an LSTM-based Variational Autoencoder (LSTM-VAE), designed to capture temporal dependencies and reduce high-dimensional sensor data to a compact latent phase space. Each input $x$ is a sequence of $k$ time steps across $d=19$ sensor channels, $$x = (x_{t-k+1},\ldots,x_t) \in \mathbb{R}^{k \times d}$$.

• Encoder: Two stacked LSTM layers extract temporal features (hidden sizes $h_1=19$, $h_2=8$), followed by a fully-connected output producing the latent mean $\mu(x) \in \mathbb{R}^m$ and log-variance $\log\sigma^2(x) \in \mathbb{R}^m$.
• Latent Variable: The latent embedding $z$ is obtained via reparameterization, $z = \mu(x) + \sigma(x) \odot \epsilon$, with $\epsilon \sim \mathcal{N}(0,I_m)$. The prior $p(z) = \mathcal{N}(0, I_m)$.
• Decoder: Mirrored stacked LSTMs reconstruct the original $k \times d$ sequence from $z$.
• Objective: The VAE loss (ELBO) comprises mean squared reconstruction error and KL divergence:

$$\mathcal{L}_{VAE}(x) = \mathbb{E}_{z \sim q_\phi(z|x)}\bigl[\|x - \hat x\|^2\bigr] + D_{KL}\bigl(q_\phi(z|x)\,\|\,p(z)\bigr)$$

These components facilitate the extraction of low-dimensional, time-aware representations tailored for downstream anomaly detection.

2. DAE-LOF Sample Selection

Prior to LSTM-VAE model training, ELSTMVAE-DAF-GMM employs a combination of Deep Autoencoder (DAE) modeling and Local Outlier Factor (LOF) filtering to remove intrinsic anomalies from the training dataset.

• DAE Training: A fully-connected autoencoder (19→16→10→8→4 encoder, 4→8→10→16→19 decoder) is trained on raw ‘normal’ data.
• Error Calculation: For each sample $x_i$, the squared reconstruction error $e_i = \|x_i - x'_i\|^2$ is computed.
• LOF Outlier Detection: LOF is applied to the error set $E = \{e_1,\ldots,e_N\}$, scoring points by local density deviation. The top $C\%$ of samples (e.g., $C=20\%$) with the highest LOF scores are removed, refining the training set to $X_\text{refined}$.
• Contamination Rate: $C$ is empirically optimized; $C=20\%$ yields best results (accuracy up to 94.6%; lower or higher $C$ impairs performance).

This sample selection mechanism reduces the likelihood of contaminating the model with inherent, undetectable anomalies.

3. Deep Advanced Feature Construction

ELSTMVAE-DAF-GMM introduces Deep Advanced Features (DAF) by hybridizing latent temporal embeddings with reconstruction discrepancy, resulting in enhanced discrimination between nominal and anomalous conditions.

• Feature Vector: For each sample $x$, compute encoded latent $z$ and reconstruction $\hat{x}$. Define the discrepancy $\delta = \|x - \hat{x}\|$ (scalar, L2 norm).
• Concatenation: Form the feature $f(x) = [z; \|x - \hat{x}\|] \in \mathbb{R}^{m+1}$.
• Separation Rationale: This feature formulation synergizes temporal dynamics and deviation from nominal patterns, improving separability of normal and abnormal states in a continuous phase space.

Ablation shows omitting the reconstruction error in features (ELSTMVAE-GMM) degrades accuracy to 74.6% (FAR: 25.4%).

4. Gaussian Mixture Modeling for Unsupervised Detection

After feature construction, ELSTMVAE-DAF-GMM models the normal data distribution using a K-component Gaussian Mixture Model (GMM).

• GMM Density Estimation:

$$p_{GMM}(f) = \sum_{k=1}^K \pi_k \mathcal{N}(f | \mu_k, \Sigma_k), \qquad \sum_{k=1}^K \pi_k = 1$$

• EM Optimization: Standard Expectation-Maximization iteratively updates component means ($\mu_k$), covariances ($\Sigma_k$), and weights ($\pi_k$).
• Anomaly Scoring: Test sample $x$ with $f=f(x)$ is scored by negative log-likelihood: $s(x) = -\log p_{GMM}(f)$. Anomaly threshold $\tau$ is set via validation percentile, controlling the false alarm rate.

This phase outputs unsupervised anomaly likelihoods, robustly flagging operational deviations.

5. Empirical Evaluation and Component Analysis

The method was evaluated on real-world steam turbine data with 19 sensor channels (1-minute sampling), partitioned for training (80% of June 5–29, 2017), validation (20%), and testing (July 13–17, 2017, containing blade wear anomalies).

Key Metrics (Best Configuration)

• Accuracy: 94.6%
• Precision: 94.9%
• Recall: 94.6%
• F1-score: 94.6%
• False Alarm Rate: 5.43%

Comparative Baseline Results

Method	Accuracy	Precision	Recall	F1	False Alarm Rate
GMM (raw)	80.7%	86.1%	80.7%	80.0%	19.3%
K-means	81.5%	86.5%	81.5%	80.9%	18.5%
VAE-GMM	80.1%	85.7%	80.1%	79.2%	19.9%
DAE-GMM	80.2%	85.8%	80.2%	79.4%	19.8%
ELSTMVAE-DAF-GMM	94.6%	94.9%	94.6%	94.6%	5.43%

Ablation and Sensitivity

• No DAE-LOF Pre-filter: Accuracy drops to 91.9%, FAR increases to 8.10%.
• No LSTM Temporal Encoder: Accuracy 92.1%, FAR 7.95%.
• LOF Contamination Parameter: $C=20\%$ optimizes performance; both lower and higher values degrade results.
• Sequence/Batches: Robust across sequence lengths $k \in \{100,150,200,250\}$ and batch sizes $\in \{64,128,256,512\}$, always maintaining AC$>$92%.

6. Integration and Contribution Summary

ELSTMVAE-DAF-GMM unifies three principal innovations:

1. LSTM-VAE for temporal-aware latent encoding of high-dimensional time series.
2. DAE-LOF-based data pre-filtering, excising intrinsic training anomalies.
3. DAF feature construction (latent code plus reconstruction error) with GMM classification.

Each architectural component demonstrably enhances overall performance. The system achieves high sensitivity and specificity for real-world, unlabeled anomaly detection in critical machinery, outperforming both conventional clustering (K-means, GMM) and competing deep frameworks (VAE-GMM, DAE-GMM), with consistent gains validated through ablation studies (Xu et al., 2024).