GAT-GAN : A Graph-Attention-based Time-Series Generative Adversarial Network

Abstract: Generative Adversarial Networks (GANs) have proven to be a powerful tool for generating realistic synthetic data. However, traditional GANs often struggle to capture complex relationships between features which results in generation of unrealistic multivariate time-series data. In this paper, we propose a Graph-Attention-based Generative Adversarial Network (GAT-GAN) that explicitly includes two graph-attention layers, one that learns temporal dependencies while the other captures spatial relationships. Unlike RNN-based GANs that struggle with modeling long sequences of data points, GAT-GAN generates long time-series data of high fidelity using an adversarially trained autoencoder architecture. Our empirical evaluations, using a variety of real-time-series datasets, show that our framework consistently outperforms state-of-the-art benchmarks based on \emph{Frechet Transformer distance} and \emph{Predictive score}, that characterizes (\emph{Fidelity, Diversity}) and \emph{predictive performance} respectively. Moreover, we introduce a Frechet Inception distance-like (FID) metric for time-series data called Frechet Transformer distance (FTD) score (lower is better), to evaluate the quality and variety of generated data. We also found that low FTD scores correspond to the best-performing downstream predictive experiments. Hence, FTD scores can be used as a standardized metric to evaluate synthetic time-series data.

• The paper introduced a novel architecture combining graph attention with GANs to capture complex spatiotemporal dependencies in time-series data.
• The methodology employs an adversarially trained autoencoder with tailored encoder, decoder, and discriminator components, evaluated via the Frechet Transformer Distance (FTD).
• Experimental results on datasets such as Motor, ECG, and Traffic demonstrate superior performance in generating high-fidelity synthetic data compared to benchmarks.

Summary of "GAT-GAN: A Graph-Attention-based Time-Series Generative Adversarial Network"

Introduction and Motivation

The paper "GAT-GAN : A Graph-Attention-based Time-Series Generative Adversarial Network" (2306.01999) addresses the challenges posed by traditional GANs when generating realistic multivariate time-series data. Traditional GANs often fail to capture complex temporal and spatial dependencies due to limitations in modeling long sequences with RNN architectures. This hampers their ability to generate high-fidelity synthetic data with dynamic relationships among features and across time steps. In response, the authors propose GAT-GAN, which utilizes graph attention to explicitly model these dependencies through a novel adversarially trained autoencoder architecture.

Proposed Model: GAT-GAN

GAT-GAN introduces a unique framework that combines graph attention with GANs to enhance long-term sequence generation. The architecture comprises three main components:

Encoder

The encoder serves as the generator, mapping input temporal and spatial features to a latent space representation. It utilizes 1D convolutional layers followed by spatial and temporal graph attention blocks, allowing dynamic attention over feature and time-oriented nodes. This setup ensures the capture of long-range dependencies and spatial interactions.

Decoder

The decoder reconstructs the latent representation back into the original feature space, sharing a similar architecture with the encoder but excluding spectral normalization. This component is crucial for fine-tuning the latent space mapping.

Discriminator

The discriminator evaluates the authenticity of generated samples against real data. It shares architectural similarities with the decoder, emphasizing adversarial loss to distinguish synthetic from real embeddings.

Figure 1: Block diagram of the proposed GAT-GAN model, showcasing the encoder, decoder, and discriminator structures.

Evaluation Metrics

The authors introduce the concept of Frechet Transformer distance (FTD) for evaluating time-series data generation, inspired by FID. By utilizing transformer-based embeddings, FTD measures the distance between synthetic and real data distributions. This innovative metric provides a standardized approach to assessing fidelity and diversity in synthetic time-series data.

Additionally, predictive performance is evaluated through the Train on Synthetic - Test on Real (TSTR) framework, measuring how well a model trained on synthetic data performs on real forecasting tasks.

Experimental Results

Through extensive experiments on diverse datasets (e.g., Motor, ECG, Traffic), GAT-GAN consistently demonstrated superior performance compared to benchmarks such as TimeGAN, SigCWGAN, and RCGAN. Notably, GAT-GAN achieved lower FTD scores, signifying higher quality generation. Moreover, its predictive scores on downstream forecasting tasks confirmed robust performance, particularly for longer sequences, underscoring the method's efficacy in capturing long-term dependencies.

Conclusions and Future Work

The paper concludes by highlighting the GAT-GAN's effectiveness in generating realistic time-series data with preserved spatiotemporal dynamics. It proposes FTD as a pivotal metric for future evaluations and establishes the method's potential for broader applications in data privacy and generalization in GANs. Moving forward, research could explore enhancements in dynamic attention mechanisms and extend GAN architectures to leverage additional contextual information.

In sum, GAT-GAN introduces a paradigm shift in time-series data generation, combining graph attention with adversarial training to address inherent GAN constraints, paving the way for more sophisticated applications in synthetic data generation.