TSER: Time Series Entity Resampler

• TSER is a two-stage bootstrap resampling method using GANs and TCNs to learn and replicate the full distribution and serial dependence in stationary time series.
• It outperforms traditional circular block bootstrap methods, particularly in AR(1) settings, by providing competitive empirical coverage across various nominal levels.
• Leveraging adversarial training and a modular TCN design, TSER allows efficient generation of synthetic series without block stitching or post-processing adjustments.

The Time Series Entity Resampler (TSER) is a two-stage bootstrap methodology for dependent data based on generative adversarial networks (GANs), designed to learn and replicate the full distributional and serial dependence structure of stationary time series processes. TSER leverages temporal convolutional networks (TCNs) for both generator and discriminator networks, enabling direct sampling of realistic synthetic time series without reliance on traditional block-based resampling schemes. The method has demonstrated competitive to superior empirical coverage properties relative to circular block bootstrap (CBB) in autoregressive (AR(1)) settings, particularly at moderate levels of temporal dependence (Dahl et al., 2021).

1. Formal Model and Adversarial Objective

TSER is instantiated in the context of stationary AR(1) processes, though the underlying design is modular in its model applicability. For observed data $x_t = \phi x_{t-1} + \epsilon_t$ with $\epsilon_t \sim \mathcal{N}(0,\sigma^2)$ and $|\phi|<1$, TSER addresses the resampling problem by adopting an adversarial generative framework.

Let $G$ denote a neural network generator that maps Gaussian noise vectors $z \sim \mathcal{N}(0,I)$ to synthetic data vectors $\hat{x}_{1:T}$ and $D$ a discriminator mapping sequences $x_{1:T}$ to scalar outputs. The standard GAN min-max problem is given by

$$\min_{\theta_G}\max_{\theta_D} V(D,G) = \mathbb{E}_{x \sim P_{\text{data}}}[\log D(x;\theta_D)] + \mathbb{E}_{z \sim \mathcal{N}(0,I)}[\log(1-D(G(z;\theta_G);\theta_D))].$$

For stability and improved empirical performance, TSER employs the Wasserstein-GAN with gradient penalty (WGAN-GP), solving the following alternating maximization and minimization:

• Discriminator (D-step):

$$\max_{\theta_D} \mathbb{E}_{x \sim P_{\text{data}}}[D(x;\theta_D)] - \mathbb{E}_{z \sim \mathcal{N}}[D(G(z;\theta_G);\theta_D)] - \lambda \mathbb{E}_{\tilde{x}}[(\|\nabla_{\tilde{x}} D(\tilde{x};\theta_D)\|_2 - 1)^2],$$

with $$\tilde{x} = a x_{\text{real}} + (1-a)x_{\text{fake}}, a \sim \text{Unif}(0,1).$$

• Generator (G-step):

$$\min_{\theta_G} -\mathbb{E}_{z \sim \mathcal{N}}[D(G(z;\theta_G);\theta_D)].$$

The gradient-penalty weight $\lambda$ enforces the necessary Lipschitz constraint; batch normalization is not used.

2. Temporal Convolutional Network Architectures

Both generator and discriminator in TSER are constructed from one-dimensional causal dilated convolutional layers, forming a TCN architecture with receptive field and depth tailored to capture temporal dependencies.

• Generator $G_\theta$: Accepts noise $Z \in \mathbb{R}^{(b+p) \times n_z}$ with i.i.d. standard normal entries as input. The architecture includes 6 dilated convolutional layers with filters (128, 64, 32, 32, 16, 1), kernel size 2, and exponential dilations $\delta_i=2^{i-1}$. All but the final layer use tanh activations; output layer is linear, producing a synthetic block $\hat{x} \in \mathbb{R}^{b}$.
• Discriminator $D_\psi$: Receives real or synthetic blocks $x \in \mathbb{R}^{b}$; utilizes 6 dilated convolutional layers (filters: 8, 16, 32, 32, 64, 64), kernel size 2, matching generator dilations. Leaky-ReLU activations are used in each layer; layer outputs are aggregated using adaptive max pooling to form a feature vector, input to two fully-connected layers yielding the scalar WGAN-GP score $D(x)$.

The receptive field is $p = 2^d - 1$ for $d$ layers, controlling maximum temporal range.

3. Two-Stage TSER Resampling Algorithm

The TSER methodology proceeds through two principal stages: adversarial training and bootstrap sampling.

Stage 1: GAN Training

1. Sample mini-batches of real blocks from the observed series $y^*_{1:T}$ of block length $b_1$.
2. Independently generate fake blocks by sampling noise $Z_j$ and evaluating $G_\theta(Z_j)$.
3. Update the discriminator $\theta_D$ multiple times per iteration using the WGAN-GP loss with gradient penalty.
4. Alternate with generator $\theta_G$ updates based on the negative expected discriminator output on generated blocks.

Stage 2: Bootstrap Sampling

1. After training, fix $\theta_G$.
2. For each bootstrap replicate, draw noise $Z$ and generate a synthetic series $x^{(s)} = G_{\theta_G}(Z)$ of target length $b_2$.
3. Compute bootstrap statistics $\hat{\theta}^{(s)} = \text{estimator}(x^{(s)})$ (e.g., least-squares AR(1) coefficient).
4. Aggregate the bootstrap estimates to obtain percentile confidence intervals.

No detrending, variance scaling, or other post-processing is required; stationarity and dependence structure are preserved by design (Dahl et al., 2021).

4. Empirical Evaluation Against Circular Block Bootstrap

TSER’s performance is benchmarked for empirical coverage accuracy against CBB in AR(1) settings:

• Data are simulated from AR(1) processes with $\phi \in \{0.5, 0.8, 0.9\}$, $\sigma^2=1$, $T=1000$.
• GAN block length $b_1=150$, mini-batch size $n_b=64$, $S=10^4$ synthetic series, $M=1000$ independent DGP draws.
• Empirical coverage of percentile confidence intervals for $\phi$ at nominal levels (80%, 90%, 95%, 99%) is evaluated.

Key results:

• For $\phi \leq 0.8$, TSER delivers empirical coverage at all nominal levels closer to target values than CBB, robust to CBB block size choices.
• TSER reliably replicates sample ACF and PACF structure of the AR(1), unlike CBB, which exhibits under-dispersion at higher $\phi$.
• For $\phi=0.9$, both methods experience degraded performance, but TSER with increased TCN depth remains competitive or superior.

5. Strengths, Limitations, and Extensions

Strengths

• TSER learns and replicates both marginal and serial dependence structures, not limited to blockwise overlap.
• Once trained, the generator can produce an arbitrary number of long synthetic trajectories without block stitching.
• Requires selection of a single TCN architecture and training block length, avoiding the need for CBB block size tuning.

Limitations

• GAN training is computationally intensive, requiring significantly more time per replication than CBB.
• Multiple GAN and TCN hyperparameters demand tuning.
• Lack of general asymptotic theory for the validity of the TSER bootstrap in settings with complex dependence.
• Limited ability to capture very long-range dependence (near-unit-root processes) unless receptive field is enlarged; risk of mode collapse.

Possible Extensions

• Multivariate TSER for vector autoregressions (VAR) by broadening convolutional filters.
• Integration of conditional GANs for exogenous covariates or regime-switching dynamics.
• Deeper or wider TCNs to accommodate ARMA($p$,$q$) models or fractional processes.
• Embedding normalizing flows or GARCH-style layers for heavy-tailed marginal distributions.
• Theoretical investigation of convergence properties under mixing conditions.

6. Context and Related Methodologies

TSER represents a departure from traditional bootstrap and surrogate data techniques for time series by leveraging adversarial training to directly match full joint distributions. The use of WGAN-GP (Arjovsky et al. 2017; Gulrajani et al. 2017) and TCN architectures enables explicit modeling of stationary dependence structures. Compared to block bootstrap methods, which are constrained by block length tuning and blockwise structural assumptions, TSER provides a model-agnostic framework that internalizes temporal relationships, subject to practical resource and hyperparameter considerations (Dahl et al., 2021).

A plausible implication is that further advances in GAN theory and architecture may directly expand the range of stationary and non-stationary processes amenable to GAN-based bootstrap resampling, particularly with extensions for multivariate, regime-switching, and long-memory contexts.