TimePFN: Foundation Models for Forecasting

• TimePFN is a family of models for time series forecasting that uses synthetic data generated via LMC-Synth and structured priors to approximate Bayesian inference accurately.
• The model employs a transformer-based backbone with 1D convolutional filtering and patch embeddings to effectively capture temporal dynamics and cross-channel dependencies.
• Empirical results show that TimePFN achieves robust zero-shot and few-shot performance on both multivariate and univariate benchmarks, setting a new standard for synthetic pretraining.

TimePFN refers to a family of foundation models for time series forecasting built upon the Prior-data Fitted Network (PFN) paradigm. TimePFN models are trained exclusively or predominantly on synthetic datasets sampled from expressive, structured priors capable of modeling a broad range of temporal and cross-channel dependencies. This approach realizes approximate Bayesian inference for time series, delivering strong zero-shot and few-shot forecasting performance, especially in multivariate or data-sparse settings. Recent TimePFN variants use either transformer-based or RNN-based architectures and have demonstrated leading results on standard benchmarks for both univariate and multivariate forecasting scenarios (Taga et al., 22 Feb 2025, Moroshan et al., 29 Oct 2025, Dooley et al., 2023).

1. The PFN Paradigm for Time Series

TimePFN builds on the PFN methodology, which aims to approximate Bayesian posterior predictive inference using a single neural network $f_\theta$. For a multivariate time series $\mathcal{D} = \{(t, X_t)\}_{t=1}^T$, with $X_t \in \mathbb{R}^N$, the model is trained on synthetic episodes drawn from a distribution induced by a prior $p(\omega)$ over generative models $\omega \in \Omega$. Each episode is split into an observed “history” and an “outcome” segment:

• At test time, the model is given a new $\mathcal{D}_{\mathrm{in}}$ and directly predicts future values via $$f_\theta(\mathcal{D}_{\mathrm{in}})\approx\mathbb{E}[x \mid T, \mathcal{D}_{\mathrm{in}}]$$.
• In TimePFN, $\Omega$ is the space of single-input, multi-output Gaussian Processes under the Linear Model of Coregionalization (LMC), providing a flexible and realistic prior over MTS (Taga et al., 22 Feb 2025).

2. Synthetic Data Generation with LMC-Synth

A crucial component of TimePFN is the LMC-Synth synthetic data generator:

• Latent Function Bank: Multiple base GP kernels (Linear, RBF, Periodic, Rational-Quadratic) are composed through random combinations as in kernel discovery [Duvenaud et al., 2013].
• Latent Sample: For each of $L$ latent processes, sample $l_j(t) \sim \mathcal{GP}(0, k_j(t, t'))$.
• Channel Mixing: For $N$ channels, Dirichlet-distributed weights $\alpha_{i,j}$ control the mixing of $L$ latent sources, yielding channel $i$ as $C_i(t)=\sum_{j=1}^L \alpha_{i,j} l_j(t)$.
• Prior Diversity: Varying Dirichlet concentration $d$ transitions synthesized data from independent to highly correlated channels, capturing realistic cross-channel variation.
• Synthetic Dataset: Typical pretraining settings include $15{,}000$ datasets of length $1024$ and $N=160$ channels, with sliding windows to generate $\sim 1.5$ million training pairs (Taga et al., 22 Feb 2025).

3. Model Architecture and Data Flow

The TimePFN architecture employs a transformer-based backbone with explicit mechanisms for temporal and cross-channel modeling:

• 1D Convolutional Filtering: Learnable $C=9$ convolutional filters are applied to each channel for temporal preprocessing; the filtered and original signals are concatenated.
• Patching and Embedding: Data are divided into overlapping patches (patch size $P=16$, stride $S=8$) across all channels. Each patch is embedded via a 2-layer MLP to a $D$-dimensional token, with 2D sinusoidal positional encoding (time and channel axes).
• Channel-Mixing Transformer Encoder: All tokens are pooled and processed jointly by a standard transformer (8 layers, 8 heads, $D=256$ embeds, latent size $1024$), enabling inter-channel attention and joint inference across the MTS.
• Output Head: Channel-wise grouped tokens are flattened and passed through a shared feedforward head to predict $T_\mathrm{out} = 96$ future points.
• Input Flexibility: The model is designed to handle arbitrary numbers of channels at inference by stacking or chunking as needed (Taga et al., 22 Feb 2025).

4. Training Schemes: Pretraining and Fine-Tuning

The training process consists of two stages:

• Synthetic Pretraining: The model is trained to minimize MSE between predicted and true synthetic outcomes, using Adam optimizer (peak LR $5 \cdot 10^{-4}$, one-cycle), with regularization via input noise and curriculum learning over the independence-correlated prior spectrum.
• Few-Shot Fine-Tuning: Given a small budget $B$ of real series-windows ($B\in\{50,100,500,1000\}$), the pretrained model is further fine-tuned for 8 epochs using AdamW (peak LR $2 \cdot 10^{-4}$), with hyperparameters held fixed across datasets. This allows rapid adaptation to new domains.

5. Inference and Generalization

TimePFN supports both zero-shot and few-shot forecasting modalities:

• Zero-Shot Prediction: The pretrained model is directly applied to real datasets without additional training. Predictions approximate the posterior mean for the task prior induced by the synthetic generator.
• Few-Shot Adaptation: The model is fine-tuned on a small subset of real data, typically matching full-data training performance with only $500$ points, and remaining competitive with as few as $50$ points.
• Uncertainty: The current TimePFN instantiation predicts point estimates (posterior means) only; full posterior predictive distributions are not computed (Taga et al., 22 Feb 2025). A plausible implication is that extending TimePFN to probabilistic outputs is a future direction.

6. Empirical Performance and Ablation Analysis

TimePFN exhibits strong empirical results across multiple benchmarks:

• Multivariate Forecasting: On nine standard datasets (input/forecast length $96$), TimePFN is best in zero-shot settings on 7/9, and in few-shot ($B=50$ and $B=500$) remains SOTA or competitive.
• Univariate Forecasting: TimePFN outperforms deep-learning baselines in zero-shot mode (input 36, forecast horizons $6$–$48$).
• Ablations:
  • Removing 1D convolutions roughly doubles zero-shot MSE.
  • Substituting PatchTST transformer backbones degrades performance relative to the channel-mixing TimePFN architecture.
  • Restricting the prior to independent channels or omitting synthetic pretraining significantly impairs both zero-shot and few-shot results.
  • Pretraining other transformer backbones with the LMC-Synth prior (e.g., iTransformer-PFN) improves their zero/few-shot performance but does not reach TimePFN levels (Taga et al., 22 Feb 2025).

Setting	Zero-shot SOTA	Few-shot SOTA ($B=50$)	Full-data SOTA
Multivariate MTS	7/9 datasets	all datasets	4/9 datasets
Univariate MTS	best DL baseline	—	—

7. Comparative Approaches and Related Models

TimePFN relates most closely to the ForecastPFN and TempoPFN families:

• ForecastPFN: A synthetically-trained, zero-shot forecasting model using a small transformer and an expressive synthetic prior, focused on univariate series. ForecastPFN achieves competitive or superior MSE versus classical and transformer baselines in low-data regimes (input lengths $\sim 36$) (Dooley et al., 2023).
• TempoPFN: Employs a PFN-style approach based on a GatedDeltaProduct linear RNN with state-weaving for efficient, parallelizable, long-horizon univariate forecasting. The training corpus is drawn from a mixture of 10 advanced synthetic generators and augmented via a unified offline and stochastic augmentation pipeline. TempoPFN outperforms other synthetic-only baselines and is competitive with, or better than, most models trained on real data on the Gift-Eval benchmark (Moroshan et al., 29 Oct 2025).
• Time Series Foundation Models: TimePFN, ForecastPFN, and TempoPFN exemplify a foundational modeling paradigm where universal neural forecasters are pretrained on highly expressive synthetic priors, enabling strong zero/few-shot adaptation and efficient scaling to unseen domains.

• "TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data" (Taga et al., 22 Feb 2025)
• "ForecastPFN: Synthetically-Trained Zero-Shot Forecasting" (Dooley et al., 2023)
• "TempoPFN: Synthetic Pre-training of Linear RNNs for Zero-shot Time Series Forecasting" (Moroshan et al., 29 Oct 2025)