Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting

Abstract: Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has been a critical driver of performance of foundation models in other modalities such as language and vision, much recent work on time series foundation modeling has focused on scaling. This has resulted in time series foundation models with hundreds of millions of parameters that are, while performant, inefficient and expensive to use in practice. This paper describes a simple recipe for learning efficient foundation models for zero-shot time series forecasting that are orders of magnitude smaller. We show that large-scale transformers are not necessary: small hybrid models that interleave long convolution and linear RNN layers (in particular DeltaNet layers) can match the performance of larger transformer-based models while being more than a hundred times smaller. We also describe several data augmentation and inference strategies that further improve performance. This recipe results in Reverso, a family of efficient time series foundation models for zero-shot forecasting that significantly push the performance-efficiency Pareto frontier.

• The paper demonstrates that a hybrid architecture combining FFT-based convolutions and DeltaNet layers achieves competitive zero-shot forecasting with under 0.1% of the parameter count of traditional models.
• It employs advanced inference strategies like algorithmic flip equivariance and adaptive downsampling to capture long-range periodicities effectively.
• The integrated data pipeline, featuring synthetic data generation and multiple augmentation techniques, is essential for robust performance across heterogeneous time series.

Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting

Motivation and Context

Time series forecasting (TSF) is central to multiple domains, including energy, traffic, finance, and healthcare. Foundation models (FMs) for TS have recently enabled domain-general forecasting capabilities, with zero-shot transfer across heterogeneous tasks, mirroring trends in NLP, CV, and biology. However, the dominant paradigm in TSFMs has been aggressive scaling—models with hundreds of millions to over a billion parameters, with non-trivial compute and deployment costs. The paper "Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting" (2602.17634) fundamentally interrogates this assumption. It demonstrates that large-scale transformer models are not required: compact hybrid models built from convolutional and linear RNN (DeltaNet) blocks can deliver state-of-the-art zero-shot TS forecasting at less than 0.1% the parameter cost.

Model Architecture

Reverso models combine long convolutional sequence-mixing primitives with linear RNNs (DeltaNet). The architecture processes a normalized input sequence $t \in \mathbb{R}^L$ via a single linear embedding layer. Sequence mixing alternates between hardware-efficient long convolutions (FFT-based) and parametric DeltaNet blocks for capturing dependencies. Channel mixing uses conventional MLP layers with dimension expansion. The prediction head employs an attention-based mechanism to decode parallel chunks ($p$ points) per autoregressive rollout, eschewing domain-specific embeddings or metadata. Notably, the final output head applies attention over transformed representations, improving performance/parameter ratio versus linear decoders.

Figure 1: The Reverso architecture, combining FFT-based long convolutions, DeltaNet layers, MLP channel mixing, and an attention-based decoder head for efficient zero-shot forecasting.

Three model variants span 0.2M–2.6M parameters. Architectural ablations confirm hybrid composition (conv+DeltaNet) offers superior tradeoff vs pure attention, linear attention, or convolution.

Data Pipeline: Augmentation and Synthetic Generation

Pretraining leverages GiftEval, a large-scale, balanced TS corpus. To avoid domain imbalance and data scarcity, Reverso incorporates a multifaceted augmentation and synthetic generation pipeline. Augmentation includes:

• Downsampling (enables learning across temporal resolutions)
• Amplitude modulation (piecewise transforms)
• Horizontal/vertical flips (temporal/sign reversal)
• Censoring
• Batch-level mixup

Synthetic data is generated via Gaussian processes with randomly composed kernels from a bank, enriched with spike, trapezoidal, trend/seasonality/irregularity, and anomalous patterns. This pipeline significantly broadens coverage of input distributions and frequency patterns.

Figure 2: Left: Data augmentation pipeline with downsampling, modulation, flips, censor, mixup. Right: Synthetic data generated via Gaussian processes, kernel composition, trend, seasonality, and spike processes.

Empirical ablations verify that augmentation/synthetic data are essential, especially for performance at low parameter counts.

Inference Strategies

Reverso adopts algorithmic flip equivariance (forward passes with both original/flipped context, output averaging), and adaptive downsampling, guided by FFT spectrum analysis. Downsampling ensures the model's context window captures long-range periodicities, which is crucial for accurate forecasting when sequence period exceeds context length. Downsampling stride is computed to maximize the number of periods covered, and only applied when a dominant peak is detected in the amplitude spectrum.

Figure 3: Downsampling enables the model to fit multiple periods within the context window, improving forecasting accuracy for long periodic sequences.

Empirical Results and Numerical Performance

GiftEval Benchmark

Reverso-2.6M achieves a MASE of 0.711, outperforming similarly sized TSFMs such as Super-Linear, FlowState, and TinyTime Mixers. Reverso-Small (550K parameters) further improves to MASE 0.726. Performance across short, medium, and long forecast horizons is competitive with several transformer baselines (e.g., Xihe-Max 1.5B, TimesFM-2.5 200M, PatchTST-FM 260M), particularly excelling in long-horizon settings.

Figure 4: Reverso sets a new Pareto frontier on the full Gift-Eval test set, achieving superior performance per parameter versus prior TSFMs.

LTSF/TSLib

On LTSF, Reverso outperforms Sundial (444M), Super-Linear (2.6M), Timer-XL (85M), TiRex (30M) and others, with particularly strong results on datasets with pronounced seasonality and trend. Even when competing against models with explicit in-domain data, Reverso ranks at or near the top by averaged MAE and rank metrics.

Figure 5: MAE performance on LTSF datasets as a function of parameter count, with Reverso outperforming much larger models.

Ablation Studies

• Architecture: Hybrid long convolution + DeltaNet layers cleanly outperform attention and linear attention variants, especially for model sizes $<10$M.
• Decoder Head: Attention-based decoder head boosts performance versus linear alternatives.
• Data/Augmentation: Removal of synthetic/augmentation steps degrades performance substantially; leave-one-out confirms robustness but necessity for the complete pipeline.
• Inference: Downsampling and flip-invariant inference are critical for maintaining accuracy on long/short horizon forecasts.

Implications and Theoretical Perspective

Reverso’s results concretely shift the TSFM Pareto frontier: high-fidelity, zero-shot forecasting is achieved with orders-of-magnitude smaller models. This implies the utility of hybrid architectures for sequence modeling—a finding converging with recent trends in NLP (Jamba, Zamba, empirical hybrid LLMs). It also suggests that, unlike LLMs where scaling is essential for emergent abilities, TSFMs benefit more from data diversity, sequence/transformation primitive selection, and tailored inference procedures.

Practically, Reverso democratizes TSFM deployment in resource-constrained environments (edge, real-time, cloud), drastically reduces training/serving costs, and opens the path to compositional/fine-grained forecasting applications. Theoretically, it prompts investigations into the universal approximation capacity of hybrid conv-SSM architectures, and their expressiveness in multivariate/multi-scale TS domains.

Limitations and Future Directions

Reverso is currently tailored to univariate TS; multivariate adaptation is possible but not investigated here. While long-sequence performance is state-of-the-art, short-sequence results lag behind some large TSFMs. Integration with uncertainty quantification (conformal prediction) or probabilistic forecast objectives remains an open avenue. Additional research into adaptive context management, state-weaving, and cross-dimensional attention for multivariate tasks could further enhance generalization.

Conclusion

The Reverso family establishes that with careful architectural design, data pipeline engineering, and inference strategy, highly efficient TSFMs can match or exceed the performance of much larger models. This shifts the scaling paradigm in time series forecasting and underlines the primacy of hybrid sequence modeling for practical zero-shot forecasting.