Long-term Time Series Forecasting

• Long-term time series forecasting is the task of predicting hundreds or thousands of future data points from extensive, high-dimensional historical records, vital in domains like energy, finance, and meteorology.
• Methodologies such as direct mapping, evolutionary forecasting, and decomposition-based models address challenges in scalability, multi-scale dependency, and computational efficiency.
• Innovations including parameter-efficient linear models, Transformer variants, and selective state space frameworks enhance prediction accuracy, model interpretability, and overall efficiency.

Long-term Time Series Forecasting (LTSF) is the problem of predicting values far into the future—often hundreds or thousands of steps ahead—given long, high-dimensional historical time series data. This task is crucial in domains such as energy management, finance, operations, meteorology, and industrial automation, where accurate long-range predictions directly affect planning and control. LTSF is distinguished from classical short-term forecasting by severe challenges in computational efficiency, runaway forecast errors, entanglement of multiple calendar patterns, and the need to learn how information at different time scales propagates across long horizons.

1. Core Methodological Paradigms in LTSF

The LTSF landscape encompasses a diverse ecosystem of paradigms, including direct and autoregressive mapping, decomposition-based modeling, multi-scale analysis, state space frameworks, and emerging multimodal or quantum schemes.

Direct Mapping (DF) and Evolutionary Forecasting (EF):

The prevailing paradigm has been direct mapping, where a model $f_\theta$ learns a mapping from a fixed input context window $X\in\mathbb{R}^{T\times C}$ to the total horizon $Y\in\mathbb{R}^{H\times C}$ in a single forward pass:

$Y = f_\theta(X)$

However, recent work uncovers an optimization pathology: as $H$ grows, gradients from distant future targets often conflict with those from the near term, destabilizing training. The Evolutionary Forecasting (EF) paradigm instead decouples model output and evaluation horizon, sequentially generating multi-block forecasts with teacher-forcing only on the initial segment. EF provably subsumes DF and greatly improves both asymptotic stability and sample efficiency, outperforming DF on all standard multivariate LTSF benchmarks and extremes in extrapolation (Ma et al., 30 Jan 2026).

Decomposition-Based Methods:

A robust trend in LTSF is additive or multi-branch decomposition—splitting each series into interpretable components such as trend, seasonality, short-term, and residual/irregular terms, modeling each with a specialized operator, then fusing at the output. HDformer is a canonical instance, achieving best-in-class parsimony and performance by separately extrapolating long-term (moving average), seasonal (periodic attention), short-term (lagged window attention), and synchronically irregular (inter-channel conditional correlation) components, then regressing a polynomial combination of the results (Deng et al., 2024).

State Space Models (SSMs):

Recent advances leverage selective state space models (notably Mamba) that admit input-conditioned dynamics and diagonalized recurrences to scale linearly in sequence length, while still capturing both order and semantic dependencies in long, noisy histories (Ahamed et al., 2024, Weng et al., 2024). SDE introduces a simplified, disentangled dependency encoding architecture, explicitly decoupling order, semantic, and cross-variate dependencies using parallel Mamba blocks and nonlinear activation removal, enhancing both accuracy and efficiency (up to 15% relative MSE reduction vs. PatchTST and DLinear) (Weng et al., 2024).

2. Model Architectures: Linear, Transformer, State-Space, and Hybrid

Parameter-Efficient Linear Networks:

Recent work has shown that, under appropriate design, purely linear models can outperform deep feature-based networks. DiPE-Linear is one such model: a pipeline of Static Frequential Attention (spectral gain vector), Static Temporal Attention (lagwise re-weighting), and Independent Frequential Mapping (per-frequency complex modulation). This sequence achieves linear parameter scaling ($O(L)$), log-linear inference complexity, and full interpretability (component-wise analysis in both time and frequency), while being competitive or better than fully connected or transformer models on 32/40 benchmark LTSF tasks (Zhao et al., 2024).

Component	Operation Domain	Parameter Scaling	Interpretability
SFA	Frequency	$O(L)$	Spectral salience
STA	Time	$O(L)$	Lagwise importance
IFM	Frequency	$O(L)$ (complex)	Spectral-to-forecast mapping

Low-rank weight sharing facilitates multivariate scaling while maintaining specialization advantage (Zhao et al., 2024).

Transformer Variants:

Transformer models and their derivatives have become the dominant architecture in LTSF due to their power to capture both local and global dependencies. Major advances have reduced quadratic complexity to $O(L\log L)$ or below, and introduced mechanisms for decomposition (Autoformer, FEDformer), sparse attention (LogTrans, Informer), and multimodal fusion (Time-VLM, DMMV). Empirically, architectures with bi-directional joint attention, direct-mapping forecast heads, and complete feature aggregation yield the best results (Shen et al., 17 Jul 2025).

State Space and Selective Models:

The introduction of Mamba-based selective SSMs—most notably in TimeMachine and SDE—allows for adaptive, content-sensitive context extraction at multiple resolutions and views (channel-mixing, channel-independent), using only linear O(T) compute and memory. TimeMachine’s quadruple-Mamba block achieves SOTA accuracy with stable, efficient scaling in both horizon and variable count (Ahamed et al., 2024).

Multi-Expert and Mixture Models:

Single-head linear or MLP models often fail on changing or multi-periodic patterns. Mixture-of-Linear-Experts (MoLE) enhances such backbones by training several specialized linear experts and a router to adaptively combine them per-window, yielding ~2% MSE improvement over plain linear models and SOTA results on 68% of PatchTST comparison tasks (Ni et al., 2023).

3. Multi-Scale, Decomposable, and Multimodal Advances

Multi-Scale and Logsparse Modeling:

Accurately forecasting over long horizons requires models that can capture relevant information at appropriate scales. Logsparse Decomposable Multiscaling (LDM) addresses the “context bottleneck” problem—where increasing input length paradoxically raises test error—by applying explicit multiscale decomposition and logsparse truncation at each scale. This separation concentrates relevant patterns, reduces overfitting, and achieves both best-in-class accuracy and memory/runtime efficiency across 8 benchmarks (Ma et al., 2024).

Dynamic Fusion and Adaptive Weighting:

MLP-centric models such as MDMixer apply parallel predictors at different granularities (trend and seasonal), fuse scale-specific predictions using adaptive weighting gates, and obtain systematic MAE reductions versus both previous MLP and Transformer baselines (Gao et al., 13 May 2025).

Multimodal and Cross-View Models:

Recent work such as DMMV leverages large vision models (e.g., ViT-based masked autoencoders) to process time series transformed into “images” reflecting periodic structure, then combines them with trend-residual numerical views via learnable gating. Adaptive decomposition and gating are essential: replacing the visual backbone or removing the numerical/visual separation sharply degrades performance (Shen et al., 29 May 2025).

4. Interpretability and Theoretical Insights

Disentanglement and Model Transparency:

Interpretability remains a major priority in high-stakes contexts. Models like DiPE-Linear allow for direct inspection of per-frequency and per-lag weights, enabling domain experts to visualize which temporal and spectral bands drive predictions. Contrast this with the distributed and often opaque parameterization in full self-attention blocks (Zhao et al., 2024).

Decomposition as Anti-Inflation:

HDformer demonstrates that a flexible yet lightweight decomposition (trend, seasonality, short-term, inter-channel irregularity) can achieve 99% parameter reduction vs. standard Transformer models without loss of forecast accuracy (Deng et al., 2024). The empirical observation is that, once major components are separated, residuals have near-zero autocorrelation and cross-correlation, making further modeling highly parameter-efficient.

Theoretical Underpinnings in LTSF:

• EF resolves the gradient conflict and “distal hijacking” present in direct mapping by decoupling block training and inference, with formal results demonstrating strictly improved optimization stability (Ma et al., 30 Jan 2026).
• SDE provides a mutual information and entropy-theoretic justification for disentangled time/variate encoding, showing parallel decoupling is at least as informative as sequential encoding (Weng et al., 2024).
• Attraos grounds LTSF in chaos theory by reconstructing phase space embeddings, projecting on multi-scale polynomial bases, and evolving memory in the frequency domain, yielding formal error bounds (approximation and attractor-evolution) and SOTA accuracy on both standard and chaotic synthetic datasets (Hu et al., 2024).

5. Empirical Benchmarks and Performance Landscape

Benchmarking in LTSF is typically conducted on multivariate datasets spanning energy (ETT, Electricity), weather (Weather), finance (Exchange), epidemiology (ILI), and high-dimensional sensor networks (Traffic, ECL), with standard horizons up to $H=720$. Evaluation metrics are mean squared error (MSE) and mean absolute error (MAE), with secondary statistics like MAE improvement relative to baselines.

Model/Class	Parameter Complexity	Multivariate Scaling	Best-in-class tasks (MSE)	Interpretability
DiPE-Linear (Zhao et al., 2024)	$O(L)$ (few 1000s)	Low-rank channel sharing	32/40 (public)	Full, disentangled
HDformer (Deng et al., 2024)	$O(10^5)$	Polynomial fusion	17/21	Component-based
TimeMachine (Ahamed et al., 2024)	$O(T)$ (linear)	Quadruple-Mamba	SOTA or best on 7 sets	SSM, via kernels
SDE (Weng et al., 2024)	Linear-log scaling	Disentangled encoding	Outperforms PatchTST etc.	Moderate
DMMV (Shen et al., 29 May 2025)	Moderate	Multi-modal integration	Wins on 6/8 datasets	Partial
CLMFormer (Li et al., 2022)	Transformer-scale	Curriculum + Memory	Up to 30% reduction	Implicit

LTSF models are increasingly evaluated not only on accuracy but also on compute/memory footprint, robustness in data-scarce settings, and data-efficiency under partial or corrupted data. For example, DiPE-Linear surpasses both linear and nonlinear baselines using only 4–17K parameters, confirming its target design criteria (Zhao et al., 2024).

6. Open Problems, Limitations, and Future Research

Several universal challenges persist:

• Data Scarcity and Nonstationarity: Robustness to distribution shift and concept drift remains an open area, with some models (e.g., DiPE-Linear, SDE) explicitly reporting performance under scarce training regimes.
• Scalability: Even models with linear or log-linear complexity may face challenges in ultra-high-dimensional settings (e.g., $>10^3$ channels), motivating further improvements in channel dependency management and parameter sharing (Gao et al., 13 May 2025, Weng et al., 2024).
• Multimodal and Cross-Domain Integration: Recent trends integrate vision or LLMs to exploit latent structure, but multimodal LTSF remains underexplored at scale (Shen et al., 29 May 2025).
• Interpretability and Model Trust: Although decomposable and frequency-domain models allow diagnostic access, many Transformer-based approaches remain black boxes. Explicit disentanglement and visualization tools will likely become increasingly important.
• Adaptive and Self-Supervised Methods: Adaptive scale selection, cross-scale attention, and pre-training protocols are open directions (Ma et al., 2024, Ahamed et al., 2024).
• Theoretical Foundations: Tighter error bounds, generalization guarantees under nonstationary sampling, and optimization landscape analysis (as initiated in EF) will be foundational for future advances (Ma et al., 30 Jan 2026).

In summary, LTSF sits at the intersection of efficient modeling, interpretable decomposition, multiscale dependency capture, and scalable architectures. The field continues to shift toward hybrid, sparse, and decomposable approaches that balance computational feasibility with scientific insight, setting the stage for further innovations robust to the increasingly complex and high-dimensional forecasting demands of modern industry and society.