LLM4FTS: Enhancing Large Language Models for Financial Time Series Prediction

Abstract: Predicting financial time series presents significant challenges due to inherent low signal-to-noise ratios and intricate temporal patterns. Traditional machine learning models exhibit limitations in this forecasting task constrained by their restricted model capacity. Recent advances in LLMs, with their greatly expanded parameter spaces, demonstrate promising potential for modeling complex dependencies in temporal sequences. However, existing LLM-based approaches typically focus on fixed-length patch analysis due to the Transformer architecture, ignoring market data's multi-scale pattern characteristics. In this study, we propose $LLM4FTS$, a novel framework that enhances LLM capabilities for temporal sequence modeling through learnable patch segmentation and dynamic wavelet convolution modules. Specifically,we first employ K-means++ clustering based on DTW distance to identify scale-invariant patterns in market data. Building upon pattern recognition results, we introduce adaptive patch segmentation that partitions temporal sequences while preserving maximal pattern integrity. To accommodate time-varying frequency characteristics, we devise a dynamic wavelet convolution module that emulates discrete wavelet transformation with enhanced flexibility in capturing time-frequency features. These three modules work together to improve LLM's ability to handle scale-invariant patterns in financial time series. Extensive experiments on real-world financial datasets substantiate the framework's efficacy, demonstrating superior performance in capturing complex market patterns and achieving state-of-the-art results in stock return prediction. The successful deployment in practical trading systems confirms its real-world applicability, representing a significant advancement in LLM applications for financial forecasting.

• The paper introduces LLM4FTS, which enhances LLMs with dynamic patch segmentation and wavelet convolution to capture non-stationary market patterns.
• It employs a two-stage training approach—Next-Patch Pre-Training and Multi-Resolution Fine-Tuning—to adapt to intricate temporal dependencies in financial data.
• Experiments on datasets like CSI 300 and NASDAQ 100 demonstrate improved risk-adjusted returns, with successful deployment in real-world trading systems.

"LLM4FTS: Enhancing LLMs for Financial Time Series Prediction"

Introduction

The paper presents a novel framework, LLM4FTS, designed to enhance LLMs for predicting financial time series data. Financial time series forecasting poses significant challenges due to the complex and non-stationary patterns prevalent in market data. Traditional models struggle to capture the intricate temporal dependencies and multi-scale characteristics of financial time series. The introduction of LLM4FTS seeks to address these shortcomings by incorporating dynamic patch segmentation and wavelet convolution modules, allowing LLMs to better model these complex dependencies.

Figure 1: Figures a and b suggest the differences in period patterns between financial and electricity data. Figures c and d show the deviation between the two in the frequency domain distribution, varying over time for financial data.

Problem Formulation

Financial time series prediction typically involves estimating future stock returns, given historical data with multiple indicators. The paper adopts a cross-sectional analysis approach, promising stable profit opportunities by predicting returns based on historical data with various metrics like opening price, closing price, and volume. This approach seeks to leverage indirect data relationships to model short-term predictions.

Scale-Invariant Pattern Recognition

The paper introduces a Scale-Invariant Pattern Recognition (SIPR) algorithm, using K-means++ clustering and Dynamic Time Warping (DTW) to identify scale-invariant patterns within market data. This method segments time series into varied-length subsequences, preserving pattern integrity and enabling the model to adaptively capture the multi-scale nature of financial sequences. The SIPR algorithm optimizes the segmentation process by minimizing the DTW distance, thereby maintaining the structural consistency required for accurate prediction.

Next-Patch Prediction Pre-Training

LLM4FTS employs a two-stage training approach. The initial stage uses dynamic patch partitioning to pre-train the model on temporal patterns, aligning LLM capabilities with complex market data. This stage facilitates the model's adaptation to the variable patterns of financial data while preserving semantic coherence. By dynamically determining patches based on segmentation references learned from index data, LLM4FTS ensures the capturing of significant trend information without redundancy.

Multi-Resolution Prediction Fine-tuning

In the fine-tuning stage, the paper introduces a dynamic wavelet convolution module that emulates discrete wavelet transformation, enabling multi-resolution learning of temporal sequences. The module models various frequency components dynamically, updating basis functions during training to accommodate the fluctuating time-frequency characteristics intrinsic to financial data. This adaptive learning mechanism enhances the model's predictive accuracy by preserving orthogonality and time-frequency characteristics in the data.

Figure 2: The architecture of the proposed LLM4FTS model, detailing modules for scale-invariant pattern recognition, dynamic stepping, and trainable wavelet convolution.

Experiments

The paper validates the efficacy of LLM4FTS through experiments on real-world datasets, such as CSI 300, CSI 500, S&P 500, and NASDAQ 100. The model demonstrates superior results in financial forecasting, achieving higher annualized returns and better risk-adjusted profit measures compared to state-of-the-art baselines. These experiments underscore the framework's ability to capture complex financial time series patterns effectively.

Figure 3: The accumulated returns gained in the test dataset (2024) by our proposed LLM4FTS and selected baselines.

Deployment and Real-world Application

LLM4FTS has been successfully deployed in real-world trading systems, confirming its operational efficacy in generating consistent excess returns over a year. This deployment illustrates its robustness and practical applicability in guiding investment strategies, substantiating its potential as a significant advancement in AI-driven financial forecasting.

Figure 4: Performance of strategy deployment on CSI 300 and CSI 500 Index constituents.

Conclusion

The introduction of LLM4FTS marks a significant advancement in modeling financial time series data with LLMs. Its novel integration of pattern recognition and wavelet convolution modules enables precise modeling of multi-scale patterns and temporal dependencies, overcoming traditional prediction limitations. The successful validation across multiple datasets and its deployment in real-world trading systems highlight the transformative potential of LLMs in financial prediction, paving the way for further exploration in adapting AI to complex temporal modeling tasks.