Multimodal Deep Learning for Finance: Integrating and Forecasting International Stock Markets
Abstract: In today's increasingly international economy, return and volatility spillover effects across international equity markets are major macroeconomic drivers of stock dynamics. Thus, information regarding foreign markets is one of the most important factors in forecasting domestic stock prices. However, the cross-correlation between domestic and foreign markets is highly complex. Hence, it is extremely difficult to explicitly express this cross-correlation with a dynamical equation. In this study, we develop stock return prediction models that can jointly consider international markets, using multimodal deep learning. Our contributions are three-fold: (1) we visualize the transfer information between South Korea and US stock markets by using scatter plots; (2) we incorporate the information into the stock prediction models with the help of multimodal deep learning; (3) we conclusively demonstrate that the early and intermediate fusion models achieve a significant performance boost in comparison with the late fusion and single modality models. Our study indicates that jointly considering international stock markets can improve the prediction accuracy and deep neural networks are highly effective for such tasks.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Overview: What this paper is about
This paper asks a simple question: Can we predict tomorrow’s stock market in one country better if we also look at today’s market in another country? The authors focus on South Korea (KR) and the United States (US). They use a type of artificial intelligence called deep learning to combine information from both countries and test whether this improves prediction accuracy.
Goals and questions the paper tries to answer
- Does using US market data help predict the next day’s return (up or down movement) of Korea’s stock market?
- What is the best way to combine information from two markets using deep learning?
- Are certain ways of combining data (called “fusion” methods) better than others?
How they did it (methods explained simply)
Think of predicting stock movements like trying to guess the weather: the more useful places you get data from, the better your guess might be. But how you mix that data matters.
The authors used:
- Public daily stock data (from Yahoo Finance) for 2006–2017.
- Korea: KOSPI index (KO).
- US: S&P 500 (SP), NASDAQ (NA), and Dow Jones (DJ).
They built features (simple numeric summaries) from each day’s prices, like:
- How much the index moved from yesterday’s close to today’s close (total daily change).
- How much the opening price jumped compared to yesterday’s close (overnight jump).
- How high and how low the price went during the day compared to the close.
Why KR and US? Because their trading hours don’t overlap. The US market closes before the KR market opens. So US closing information can naturally influence KR’s next opening and day.
They trained deep neural networks (DNNs), which are computer models that learn patterns from data, to predict the next day’s KOSPI return. They tried three main ways to combine (fuse) KR and US information:
- Early fusion: Mix both countries’ features together at the start and feed them into one model, like blending all ingredients before you start cooking.
- Intermediate fusion: First, let one model learn patterns from KR data and another from US data. Then combine what each learned and make a final prediction, like cooking two dishes separately and then combining them.
- Late fusion: Make two separate predictions (one from KR-only and one from US-only) and average them at the end, like mixing two finished sauces.
They also compared these to single-source models (KR-only or US-only) and to simple rule-of-thumb strategies (like “if US went up today, KR will go up tomorrow”).
To keep the models from overfitting (memorizing instead of learning), they used common techniques:
- Dropout (randomly turning off parts of the model during training).
- Batch normalization (keeping values balanced during training).
- Early stopping (stopping training when validation performance stops improving).
They tested performance across three different time windows (2006–2017, 2010–2017, 2014–2017) and measured:
- Mean squared error (how far the prediction is from the actual number).
- Hit ratio (how often the model correctly predicts the direction: up or down).
What they found and why it matters
- Combining markets helps: Models that used both KR and US data did better than models that used just one market.
- Early and intermediate fusion worked best: These two methods consistently beat both single-market models and late fusion. In plain terms, models that learned the relationship between KR and US data together (rather than combining at the very end) made more accurate up/down calls.
- Directional accuracy improved: The best fusion models achieved hit ratios around 0.58–0.62 (meaning they guessed the direction correctly about 58–62% of the time), which beat simple rules like “buy and hold” or “follow US momentum” (about 0.53–0.56) and clearly beat single-market deep learning models (about 0.49).
This shows that:
- There is meaningful “spillover” from US market movements to Korea’s next day.
- Deep learning can capture complex, cross-market patterns that simple rules or single-market models miss.
What this could mean going forward
- For investors and analysts: Looking at international markets together—especially those with non-overlapping trading hours—can improve short-term predictions. This could help with timing decisions or managing risk.
- For researchers and builders: How you combine data matters. Letting models learn shared patterns early (early fusion) or after learning within each source (intermediate fusion) is better than just averaging predictions at the end.
- Next steps: The authors suggest adding more types of data (like company fundamentals or news sentiment) and using explainable AI techniques to understand why the model makes certain predictions.
In short, the study shows that smartly combining information from different markets using deep learning can make stock predictions more accurate, even when markets behave differently and data is noisy.
Collections
Sign up for free to add this paper to one or more collections.