Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs

Abstract: This paper studies the problem of forecasting general stochastic processes using a path-dependent extension of the Neural Jump ODE (NJ-ODE) framework \citep{herrera2021neural}. While NJ-ODE was the first framework to establish convergence guarantees for the prediction of irregularly observed time series, these results were limited to data stemming from It^o-diffusions with complete observations, in particular Markov processes, where all coordinates are observed simultaneously. In this work, we generalise these results to generic, possibly non-Markovian or discontinuous, stochastic processes with incomplete observations, by utilising the reconstruction properties of the signature transform. These theoretical results are supported by empirical studies, where it is shown that the path-dependent NJ-ODE outperforms the original NJ-ODE framework in the case of non-Markovian data. Moreover, we show that PD-NJ-ODE can be applied successfully to classical stochastic filtering problems and to limit order book (LOB) data.

• The paper introduces PD-NJ-ODE to forecast non-Markovian stochastic processes with incomplete observations using signature transforms.
• It enhances the Neural Jump ODE framework by incorporating recurrent jump networks and high-order truncated paths for robust estimation.
• The method demonstrates improved convergence in predicting irregular time series, with potential applications in finance, healthcare, and more.

Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs

Introduction

The research addresses the challenge of forecasting stochastic processes through a novel extension of the Neural Jump ODE (NJ-ODE) framework, named Path-Dependent Neural Jump ODE (PD-NJ-ODE). This approach generalizes previous methodologies to accommodate non-Markovian processes and incomplete observations—extending the convergent prediction capabilities beyond the scope of Itô diffusions.

Methodology

Neural Jump ODE Recap

The NJ-ODE architecture is built on RNN and neural ODE frameworks, designed for irregular time series. The NJ-ODE utilizes neural networks to model continuous latent dynamics and update mechanisms at every observation jump, ensuring optimal convergence.

Signature Transformation

The PD-NJ-ODE leverages signature transforms of paths, enhancing representation by capturing path-dependent effects through iterated integrals, thus allowing the modeling of complex stochastic behaviors not observable with NJ-ODE alone.

Path-Dependent NJ-ODE

Extending NJ-ODE, the PD-NJ-ODE introduces additional network components and inputs, like high-order truncated signatures, to account for underlying path dependencies. By incorporating a recurrent jump network structure, the PD-NJ-ODE effectively manages non-Markovian traits and performs optimally even with random and incomplete data.

Implementation

The implementation strategy for PD-NJ-ODE includes:

1. Neural Network Design:
   • Three primary networks handle dynamics (ODE), jump updates, and output mapping.
   • Signature-based inputs and recurrent structures enrich the prediction model.
2. Training and Evaluation:
   • Model training involves minimizing the defined objective function against empirical observations.
   • Monte Carlo methods are employed to approximate the objective function for large-scale datasets.
3. Computational Design:
   • The application leverages numerical solvers (e.g., Euler) for handling differential system solutions.
   • Use of existing libraries for signature transformation computations, ensuring scalability.

Practical Considerations

Applications

While primarily focusing on financial LOB data, PD-NJ-ODE can improve predictions in fields requiring time-series forecasting where incomplete or irregular observations are a reality. This includes healthcare, finance, and other domains handling high-frequency temporal data.

Challenges

• Choice of signature truncation levels significantly impacts computation and precision, requiring careful consideration in settings with high dimensionality.
• Managing computational overhead is critical—optimizing neural network complexity and leveraging GPU accelerations are practical strategies.

Conclusion

PD-NJ-ODE represents a versatile and robust methodology for handling complex stochastic processes and provides a framework for real-world prediction tasks where conventional ODE-based models may fall short. This advancement has implications not only in theoretical domains but also across practical applications that demand precision with incomplete data.

The pathway forward involves refining computational techniques and exploring further extensions of the signature-based methodologies to broader classes of stochastic processes beyond its current applications.