Systems of ODEs Parameters Estimation by Using Stochastic Newton-Raphson and Gradient Descent Methods

Abstract: Ordinary differential equations (ODEs) are widely used to describe the time evolution of natural phenomena across various scientific fields. Estimating the parameters of these systems from data is a challenging task, particularly when dealing with nonlinear and high-dimensional models. In this paper, we propose novel methodologies for parameter estimation in systems of ODEs by using the Newton-Raphson (NR) method and Gradient Descent (GD) method. By leveraging the discrete derivative and Taylor expansion, the problem is formulated in a way that enables the application of both methods, allowing for flexible, efficient solutions. Additionally, we extend these approaches to stochastic versions - Stochastic Newton-Raphson (SNR) and Stochastic Gradient Descent (SGD) - to handle large-scale systems with reduced computational cost. The proposed methods are evaluated by using numerical examples, including both linear and nonlinear parameter models, and compare the results to the well-known Nonlinear Least Squares (NLS) method. While NR converges rapidly to the optimal solution, GD demonstrates robustness in handling chaotic systems, though it may occasionally lead to suboptimal results. Overall, the proposed methods provide improved accuracy in parameter estimation for ODE systems, outperforming NLS in terms of error metrics such as bias, mean absolute error (MAE), mean absolute percentage error (MAPE), root mean square error (RMSE), and coefficient of determination R2. These methods offer a valuable tool for fitting ODE models, particularly in scenarios involving big data and complex dynamics.