Dynamical System Identification, Model Selection and Model Uncertainty Quantification by Bayesian Inference

Abstract: This study presents a Bayesian maximum \textit{a~posteriori} (MAP) framework for dynamical system identification from time-series data. This is shown to be equivalent to a generalized Tikhonov regularization, providing a rational justification for the choice of the residual and regularization terms, respectively, from the negative logarithms of the likelihood and prior distributions. In addition to the estimation of model coefficients, the Bayesian interpretation gives access to the full apparatus for Bayesian inference, including the ranking of models, the quantification of model uncertainties and the estimation of unknown (nuisance) hyperparameters. Two Bayesian algorithms, joint maximum \textit{a~posteriori} (JMAP) and variational Bayesian approximation (VBA), are compared to the {LASSO, ridge regression and SINDy algorithms for sparse} regression, by application to several dynamical systems with added {Gaussian or Laplace} noise. For multivariate Gaussian likelihood and prior distributions, the Bayesian formulation gives Gaussian posterior and evidence distributions, in which the numerator terms can be expressed in terms of the Mahalanobis distance or ``Gaussian norm'' $||\vec{y}-\hat{\vec{y}}||^2_{M^{-1}} = (\vec{y}-\hat{\vec{y}})^\top {M^{-1}} (\vec{y}-\hat{\vec{y}})$, where $\vec{y}$ is a vector variable, $\hat{\vec{y}}$ is its estimator and $M$ is the covariance matrix. The posterior Gaussian norm is shown to provide a robust metric for quantitative model selection {for the different systems and noise models examined.}