Approximate Inference for Observation Driven Time Series Models with Intractable Likelihoods

Abstract: In the following article we consider approximate Bayesian parameter inference for observation driven time series models. Such statistical models appear in a wide variety of applications, including econometrics and applied mathematics. This article considers the scenario where the likelihood function cannot be evaluated point-wise; in such cases, one cannot perform exact statistical inference, including parameter estimation, which often requires advanced computational algorithms, such as Markov chain Monte Carlo (MCMC). We introduce a new approximation based upon approximate Bayesian computation (ABC). Under some conditions, we show that as $n\rightarrow\infty$, with $n$ the length of the time series, the ABC posterior has, almost surely, a maximum \emph{a posteriori} (MAP) estimator of the parameters which is different from the true parameter. However, a noisy ABC MAP, which perturbs the original data, asymptotically converges to the true parameter, almost surely. In order to draw statistical inference, for the ABC approximation adopted, standard MCMC algorithms can have acceptance probabilities that fall at an exponential rate in $n$ and slightly more advanced algorithms can mix poorly. We develop a new and improved MCMC kernel, which is based upon an exact approximation of a marginal algorithm, whose cost per-iteration is random but the expected cost, for good performance, is shown to be $\mathcal{O}(n^2)$ per-iteration.