Enhanced Monte Carlo Estimation of the Fisher Information Matrix with Independent Perturbations for Complex Problems

Abstract: The Fisher information matrix provides a way to measure the amount of information given observed data based on parameters of interest. Many applications of the FIM exist in statistical modeling, system identification, and parameter estimation. We sometimes use the Monte Carlo-based method to estimate the FIM because its analytical form is often impossible or difficult to be computed in real-world models. In this paper, we review the basic method based on simultaneous perturbations and present an enhanced resampling-based method with independent simultaneous perturbations to estimate the Fisher information matrix. We conduct theoretical and numerical analysis to show its accuracy via variance reduction from $O(1/N)$ to $O(1/(nN))$, where $n$ is the sample size of the data and $N$ is a measure of the Monte Carlo averaging. We also consider the trade-off between accuracy and computational cost.