Demonstration of Enhanced Monte Carlo Computation of the Fisher Information for Complex Problems

Abstract: The Fisher information matrix summarizes the amount of information in a set of data relative to the quantities of interest. There are many applications of the information matrix in statistical modeling, system identification and parameter estimation. This short paper reviews a feedback-based method and an independent perturbation approach for computing the information matrix for complex problems, where a closed form of the information matrix is not achievable. We show through numerical examples how these methods improve the accuracy of the estimate of the information matrix compared to the basic resampling-based approach. Some relevant theory is summarized.