Evaluation of the Rate of Convergence in the PIA

Abstract: Folklore says that Howard's Policy Improvement Algorithm converges extraordinarily fast, even for controlled diffusion settings. In a previous paper, we proved that approximations of the solution of a particular parabolic partial differential equation obtained via the policy improvement algorithm show a quadratic local convergence. In this paper, we show that we obtain the same rate of convergence of the algorithm in a more general setup. This provides some explanation as to why the algorithm converges fast. We provide an example by solving a semilinear elliptic partial differential equation numerically by applying the algorithm and check how the approximations converge to the analytic solution.