Convergence Analysis of the Upwind Difference Methods for Hamilton-Jacobi-Bellman Equations

Abstract: The convergence properties of the upwind difference scheme for the Hamilton-Jacobi-Bellman (HJB) equation, which is a fundamental equation for optimal control theory, are investigated. We first perform a convergence analysis for the solution of the scheme, which eliminates ambiguities in the proofs of existing studies. We then prove the convergence of the spatial difference of the solution in the scheme by the correspondence between the HJB equations and the conservation laws. This result leads to a property of the objective function called epi-convergence, by which the convergence property of the input function is shown. The latter two results have not been addressed in existing studies. Numerical calculations support the obtained results.