Approximating the diagonal of a Hessian: which sample set of points should be used

Abstract: An explicit formula to approximate the diagonal entries of the Hessian is introduced. When the derivative-free technique called \emph{generalized centered simplex gradient} is used to approximate the gradient, then the formula can be computed for only one additional function evaluation. An error bound is introduced and provides information on the form of the sample set of points that should be used to approximate the diagonal of a Hessian. If the sample set of points is built in a specific manner, it is shown that the technique is $\mathcal{O}(\Delta_S^2)$ accurate approximation of the diagonal entries of the Hessian where $\Delta_S$ is the radius of the sample set.