Limit formulas for the normalized fundamental matrix of the northwest-corner truncation of Markov chains: Matrix-infinite-product-form solutions of block-Hessenberg Markov chains

Abstract: This paper considers the normalized fundamental matrix for the northwest-corner (NW-corner) truncation of ergodic continuous-time Markov chains, technically, of their infinitesimal generators. We first present a limit formula for the normalized fundamental matrix of the NW-corner truncation of the ergodic (infinitesimal) generator. The limit formula shows that, as the order (size) of the NW-corner truncation diverges to infinity, the corresponding normalized fundamental matrix converges to a stochastic matrix whose rows are all equal to the stationary distribution vector of the ergodic generator. Using the limit formula, we also derive the matrix-infinite-product form (MIP-form) solutions of the stationary distribution vectors of upper and lower block-Hessenberg Markov chains. In addition, from the MIP-form solutions, we develop numerically stable and easily implementable algorithms that generate the sequences of probability vectors convergent to the corresponding stationary distribution vectors of block-Hessenberg Markov chains.