On Determining and Qualifying the Number of Superstates in Aggregation of Markov Chains

Abstract: Many studies involving large Markov chains require determining a smaller representative (aggregated) chains. Each {\em superstate} in the representative chain represents a {\em group of related} states in the original Markov chain. Typically, the choice of number of superstates in the aggregated chain is ambiguous, and based on the limited prior know-how. In this paper we present a structured methodology of determining the best candidate for the number of superstates. We achieve this by comparing aggregated chains of different sizes. To facilitate this comparison we develop and quantify a notion of {\em marginal return}. Our notion captures the decrease in the {\em heterogeneity} within the group of the {\em related} states (i.e., states represented by the same superstate) upon a unit increase in the number of superstates in the aggregated chain. We use Maximum Entropy Principle to justify the notion of marginal return, as well as our quantification of heterogeneity. Through simulations on synthetic Markov chains, where the number of superstates are known apriori, we show that the aggregated chain with the largest marginal return identifies this number. In case of Markov chains that model real-life scenarios we show that the aggregated model with the largest marginal return identifies an inherent structure unique to the scenario being modelled; thus, substantiating on the efficacy of our proposed methodology.