Hierarchical Decompositions of Stochastic Pursuit-Evasion Games

Abstract: In this work we present a hierarchical framework for solving discrete stochastic pursuit-evasion games (PEGs) in large grid worlds. With a partition of the grid world into superstates (e.g., "rooms"), the proposed approach creates a two-resolution decision-making process, which consists of a set of local PEGs at the original state level and an aggregated PEG at the superstate level. Having much smaller cardinality, both the local games and the aggregated game can be easily solved to a Nash equilibrium. To connect the decision-making at the two resolutions, we use the Nash values of the local PEGs as the rewards for the aggregated game. Through numerical simulations, we show that the proposed hierarchical framework significantly reduces the computation overhead, while still maintaining a satisfactory level of performance when competing against the flat Nash policies.