Stationary and Non-Stationary Transition Probabilities in Decision Making: Modeling COVID-19 Dynamics

Abstract: This study explores the complexities of stationary and non-stationary transition probabilities within the framework of a Markov Decision Process (MDP), specifically applied to COVID-19. The research highlights the critical role these probabilities play in accurately modeling disease dynamics and informing evidence-based decisions by policymakers and public health authorities. By incorporating both stationary transition probabilities (which assume constant rates of state changes) and non-stationary transition probabilities (which adapt to evolving conditions), the findings are pivotal for offering practical insights into optimizing resource allocation and intervention strategies to mitigate the pandemic's impacts. The structured analysis within this paper includes a detailed model description, derivation of balanced systems, and formulation of transition probabilities, all contextualized within a COVID-19 scenario. These contributions are invaluable for enhancing the efficacy of pandemic response strategies, ultimately improving public health outcomes and economic efficiency.