Implicit MEC-search for objectives requiring agent fixation

Derive an implicit maximal end-component search procedure that identifies simple end component candidates for minimizing total reward with star=c, maximizing total reward with star=∞, and long-run average reward objectives in polytopic robust Markov decision processes where the agent policy must be fixed, thereby avoiding explicit construction while preserving the correctness of the bounded value iteration approach.

Background

The authors’ implicit anytime algorithm relies on identifying simple end components (SECs) by searching for maximal end components (MECs) under a fixed policy. For the objectives they handle implicitly (SSP and maximizing TR with star=c), fixing the environment policy suffices and yields a finite-action MDP amenable to implicit MEC search.

For other objectives (minimizing TR with star=c, maximizing TR with star=∞, and LRA), fixing the agent policy would result in exponential or infinite action spaces, making implicit MEC discovery nontrivial. The authors conjecture that an implicit MEC-search algorithm for these cases exists and leave this as future work.