A Fenchel-Moreau-Rockafellar type theorem on the Kantorovich-Wasserstein space with Applications in Partially Observable Markov Decision Processes

Abstract: By using the fact that the space of all probability measures with finite support can be somehow completed in two different fashions, one generating the Arens-Eells space and another generating the Kantorovich-Wasserstein (Wasserstein-1) space, and by exploiting the duality relationship between the Arens-Eells space with the space of Lipschitz functions, we provide a dual representation of Fenchel-Moreau-Rockafellar type for proper convex functionals on Wasserstein-1. We retrieve dual transportation inequalities as a Corollary and we provide examples where the theorem can be used to easily prove dual expressions like the celebrated Donsker-Varadhan variational formula. Finally our result allows to write convex functions as the supremum over all linear functions that are generated by roots of its conjugate dual, something that we apply to the field of Partially observable Markov decision processes (POMDPs) to approximate the value function of a given POMDP by iterating level sets. This extends the method used in Smallwood 1973 for finite state spaces to the case were the state space is a Polish metric space.