An Entropy Regularized BSDE Approach to Bermudan Options and Games

Abstract: In this paper, we investigate optimal stopping problems in a continuous-time framework where only a discrete set of stopping dates is admissible, corresponding to the Bermudan option, within the so-called exploratory formulation. We introduce an associated control problem for the value function, represented as a non-c`adl`ag reflected backward stochastic differential equation (RBSDE) with an entropy regulariser that promotes exploration, and we establish existence and uniqueness results for this entropy-regularised RBSDE. We then compare the entropy-regularised RBSDE with the theoretical value of a Bermudan option and propose a reinforcement learning algorithm based on a policy improvement scheme, for which we prove both monotone improvement and convergence. This methodology is further extended to Bermudan game options, where we obtain analogous results. Finally, drawing on the preceding analysis, we present two numerical approximation schemes - a BSDE solver based on a temporal-difference scheme and neural networks and the policy improvement algorithm - to illustrate the feasibility and effectiveness of our approach.