Probabilistically safe and efficient model-based Reinforcement Learning

Abstract: This paper proposes tackling safety-critical stochastic Reinforcement Learning (RL) tasks with a samplebased, model-based approach. At the core of the method lies a Model Predictive Control (MPC) scheme that acts as function approximation, providing a model-based predictive control policy. To ensure safety, a probabilistic Control Barrier Function (CBF) is integrated into the MPC controller. A sample-based approach with guarantees is employed to approximate the effects of stochasticies in the optimal control formulation and to guarantee the probabilistic CBF condition. A learnable terminal cost formulation is included in the MPC objective to counterbalance the additional computational burden due to sampling. An RL algorithm is deployed to learn both the terminal cost and the CBF constraint. Results from our numerical experiment on a constrained LTI problem corroborate the effectiveness of the proposed methodology in reducing computation time while preserving control performance and safety.