Reinforcement learning for traversing chemical structure space: Optimizing transition states and minimum energy paths of molecules

Abstract: In recent years, deep learning has made remarkable strides, surpassing human capabilities in tasks like strategy games, and it has found applications in complex domains, including protein folding. In the realm of quantum chemistry, machine learning methods have primarily served as predictive tools or design aids using generative models, while reinforcement learning remains in its early stages of exploration. This work introduces an actor-critic reinforcement learning framework suitable for diverse optimization tasks, such as searching for molecular structures with specific properties within conformational spaces. As an example, we show an implementation of this scheme for calculating minimum energy pathways of a Claisen rearrangement reaction and a number of SN2 reactions. Our results show that the algorithm is able to accurately predict minimum energy pathways and thus, transition states, therefore providing the first steps in using actor-critic methods to study chemical reactions.