Model-free stochastic linear quadratic design by semidefinite programming

Abstract: In this article, we study a model-free design approach for stochastic linear quadratic (SLQ) controllers. Based on the convexity of the SLQ dual problem and the Karush-Kuhn-Tucker (KKT) conditions, we find the relationship between the optimal point of the dual problem and the Q-function, which can be used to develop a novel model-free semidefinite programming (SDP) algorithm for deriving optimal control gain. This study provides a new optimization perspective for understanding Q-learning algorithms and lays a theoretical foundation for effective reinforcement learning (RL) algorithms. Finally, the effectiveness of the proposed model-free SDP algorithm is demonstrated by two case simulations.