Linear Quadratic Regulator (LQR)

Updated 26 January 2026

Linear Quadratic Regulator (LQR) is a control method that optimally computes state-feedback gains for linear systems by minimizing a quadratic cost.
It employs Riccati equations to derive the optimal feedback gain, ensuring closed-loop stability in both continuous and discrete time.
Numerical methods like the Newton–Kleinman iteration enhance LQR's robustness and computational efficiency under model uncertainties.

The Linear Quadratic Regulator (LQR) is a cornerstone of optimal control theory, providing explicit state-feedback controllers for linear dynamical systems with quadratic performance objectives. LQR arises in both continuous and discrete time, and serves as the canonical foundation for optimal control, state estimation, robust control, and reinforcement learning for linear systems. Its solution is based on Riccati equations whose properties underpin closed-loop stability, optimality, and computational methods in modern control.

1. Mathematical Formulation and Problem Class

LQR concerns control of a linear dynamical system with quadratic cost. In continuous time, the state-space dynamics are

$\dot{x}(t) = A x(t) + B u(t), \qquad x(0) = x_0,$

where $x(t) \in \mathbb{R}^n$ is the state and $u(t) \in \mathbb{R}^m$ is the control input. For discrete time: $x_{k+1} = A x_k + B u_k, \qquad x_0 \in \mathbb{R}^n.$ The performance objective, in infinite-horizon form, is to minimize the functional

$J[u(\cdot)] = \int_0^\infty x(t)^\top Q x(t) + u(t)^\top R u(t) \,dt$

(or, discretized,

$J[u_{0:\infty}] = \sum_{k=0}^\infty x_k^\top Q x_k + u_k^\top R u_k)$

subject to the system dynamics. The weighting matrices satisfy $Q = Q^\top \succeq 0$ , $R = R^\top \succ 0$ . The fundamental assumptions for regularity of the problem are that the pair $(A,B)$ is controllable and the pair $(A,Q^{1/2})$ is detectable (Augustine, 2023, Ladnik, 21 Apr 2025).

The problem asks for the optimal state feedback policy of the form

$u^*(t) = -K x(t)$

that minimizes $J$ for every initial state $x_0$ .

2. Solution via Riccati Equations

Continuous-Time

The LQR problem is solved using the Hamiltonian optimality framework. The adjoint state (costate) $\lambda(t) \in \mathbb{R}^n$ is introduced, and one solves the optimality conditions: \begin{align*} &\dot{x} = A x + B u \ &\dot{\lambda} = -Q x - A^\top \lambda \ &0 = 2 R u + B^\top \lambda \end{align*} Pointwise minimization yields the quadratic relation $\lambda(t) = P(t) x(t)$ , where $P(t)$ solves the Riccati differential equation (RDE): $\dot{P}(t) = -A^\top P - P A + P B R^{-1} B^\top P - Q,$ with terminal boundary $P(\infty)=0$ in the infinite-horizon limit. As $t \to \infty$ , $P(t) \to P$ , where $P$ solves the algebraic Riccati equation (ARE): $A^\top P + P A - P B R^{-1} B^\top P + Q = 0$ The optimal steady-state controller is then

$u^*(t) = -K x(t), \qquad K = R^{-1} B^\top P$

(Augustine, 2023).

Discrete-Time

For the discrete-time case, dynamic programming leads to a recursion over the value function $V_k(x) = x^\top P_k x$ : $P_k = Q + A^\top P_{k+1} A - A^\top P_{k+1} B (R + B^\top P_{k+1} B)^{-1} B^\top P_{k+1} A,$ with boundary $P_N = Q_f$ . For infinite horizon, $P_k \to P$ where

$P = Q + A^\top P A - A^\top P B (R + B^\top P B)^{-1} B^\top P A$

and the optimal linear feedback is $u_k = -K x_k$ with $K = (R + B^\top P B)^{-1} B^\top P A$ (Ladnik, 21 Apr 2025).

3. Closed-Loop Stability and Performance

For the closed-loop system,

$\dot{x} = (A - B K) x$

in continuous time, or

$x_{k+1} = (A - B K) x_k$

in discrete time, solutions to the Riccati equation guarantee that $A - B K$ is Hurwitz (all eigenvalues have negative real part) in continuous time and Schur-stable (all eigenvalues in the open unit disk) in discrete time.

The ARE/Lyapunov equation guarantees that

$(A - B K)^\top P + P (A - B K) = -(Q + K^\top R K) \prec 0$

which certifies strict dissipativity and exponential convergence to the origin (Augustine, 2023).

LQR minimizes the $\mathcal{H}_2$ norm from process noise/disturbance to regulated state and can be interpreted in terms of minimizing weighted state and input norms while providing optimal energy-to-go (Augustine, 2023, Ladnik, 21 Apr 2025).

The policy is robust: small modeling errors in $(A,B)$ , $Q$ , and $R$ do not destroy closed-loop stability, and LQR controllers exhibit guaranteed gain and phase margins under such perturbations.

4. Numerical Solution and Implementation

Solving the ARE is numerically stable and is implemented in several canonical algorithms:

Schur-vector method (using real Schur decomposition of the Hamiltonian matrix)
Newton–Kleinman iteration: iteratively solves Lyapunov equations for improved Riccati solution, offering quadratic convergence.
Dedicated software: MATLAB’s care/lqr routines, SLICOT (SB02BD), and PySLICOT.

The typical algorithm for the infinite-horizon case:

Initialize $P_0 = Q$ .
At each iteration, set $K_k = R^{-1} B^\top P_k$ , then solve the Lyapunov equation for the closed-loop:

$(A - B K_k)^\top X + X (A - B K_k) = -(Q + K_k^\top R K_k)$

and set $P_{k+1} = X$ .

Iterate until convergence (Augustine, 2023).

For discrete time, the Riccati difference equation is integrated backward from $P_N = Q_f$ to $P_0$ , storing corresponding gain matrices (Ladnik, 21 Apr 2025).

5. Structural Properties, Extensions, and Theoretical Guarantees

Structural Optimality

The optimal LQR gain $K^*$ is globally optimal among all static state-feedback policies. Under standard stabilizability and detectability conditions, strong duality holds for the nonconvex control synthesis problem: the associated semidefinite program (SDP) relaxation is tight, and the Riccati ARE solution exactly recovers the global minimizer (Watanabe et al., 14 Mar 2025). The hidden convexity present in the extended convex lifting framework (ECL) precludes spurious local minima and ensures the landscape is Polyak–Łojasiewicz (PL), i.e., the cost functional is gradient dominated on all compact sublevel sets (Watanabe et al., 14 Mar 2025).

Regularization and Structured Control

LQR can be regularized to induce structure—sparsity, block sparsity, or low rank—on the controller via convex penalties, leading to regularized optimizations solved by proximal-gradient or structured policy iteration schemes (Park et al., 2020).

Robust and Risk-Constrained LQR

Extensions such as the risk-constrained LQR explicitly address rare-but-significant stochastic disturbances by imposing variance or higher-moment constraints on the state cost. The optimal feedback in these cases remains affine in state, with the Riccati recursion using an inflated state penalty and an additional linear-precompensation term that hedges directions exposed to non-Gaussian noise (Tsiamis et al., 2020).

Robust LQR under open-loop plant uncertainty can be formulated and implemented via scenario approaches, optimizing over worst-case scenarios sampled from the uncertainty set. Lyapunov-based LMI constraints are used to guarantee that the computed controller provides provable probabilistic robustness margins (Scampicchio et al., 2020).

Output Feedback and Infinite-Dimensional LQR

When only a noisy or partial observation of the state is available, the LQR solution can be extended using a Kalman filter as an observer to provide the optimal estimate-driven control law (Augustine, 2023). For boundary control of infinite-dimensional PDEs, e.g., the Euler–Bernoulli beam, LQR decomposes into a family of decoupled finite-dimensional regulator problems, each solved by its own Riccati equation (Krener, 2021).

6. Computational Complexity and Modern Optimization Viewpoints

The LQR objective is nonconvex in the static feedback gain but has no spurious local minima; all stationary points are global (Watanabe et al., 14 Mar 2025, Feng et al., 2023). First-order methods—gradient flow, natural gradient, and quasi-Newton—admit global linear (respectively, quadratic) convergence rates to the optimal feedback when initialized in the stabilizing set. The landscape is real-analytic, smooth, and coercive, and the Lyapunov functionals decay exponentially. The Kleinman–Newton method, in particular, recovers the Newton iteration for the Riccati equation (Bu et al., 2020, Bu et al., 2019).

Accelerated gradient methods and their discrete variants (e.g., Nesterov-type methods for LQR) converge at the optimal accelerated rate $O(1/\sqrt{\kappa})$ , where $\kappa$ is a condition number determined by the problem data (Feng et al., 2023).

Second-order policy gradient methods for LQR use analytic expressions for the Hessian and the Gauss–Newton approximation to accelerate policy learning, achieving quadratic convergence in certain settings (Valaei et al., 3 Nov 2025).

7. Connections to Other Areas and Applications

LQR is central to both state feedback and estimation (via the duality with the Kalman filter), and underpins classical MPC, robust control, and system identification. LQR policies are foundational in linear system reinforcement learning, serving as the reference for model-based and model-free learning methods, including those based on policy iteration, Q-learning, or adaptive dynamic programming.

LQR’s theoretical guarantees closely inform policy optimization for more general nonlinear, stochastic, or data-driven control systems, with iterative LQR (iLQR) forming the basis for trajectory optimization in nonlinear and robotic systems (Ladnik, 21 Apr 2025). The algebraic and variational perspectives further connect LQR to module-theoretic system theory, flatness, and turnpike phenomena in optimal control (Join et al., 11 Dec 2025).

References:

(Augustine, 2023, Ladnik, 21 Apr 2025, Watanabe et al., 14 Mar 2025, Tsiamis et al., 2020, Scampicchio et al., 2020, Feng et al., 2023, Join et al., 11 Dec 2025, Valaei et al., 3 Nov 2025, Bu et al., 2020, Bu et al., 2019, Park et al., 2020, Krener, 2021).