Distributed LQR Control Strategies

Updated 18 January 2026

Distributed LQR control strategies are defined as approaches for synthesizing optimal feedback controllers for large-scale dynamical systems with local communication constraints.
They leverage techniques such as distributed Q-learning, policy iteration, and system level synthesis to ensure stability, convergence, and performance.
These strategies balance performance and communication trade-offs, achieving near-optimal outcomes with reduced computational complexity in decentralized settings.

A distributed Linear Quadratic Regulator (LQR) control strategy aims to synthesize optimal or near-optimal feedback controllers for large-scale or networked systems, where communication, computation, and information constraints prevent centralized design or implementation. In such architectures, agents or subsystems exchange local state and control information, compute local or distributed policies that respect structural or information constraints, and collectively achieve stabilization and quadratic performance objectives. This article provides a comprehensive overview of key theoretical formulations, algorithmic paradigms, and rigorous guarantees underlying distributed LQR control strategies.

1. Fundamental Problem Formulation and Structural Paradigms

Distributed LQR control considers a network of $N$ agents, each with local state $x^{(i)}_t$ and input $u^{(i)}_t$ , and system dynamics coupled by an underlying (possibly undirected or directed) interaction graph $\mathcal{G}$ capturing communication or physical interconnections. Local dynamics may be decoupled or coupled:

Decoupled: $x_{t+1}^{(i)} = A x_t^{(i)} + B u_t^{(i)}$
Coupled: $x_{t+1}^{(i)} = \sum_{j \in \mathcal{N}_i} A_{ij} x_t^{(j)} + B_i u_t^{(i)}$

The canonical infinite-horizon quadratic cost is

$J = \sum_{t=0}^\infty \sum_{i=1}^N \left( x_t^{(i)\top} Q^{(i)} x_t^{(i)} + u_t^{(i)\top} R^{(i)} u_t^{(i)} \right)$

with cross-agent or agreement penalties introduced as $\sum_{j\in\mathcal{N}_i} (x_t^{(i)} - x_t^{(j)})^\top \bar Q (x_t^{(i)} - x_t^{(j)})$ (Alemzadeh et al., 2018, Wang et al., 2020).

Distributed constraints may encode:

Structural sparsity: $K_{ij}=0$ if $j$ not in the observation or communication neighborhood of controller $i$
Quadratic Invariance (QI): The feedback subspace is QI with respect to the plant, allowing convex reformulations of the distributed LQR objective (Furieri et al., 2019)
Spatial locality: $\kappa$ -hop policies in which control at $i$ depends on $x^{(j)}$ only for $j$ in the $\kappa$ -neighborhood $N_G^\kappa[i]$ (Shin et al., 2022); or exponential decay of optimal gains with spatial distance (Olsson et al., 2024)

2. Distributed Model-Free and Learning-Based Algorithms

Model-free reinforcement learning (RL) approaches to distributed LQR include localized $Q$ -learning, policy-gradient, actor-critic, and distributed zeroth-order optimization:

Distributed $Q$ -Learning: Each agent parameterizes a local $Q$ -function as a quadratic form in its state, action, and neighbors' states. Agents perform temporal-difference updates, running independent recursive least-squares (RLS) to fit local value parameters and then extracting local feedback gains (Alemzadeh et al., 2018). For networked stochastic systems with parametric uncertainty, distributed agents seek the zero of a Bellman operator and run stochastic approximation plus neighbor consensus to achieve convergence to the global fixed point (Zhang et al., 2022).
Policy Iteration with State Tracking: Agents maintain distributed state estimates using local communication and consensus steps. They use these to estimate global value functions, update local policies via decentralized Q-learning-based policy iteration, and converge to the centralized LQR gain in the fully decoupled or appropriately connected case (Wang et al., 2020).
Distributed Policy Gradient and Actor-Critic: Agents run local or block-sparse policy gradient/actor-critic algorithms by exchanging neighbor state and action information, using local samples to estimate gradients of cost with respect to block-local policy parameters (Yan et al., 2024, Olsson et al., 2024, Furieri et al., 2019). Model-free zeroth-order distributed policy-gradient methods achieve global optimum under QI or exponential decay assumptions, with the performance gap decaying exponentially in the communication/gradient radius.
Asynchronous Heterogeneous Aggregation: In settings where agents are not identical, an asynchronous policy gradient descent is performed, where gradient updates from heterogeneous agents are aggregated with controlled staleness, yielding $\epsilon$ -near optimality up to a bias scaling with the degree of model heterogeneity (Toso et al., 2024).

3. Convex Synthesis and System Level Synthesis (SLS)

Distributed LQR synthesis leverages convex reformulations over finite or infinite-dimensional decision spaces:

System Level Synthesis (SLS): The closed-loop system is parameterized by stable transfer matrices $(\Phi_x,\Phi_u)$ mapping disturbances to states and controls, constrained by affine dynamics and locality/sparsity requirements:

$(zI - A)\Phi_x(z) - B\Phi_u(z) = I, \quad (\Phi_x,\Phi_u)\in \mathcal{RH}_\infty$

with entrywise zero constraints for $(i,j)$ not allowed by locality at each impulse step (Kjellqvist et al., 2022, Alonso et al., 2021). The LQR cost is the $\mathcal{H}_2$ -norm squared of the closed-loop transfer.

Infinite-horizon distributed LQR as direct convex program: Pointwise in the transfer domain on $(\Phi_x, \Phi_u)$ , with FIR approximations for tractability if necessary. Agent-level implementations map the optimal impulse responses to local state-space filters exchanging messages over the communication graph (Kjellqvist et al., 2022, Alonso et al., 2021, Fattahi et al., 2019).
Distributed Robust Synthesis and Dropout Robustness: With random communication dropouts, convex programs with robust stability constraints (e.g., $\|\Delta\|_{L_1} < 1$ over model error) are solved. Controller implementations can switch among precomputed local designs according to realized dropout patterns, with guaranteed closed-loop stability (Alonso et al., 2021).

4. Performance-Communication Trade-offs and Scalability

Performance loss in distributed LQR controllers is tied to the degree of decentralization and communication locality:

$\kappa$ -Localized Control: Restricting each agent's feedback to a $\kappa$ -neighborhood results in control policies whose cost is exponentially close ( $O(\delta^\kappa)$ ) to centralized optimum under stabilizability, detectability, and subexponential graph growth (Shin et al., 2022). Computational complexity per agent grows polynomially with $\kappa$ in many practical topologies.
Exponential Decay and Spatial Networks: In spatially decaying systems, truncating state-feedback to $k$ hops yields an $O(e^{-\alpha_P\,k})$ suboptimality, with the empirical gap becoming negligible for moderate $k$ (Olsson et al., 2024). This justifies near-optimality with purely local information processing.
Distributed Q-learning and computational savings: Model-free distributed Q-learning of $N$ decoupled subsystems achieves centralized-optimal performance with parallel computational savings, as agents independently solve much smaller parameter estimation problems relative to the centralized case (Alemzadeh et al., 2018).

5. Rigorous Convergence and Stability Guarantees

Convergence and stability guarantees are central in distributed LQR strategy design:

RLS/Policy Iteration: Under controllability, persistent excitation, and stabilizing initialization, distributed $Q$ -learning converges to the unique optimal Riccati feedback, with vanishing disagreement among agents due to state-difference penalties (Alemzadeh et al., 2018).
Gradient Dominance and Sample Complexity: For QI or related settings, the LQR cost objective satisfies a Polyak–Łojasiewicz (PL) inequality on compact sublevel sets, enabling distributed policy-gradient methods with polynomial sample complexity and finite time global optimality (Furieri et al., 2019).
Robust Distributed Synthesis: Structured and robust RL schemes with matrix Riccati iterations incorporate explicit stability and performance margins, with distributed least-squares evaluation and block-structured implementation, guaranteeing input-to-state stability under bounded or intermittent disturbances (Mukherjee et al., 2020).
Asynchronous and Heterogeneous Aggregation: Asynchronous gradient aggregation under bounded staleness achieves sublinear convergence to a ball around the global optimum, with bias depending on heterogeneity and noise as shown in finite-sample bounds (Toso et al., 2024).

6. Implementation, Practical Considerations, and Case Studies

Distributed LQR algorithms are validated and synthesized under various practical constraints:

Information Exchange Patterns: Strategies minimize communication load to neighbor-only messaging per time-step. Efficient message-passing protocols, local cluster updates, and structured block-coordinate schemes facilitate scalability (Duan et al., 2021, Jing et al., 2021).
Robustness to Dropouts: Dropouts are addressed either by designing a single robust controller or preparing a finite library of controllers for all possible dropout patterns, with agent-level runtime switching (Alonso et al., 2021).
Data-driven Control: Some paradigms bypass model identification entirely by reconstructing feasible local trajectories using data-based representations (e.g., Hankel matrices), with receding horizon predictive control implemented via distributed QPs or primal–dual flows (Allibhoy et al., 2020).
Empirical validation: Numerical studies, such as multi-agent UAVs or consensus protocols, demonstrate convergence, empirical performance gaps, and dramatic savings of distributed versus centralized methods (Alemzadeh et al., 2018, Duan et al., 2021, Olsson et al., 2024, Shin et al., 2022).

In summary, distributed LQR control strategies integrate structural restrictions, model-free learning, robust synthesis, and scalable optimization to efficiently stabilize and optimize large networked dynamical systems under locality, communication, or uncertainty constraints. The literature provides rigorous guarantees, explicit performance-communication trade-offs, and practical algorithms for both ideal and realistic networked environments (Alemzadeh et al., 2018, Furieri et al., 2019, Wang et al., 2020, Shin et al., 2022, Alonso et al., 2021, Toso et al., 2024, Olsson et al., 2024, Zhang et al., 2022, Mukherjee et al., 2020, Allibhoy et al., 2020, Fattahi et al., 2019, Jing et al., 2021, Chang et al., 2021, Chang et al., 2020).