Informed Policies in Control Systems

• Informed policies are decision-making frameworks that incorporate future preview information about system errors to adjust control actions dynamically.
• They leverage an affine-in-error structure and fixed-point formulations, ensuring tractable, unique solutions via Banach’s theorem and convexity assumptions.
• Empirical results demonstrate reduced conservatism with significantly improved performance in both affine and nonlinear systems under safety-critical conditions.

Informed policies are a class of decision-making frameworks wherein the policy function explicitly incorporates preview information about the future evolution or error dynamics of a system, beyond immediate state observations. Originating from advances in control theory and reinforcement learning, informed policies use predictive models, over-approximation errors, or other forms of lookahead as “input-dependent” auxiliary variables, enabling the policy to act with less conservatism and increased adaptability—particularly in nonlinear, constrained, or safety-critical settings.

1. Mathematical Formulation and General Structure

Informed policies are fundamentally characterized by their explicit dependence on both state and auxiliary preview information. Consider a discrete-time nonlinear dynamical system: $$x_{t+1} = f(x_t, u_t),\quad (x_t, u_t) \in \mathcal{X} \times \mathcal{U}$$ where $x_t \in \mathbb{R}^{n_x}$, $u_t \in \mathbb{R}^{n_u}$, and $f$ is continuous.

To facilitate tractable synthesis, a simplified "over-approximation" model is constructed: $$\hat{f}(x, u) := \hat{f}(x, u) + \bar{e},\quad \bar{e} \in \mathcal{E}$$ Here $\hat{f}(x, u)$ is typically obtained via linearization, hybridization, or other model reduction techniques. The over-approximation error

$e(x, u) := f(x, u) - \hat{f}(x, u)$

is guaranteed to lie within a convex set $\mathcal{E}$ for all admissible $(x, u)$.

An informed policy is then structured as

$$\pi : \mathcal{X} \times \mathcal{E} \to \mathcal{U}, \quad u_k = \pi(x_k, e_k)$$

where $e_k = e(x_k, u_k)$ is not simply treated as a disturbance but as "preview information": a deterministic, input-dependent correction available to the policy at planning time (Aspeel et al., 5 Nov 2025).

Commonly, $\pi$ is chosen to be affine-in-error: $\pi(x, e) = \pi_x(x) + \pi_e(x) e$ with $\pi_x: \mathcal{X} \to \mathbb{R}^{n_u}$, $\pi_e: \mathcal{X} \to \mathbb{R}^{n_u \times n_x}$, ensuring computational tractability and facilitating fixed-point arguments.

A Lipschitz continuity condition is imposed for theoretical guarantees: $$\|\pi(x, e_1) - \pi(x, e_2)\| \leq L_\pi(x) \|e_1 - e_2\|, \quad \forall x, e_1, e_2$$ with further contraction requirements, such as $L_\pi(x) L_e(x) < 1$ (where $L_e(x)$ quantifies the local sensitivity of the error to the input), to admit unique, efficiently computable solutions via Banach’s theorem.

2. Concretization via Fixed-Point Formulation

At deployment, the system is in state $x$ and the informed policy must produce a valid input $u$ that is consistent with both the policy’s structure and the system dynamics. This is formalized as a fixed-point equation—termed concretization—rather than a direct policy evaluation: $u = \pi(x, e(x, u)) \equiv: \mathcal{F}_x(u)$ The solution $u \in \mathcal{U}$ is a fixed point of $\mathcal{F}_x$. Existence is guaranteed under compactness and continuity assumptions by Brouwer’s fixed-point theorem:

• If $\mathcal{U}$ is non-empty, compact, and convex;
• If $f, \hat{f}, \pi$ are continuous in $u$;
• Then, $\mathcal{F}_x$ is continuous, and a solution to $u = \mathcal{F}_x(u)$ exists for all $x \in \mathcal{X}$ (Aspeel et al., 5 Nov 2025).

For input-affine systems,

$$f(x, u) = f_x(x) + f_u(x) u,\quad \hat{f}(x, u) = \hat{f}_x(x) + \hat{f}_u(x) u$$

and an error-affine policy, the fixed-point equation reduces to a linear constraint in $u$: $$u - \pi_e(x) [\Delta f_u(x) u] = \pi_x(x) + \pi_e(x) \Delta f_x(x)$$ Hence, explicit or convex programming solutions are available. In the nonlinear case, iterative methods (e.g., Banach contraction-mapping iteration) are invoked, given sufficiently small product Lipschitz constants $L_\pi(x) L_{f-\hat{f}}(x) < 1$.

3. Computational and Theoretical Properties

Distinct strategies are adopted for different system structures:

• Affine systems: If the discrepancy in the control matrix, $\Delta f_u(x)$, vanishes, a unique closed-form solution for $u$ is directly available. Otherwise, the problem reduces to a small-scale convex or linear program constrained to $\mathcal{U}$.
• General nonlinear systems: Banach’s theorem ensures a unique fixed point, with convergence achieved via simple fixed-point iteration:

$$u^{(k+1)} = \pi(x, f(x, u^{(k)}) - \hat{f}(x, u^{(k)}))$$

Convergence is linear with error controlled as

$$\|u^{(k)} - u^*\| \leq \frac{q^k}{1-q} \|u^{(1)} - u^{(0)}\|$$

provided the contraction modulus $q = L_\pi(x) L_{f-\hat{f}}(x) < 1$.

The approach’s tractability hinges on the error set $\mathcal{E}$ and admissible input set $\mathcal{U}$ being convex; nonconvexity requires different topological or multi-valued fixed-point techniques. If the over-approximation error is highly sensitive ($L_{f-\hat{f}}$ large), the required contraction may not hold, limiting applicability.

4. Practical Impact and Reduction of Conservatism

By directly exploiting the over-approximation error as a deterministic, input-coupled signal (rather than treating it adversarially), informed policies exhibit reduced conservatism in admissible control choices. In the input-affine case, runtime cost is negligible; in the nonlinear regime, iterative concretization remains efficient due to mild small-gain-type contraction conditions.

Quantitatively, informed policies have demonstrated significant performance improvements:

• For an affine dynamical system, informed policies yield $x_{1,T} \geq 0.81$ at horizon $T$, compared to $x_{1,T} \geq 0.62$ under the best uninformed (disturbance-robust) policy.
• In a nonlinear experiment with a nontrivial trigonometric nonlinearity, enforcing $L_\pi < 1/(L_f + L_{\hat{f}})$ via SLS synthesis and the Banach fixed-point iteration produced $x_{1,T} \geq 3.10$ for informed policies against 2.70 for uninformed ones (Aspeel et al., 5 Nov 2025).

5. Related Methodological Context

Informed policies should be distinguished from purely disturbance-robust or min-max policies. The core innovation is the explicit, deterministic use of preview information about system discrepancies, synthesized as part of the policy input and coupling to the selection of control actions through fixed-point concretization. This extends the reach of preview-based, lookahead, and MPC-style methods to settings where preview information is not externally sensed but internally constructed via over-approximation error modeling.

Relevant methodological connections include:

• Policies augmented with explicit preview steps in RL/planning, where imagined rollouts or value-of-information estimates supplement myopic action selection (e.g., in ProSpec RL (Liu et al., 2024) or preview-based Q-learning (Mazouchi et al., 2021)).
• The use of auxiliary models (e.g., linearizations, hybridizations) to facilitate real-time implementability while still enabling non-conservative control.

6. Limitations, Extensions, and Open Directions

Current limitations arise mainly from the sensitivity of the over-approximation error to inputs and potential nonconvex domains for error or input sets, which break fixed-point existence/uniqueness criteria or computational tractability. Extending informed policies to these regimes requires advanced tools from non-constructive fixed-point theory or multi-valued solver approaches.

Future research may address the automatic synthesis of the affine-in-error gain structure, robustification to modeling mismatch beyond the guaranteed error sets, and the integration of probabilistic preview (e.g., distributional uncertainty) rather than deterministic over-approximation errors. Extensions to multi-step preview, learned models, or online adaptation are plausible and would further bridge the gap between learning-based and classical control-centric policy synthesis in nonlinear, constrained domains.

---

Summary Table: Key Aspects of Informed Policies

Aspect	Affine Systems	Nonlinear Systems
Policy form	$\pi(x,e) = \pi_x(x) + \pi_e(x)e$	Any continuous, Lipschitz-in-$e$ function
Concretization	Linear/convex program; explicit if $\Delta f_u = 0$	Banach contraction iteration
Existence guarantee	Brouwer fixed-point theorem	Banach fixed-point theorem (if contractive)
Runtime cost	Negligible	Low, linear convergence
Limitation	Structure/convexity in $\mathcal{U}, \mathcal{E}$	Contraction condition on error sensitivity
Quantitative gain (example)	$x_{1,T}\geq0.81$ vs $0.62$	$x_{1,T}\geq3.10$ vs $2.70$

Informed policies represent a formal, fixed-point-theoretic generalization of preview-based control for nonlinear and constrained systems, reducing conservatism and enabling efficient, real-time synthesis provided modeling error can be tightly characterized and computational prerequisites are met.