Horizon-Weighted Objectives

• Horizon-weighted objectives are optimization formulations that modulate the impact of system states by their temporal distance, balancing short-term performance with long-term outcomes.
• They integrate methods such as finite-horizon reachability, discounted cost functionals, and geometric weighting to regularize and guide policy synthesis.
• Applications span robotics, shape optimization, geometric analysis, and stochastic control, leveraging time-dependence for improved reliability and efficiency.

Horizon-weighted objectives comprise a class of optimization formulations and measures where the contribution of system states, actions, or outcomes is explicitly modulated by their temporal distance to a designated planning or evaluation horizon. Such objectives arise in deterministic and stochastic optimal control, reinforcement learning, control under uncertainty, geometric analysis of spacetime horizons, and algorithmic synthesis under quantitative automata. Approaches to horizon weighting include explicit finite-horizon criteria, infinite-horizon formulations with exponential discount, ratio- and average-type functionals, and variational principles in geometric settings. Theoretical and algorithmic developments leverage the horizon-weighted structure to capture trade-offs between immediacy and long-term reliability, regularize objective landscapes, or encode physical invariants associated with causal boundaries.

1. Formulations of Horizon-Weighted Objectives

Horizon-weighted objectives are instantiated through explicit dependency on time horizon in the cost or value functionals.

• Finite-horizon cumulative reachability: In goal-conditioned MDPs, a cumulative accessibility function $C^\pi(s,a,g,h)$ denotes the probability, under a policy $\pi$, of reaching goal $g$ from state $s$, taking action $a$, within a finite horizon $h$ (Naderian et al., 2020):

$$C^\pi(s,a,g,h) = \Pr_{\substack{a_0=a,\,s_0=s,\,a_t\sim\pi}} \left[ \max_{0\le t\le h} G(s_t,g) = 1 \right]$$

where $G(s,g)$ is a binary goal indicator.

• Weighted averages and ratios: In deterministic planar control problems, the infinite-horizon weighted average

$$v^* = \limsup_{T\to\infty} \frac{\int_0^T p(x(t),u(t)) \, dt}{\int_0^T q(x(t),u(t)) \, dt}$$

generalizes to formulations involving cost, reward, and constraints, modulated by time-weighting and occupational measures (Bright, 2013).

• Discounted cost functionals: In stochastic control and synthesis problems, objectives often take the discounted-sum form, e.g.,

$$J(u) = \mathbb{E}\left[\int_0^\infty e^{-\rho t} L(t, X^u(t), u(t))\, dt \right]$$

where $\rho>0$ controls the effective horizon (Hamaguchi, 2021, Filiot et al., 2021).

• Geometric horizon-weighted functionals: In Lorentzian geometry, horizon-weighted objectives appear as integrals over null surfaces, weighted by scalar fields $f$ and involving weighted curvatures, such as

$$\mathcal{A}_f(\Sigma) = \int_\Sigma e^{-f} dA_g\,, \quad \mathcal{K}^N_f(\Sigma) = \int_\Sigma R^N_f(\Sigma) e^{-f} dA_g$$

where $\Sigma$ is a horizon cross-section and $R^N_f$ is the weighted scalar curvature (Ling et al., 30 Oct 2025).

2. Theoretical Properties and Recurrence Relations

Horizon-weighted objectives introduce temporal structure and admit monotonicity and optimality properties absent in time-agnostic formulations.

• Bellman recursions for finite-horizon reachability: The optimal cumulative accessibility $C^*(s,a,g,h)$ satisfies a horizon-recursive Bellman equation:

$$C^*(s,a,g,h) = \begin{cases} G(s,g) & G(s,g)=1\ \text{or}\ h=0 \ \mathbb{E}_{s'\sim p(\cdot|s,a)} [\max_{a'} C^*(s', a', g, h-1)] & \text{otherwise} \end{cases}$$

This structure enables offline learning with unbiased cross-entropy loss (Naderian et al., 2020).

• Monotonicity in horizon: For cumulative accessibility, $h \mapsto C^*(s,a,g,h)$ is non-decreasing; additional time never reduces success probability (Naderian et al., 2020).
• Poincaré–Bendixson–type minimizer structure: Infinite-horizon weighted average problems in the plane have optima attained at stationary or periodic (Jordan curve) solutions; under $m$ constraints, minimizers are convex combinations of at most $m+1$ extremal arcs (Bright, 2013).
• Weighted geometric obstructions and rigidity: In spacetimes, horizon-weighted functionals are constrained by weighted singularity and topology theorems, which ensure non-trivial lower bounds or topological restrictions depending on weighted Ricci and scalar curvature terms (Ling et al., 30 Oct 2025).

3. Algorithmic Approaches and Computational Methods

Implementation of horizon-weighted objectives varies by domain but generally exploits the horizon structure to guide policy search, synthesis, or optimization.

• C-learning for multi-goal RL: Parameterization of $C_\theta(s,a,g,h)$ with neural networks, trained by cross-entropy loss using Bellman one-step Monte Carlo targets. Allows dynamic selection of horizon $h$ at test time to balance speed and reliability (Naderian et al., 2020).
• Occupational measures and convexity arguments: Infinite-horizon weighted control leverages limiting occupational measures and convexity under uniform controllability to realize optimal trajectories as simple (stationary or periodic) or cyclically concatenated arcs under constraints (Bright, 2013).
• Discounted dynamic programming and synthesis: Threshold, best-value, and $\varepsilon$-approximate discounted-sum synthesis reduce to value iteration or policy iteration procedures over explicit weighted arenas (critical-prefix games), with pseudo-polynomial or polynomial complexity (Filiot et al., 2021).
• Infinite-horizon BSVIEs and adjoint equations: Discounted infinite-horizon control for SVIEs/FSDEs is handled with BSVIEs in weighted $L^2$-spaces; optimality follows from Pontryagin's maximum principle adapted to the horizon-weighted context (Hamaguchi, 2021).

4. Applications and Exemplary Problems

Horizon-weighted objectives are instrumental in a diverse set of applications:

• Robotics and control: Multi-goal RL tasks (FrozenLake, Dubins’ Car, FetchPickAndPlace, HandManipulatePen) leverage $C$-learning for sample-efficient planning, adaptive path-speed trade-offs, and reachability estimation conditioned on planning horizon (Naderian et al., 2020).
• Shape optimization: The planar Cheeger set problem becomes an infinite-horizon weighted-average control task, with the area-to-perimeter ratio maximized over periodic Jordan curve solutions (Bright, 2013).
• Phase separation: The singular limit of the one-dimensional Cahn–Hilliard energy, under mass constraints, reduces to a constrained infinite-horizon problem with the minimizing occupational measure as a convex combination of two stationary states, recovering two-phase separation (Bright, 2013).
• Geometric analysis of spacetimes: Minimization or extremization of horizon-weighted area and curvature functionals encode quasi-local invariants of black hole and trapping horizons, fundamental to singularity and topology theorems in general relativity (Ling et al., 30 Oct 2025).
• Reactive synthesis: For terminating transducers subject to discounted-sum objectives, synthesis for threshold, best-value, or approximate optimality is algorithmically characterized by discounted-sum games with horizon-weighted payoffs, modulo partial domain constraints (Filiot et al., 2021).
• Stochastic control with delay/memory: Discounted infinite-horizon stochastic control is formulated for fractional SDEs and stochastic integro-differential equations; adjoint BSVIEs naturally inherit horizon-weighted kernels (Hamaguchi, 2021).

5. Impact of Horizon Weighting on Solution Structure

Explicit horizon dependence introduces qualitative effects on optimal strategies, regularity, and tractability:

• Speed-reliability trade-off: In settings such as C-learning, the explicit horizon parameter interpolates between risky, fast policies and reliable, slower strategies, yielding adaptive behaviors not accessible to time-invariant or discount-based criteria (Naderian et al., 2020).
• Regularization and tractability: Horizon weighting via discounting (e.g., $\lambda$-discounted sum with $0<\lambda<1$) ensures convergence, enables memoryless optimal strategies, and regularizes synthesis and game-solving complexity to NP $\cap$ coNP or polynomial classes, in contrast with undecidability for undiscounted sum objectives (Filiot et al., 2021).
• Monotonicity constraints: Horizon-weighted functionals often obey monotonicity in horizon or discount parameter, constraining the value landscape and ensuring that longer effective control windows cannot decrease achievable objectives (Naderian et al., 2020).
• Extremal solution representation: Infinite-horizon weighted problems with constraints admit solutions constructed by cycling among a finite set of extremal (stationary/periodic) primitives, mirroring Carathéodory-type theorems for convex hulls in measure space (Bright, 2013).

6. Geometric and Physical Interpretations

Within geometric analysis, horizon-weighted objectives capture the interplay of local curvature, quasi-local invariants, and global constraints:

• Weighted area and curvature as horizon invariants: Functionals such as $\mathcal{A}_f(\Sigma)$ and $\mathcal{K}^N_f(\Sigma)$ encode the effect of matter or dilaton fields on the effective geometry of marginally trapped surfaces and horizons, serving as targets for variational and minimization procedures (Ling et al., 30 Oct 2025).
• Penrose-type inequalities: Weighted-area lower bounds relate geometric measurements at the horizon to asymptotic charges, mediated by horizon weighting; failure of these bounds signals curvature singularities or topological obstructions (Ling et al., 30 Oct 2025).
• Rigidity and stability: Weighted analogs of classical theorems (Penrose, Hawking, Galloway–Schoen) dictate that horizon-weighted invariants admit only constrained variation within prescribed energy and curvature conditions.

7. Decidability, Complexity, and Contrasts with Untimed Objectives

The nature and hardness of horizon-weighted objectives often contrast sharply with their undiscounted or unweighted counterparts.

• Discounting ensures decidability: Strict exponential discount ($\lambda<1$) in automata-theoretic synthesis regularizes otherwise undecidable sum problems, yielding value and threshold problems in NP $\cap$ coNP with associated pseudo-polynomial-time algorithms (Filiot et al., 2021).
• Partial domains and dead states: In partial-domain synthesis, the presence of dead states (with value $-\infty$) induces adversarial stopping in prefix games, underlining the delicate balance between horizon-weighted regularity and structural intractability (Filiot et al., 2021).
• Long-run average versus discounted sum: While both are horizon-weighted, discounted sum games admit memoryless determinacy and more favorable mathematical structure compared to long-run average games, which typically require region-graph or mean-payoff analysis at higher computational cost (Filiot et al., 2021).

---

References:

• (Naderian et al., 2020): C-Learning: Horizon-Aware Cumulative Accessibility Estimation
• (Ling et al., 30 Oct 2025): The Penrose singularity theorem, MOTS stability, and horizon topology in weighted spacetimes
• (Bright, 2013): Planar Infinite-Horizon Optimal Control Problems with Weighted Average Cost and Constraints, Applied to Cheeger Sets
• (Hamaguchi, 2021): Infinite horizon backward stochastic Volterra integral equations and discounted control problems
• (Filiot et al., 2021): Synthesis from Weighted Specifications with Partial Domains over Finite Words