Approximately Optimal Monitoring of Plan Preconditions

Abstract: Monitoring plan preconditions can allow for replanning when a precondition fails, generally far in advance of the point in the plan where the precondition is relevant. However, monitoring is generally costly, and some precondition failures have a very small impact on plan quality. We formulate a model for optimal precondition monitoring, using partially-observable Markov decisions processes, and describe methods for solving this model efficitively, though approximately. Specifically, we show that the single-precondition monitoring problem is generally tractable, and the multiple-precondition monitoring policies can be efficitively approximated using single-precondition soultions.

• The paper’s main contribution is a decision-theoretic model for plan precondition monitoring, framing it as a sequential decision-making problem using POMDPs.
• It introduces two heuristics, NPC and VAPC, with VAPC offering up to 28.5% performance improvement while significantly reducing computational effort.
• Empirical results show that the approximate methods achieve near-optimal policy values (within 5% error) and scale efficiently to large, real-world plans.

Decision-Theoretic Approaches to Approximately Optimal Plan Monitoring

Problem Formulation and Motivation

Plan precondition monitoring is essential in dynamic and uncertain environments where the utility of classical planning degrades due to unforeseen exogenous events. Traditional execution monitoring strategies either require monitoring all plan preconditions, incurring prohibitive observation costs, or detect failures only reactively at execution time, yielding suboptimal adaptation and increased plan repair costs. Neither approach adequately balances the computational cost, value of early information, and the (potentially significant) expense of monitoring actions themselves.

The paper provides a rigorous formalization of the precondition monitoring problem as a sequential decision-making task, leveraging the POMDP framework. This captures (i) the cost of monitoring, (ii) the probabilistic evolution (failure/repair) of preconditions due to exogenous events, (iii) the inaccuracy of information sources (sensors), and (iv) the expected value of replanning at any stage. The model assumes knowledge or accurate estimation of these parameters and focuses on fully observable deterministic planning domains, treating precondition status as independent boolean variables subject to stochastic transitions.

POMDP Formulation and Computational Challenges

The plan monitoring problem is structured as a two-stage decision process for each time step: first, determine which preconditions to monitor and update the associated belief states; second, decide whether to persist with the original plan or switch to an alternative due to perceived vulnerabilities. The POMDP state space is exponential in the number of preconditions (i.e., $2^n$ for $n$ preconditions), and the set of possible monitoring actions and observations grows accordingly. While exact dynamic programming algorithms (Monahan's, Witness, etc.) are theoretically applicable, their practical deployment is restricted to trivial problem sizes (e.g., plans with three actions), due to intractable computational and memory demands—even for highly simplified sensor and transition models.

Heuristic Decomposition and Approximate Policy Construction

The central contribution of the work is the development of tractable, decision-theoretic approximations. The original high-dimensional POMDP is decomposed into $n$ independent "single-precondition" monitoring problems under a conditional independence assumption. Each subproblem is a much smaller POMDP involving only two states (precondition holds/fails), one monitoring action per time step, and two possible observations (true/false), enabling efficient solution by standard POMDP algorithms.

Two policy combination heuristics are introduced:

• Naïve Policy Combination (NPC): At each stage, decisions to monitor are made in parallel using the current belief for each individual precondition. The execution policy is to continue the original plan only if all single-precondition policies suggest so; if any suggests abandonment, the entire plan is abandoned.
• Value-Adjusted Policy Combination (VAPC): This corrects the naivety of NPC by accounting for aggregate risk. At each step, the value functions of individual subproblems are recursively adjusted to reflect the expected impact of downstream precondition failures, yielding a more accurate global assessment for the continue/abandon decision. This adjustment is computationally trivial, requiring little extra overhead per decision point.

Both methods maintain a factored belief state over preconditions and exploit the independence assumption for efficient online policy execution.

Empirical Evaluation

Empirical analysis demonstrates that the approximation approach yields policies whose value is within 5% of optimal on three-stage problems. On a suite of 1331 belief states, average relative error for NPC was 0.049 and for VAPC 0.047, with maximum error below 0.17. The VAPC method achieves up to 28.5% improvement over NPC in the critical belief regions where continue/abandon choices are sensitive. Crucially, the computational time is reduced by orders of magnitude: solving a three-stage full POMDP requires hours, whereas the approximation takes only seconds. For large problems (up to 400 stages), approximate methods exhibit near-linear scaling in solution time, with quadratic worst-case dependence on plan size, making practical monitoring policies computable for real-world-sized plans.

Quality of approximation is belief-dependent, with errors largest in intermediate belief states. When precondition success is highly probable ($>0.9$), approximations are effectively optimal—matching the domains where classical plan monitoring is most likely to be applied in practice.

Theoretical and Practical Implications

The study formalizes optimal monitoring as a value-of-information problem, moving beyond reactive/execution-time-only monitoring paradigms in planning. By introducing tractable approximation methods compatible with classical planning pipelines, it enables scalable integration of decision-theoretic execution monitoring into complex workflows.

The independence assumption is strong—correlations between precondition failures may exist in many domains. The extension of these methods to models with correlated failures (e.g., by employing Bayesian networks for belief tracking) or partially-ordered plans is identified as a natural future direction. More granular models of observation costs and alternative plan valuation would further enhance applicability.

Formal error bounds for the approximation policies remain an open avenue. Furthermore, identifying and focusing monitoring resources on a subset of "critical point" actions—where deviation cost is highest—offers promising heuristics for further computation reduction.

Conclusion

This paper delivers an explicit decision-theoretic model for the plan precondition monitoring problem, supported by scalable approximation algorithms that make policy computation tractable for complex plans at little cost to decision quality. These developments underpin more robust, cost-sensitive execution monitoring and should prove influential in both sequential decision-making research and practical AI planning systems. Future work will likely extend these techniques to richer dependency models and further formalize the approximation guarantees.