Budget-Aware Planning

• Budget-aware planning is a strategy that incorporates explicit resource constraints into optimization, enabling efficient performance-cost trade-offs.
• It employs methods like budget-conditioned reinforcement learning, hierarchical decision-making, and adaptive routing to manage dynamic resource allocation.
• Applications span LLM reasoning, cloud autoscaling, and robotic control, consistently achieving Pareto-optimal balances between accuracy and expenditure.

Budget-aware planning refers to a broad set of algorithmic and modeling strategies which explicitly incorporate resource constraints—such as monetary, computational, token, time, or capacity budgets—into the decision-making or optimization process of agents, planners, or systems. Rather than treating cost as an afterthought or a downstream filter, budget-aware planning integrates budget signals directly into the planning objective, data structures, learning criteria, and control policy, thereby enabling dynamic and granular trade-offs between task performance and cost throughout sequential or multi-stage workflows.

1. Fundamental Principles and Formalization

Budget-aware planning is characterized by the constrained optimization or sequential decision-making structure: $$\max_\pi\, \mathbb{E}[R(\tau)] \quad \text{subject to} \quad \mathbb{E}[C(\tau)] \le B,$$ where $\pi$ is a policy, $R(\tau)$ is the (often stochastic) task reward for a trajectory $\tau$, $C(\tau)$ is its incurred cost, and $B$ is the (hard or soft) budget. Multiple works recast this problem in Lagrangian form, optimizing $R(\tau) - \lambda C(\tau)$ with $\lambda$ as a budget-preference hyperparameter (Zhang et al., 5 Feb 2026, Zhang et al., 20 May 2025, Yang et al., 26 Nov 2025).

Budget models are domain-specific: monetary costs for cloud resources (Ilyushkin et al., 2019), token counts or API call charges for LLM pipelines (Zhang et al., 5 Feb 2026, Yang et al., 26 Nov 2025, Wen et al., 24 Aug 2025), physical resources for robotics (Cherenson et al., 3 Apr 2025), or repair/maintenance interventions in budget-constrained MDPs (Vora et al., 2024).

Budget-awareness is operationalized in several forms:

• Real-time cost tracking and signaling: Prompt-level, action-level, or system-level budget states are surfaced during execution (Liu et al., 21 Nov 2025).
• Budget-conditioned policies or reward functions: Agents are provided explicit budget signals and incentivized to plan or act adaptively (Wen et al., 24 Aug 2025, Niu et al., 3 Nov 2025, Lyu et al., 21 Jul 2025).
• Sequential/iterative planning under cumulative cost: Multi-stage planning with dynamic budget reallocation, carry-over, or adaptive horizon (Wihidayat et al., 18 Dec 2025, Belakaria et al., 2022).
• Structured planning hierarchies or group assignments to decouple budget and capacity dimensions (Vora et al., 2024).

2. Algorithmic and Architectural Approaches

Budget-aware planning is instantiated via a variety of algorithmic paradigms:

Class	Representative Methods	Notable Features
Reinforcement Learning	PPO-based routers, hierarchical RL, GRPO, meta-RL	Budget signals in state/reward, sometimes multi-budget sampling (Zhang et al., 5 Feb 2026, Lyu et al., 21 Jul 2025, Wen et al., 24 Aug 2025, Vora et al., 2024)
Heuristic & Greedy	Gain-to-cost, thresholding, batch allocations	Fast, interpretable, competitive in online/adaptive settings (Wihidayat et al., 18 Dec 2025, Liu et al., 2018)
Combinatorial Optimization	ILP/LSAP partitioning, MILP, assignment	Exact or lexicographic budget-to-performance selection (Wihidayat et al., 18 Dec 2025, Yang et al., 26 Nov 2025, Vora et al., 2024)
Surrogate Model-Based	Structured GP, Bayesian optimization	Explicit cost and improvement modeling for HPO and resource allocation (Belakaria et al., 2022)
Control-Theoretic	Online feedback loops, safety invariance	Guarantees constraint satisfaction, recursive feasibility (Cherenson et al., 3 Apr 2025)
MCTS and Search-Based	Cost-augmented tree search, action generation	Budget-pruned MCTS nodes, LLM proposal/integration (Zhang et al., 20 May 2025)

A distinguishing pattern is the use of explicit control and feedback channels for the budget (e.g., persistent control tokens, prompt-injected budget blocks), facilitating fine-grained, context-aware adaptation at each planning/decision step (Wen et al., 24 Aug 2025, Liu et al., 21 Nov 2025).

3. Learning and Control Mechanisms for Budget Awareness

Modern methods leverage combinations of supervised pretraining and budget-aware reinforcement learning to achieve both fidelity to budget and high task performance:

• Budget-conditioned Supervised Learning: Models are trained on multi-budget data, e.g., CoT sequences compressed to various budgets (BARD (Niu et al., 3 Nov 2025), BudgetThinker (Wen et al., 24 Aug 2025)), learning to interpret explicit budget tokens or control signals as instructions about resource use.
• Budget-aware RL: Multiplicative or piecewise reward designs penalize overspending while encouraging accuracy; hierarchical or curriculum training (e.g., HBPO (Lyu et al., 21 Jul 2025), BudgetThinker (Wen et al., 24 Aug 2025)) prevents mode collapse toward overly short or long solutions by forcing exploration across budget regimes. Clipped PPO or groupwise ranking objectives are common (Zhang et al., 5 Feb 2026, Lyu et al., 21 Jul 2025, Niu et al., 3 Nov 2025).
• Adaptive Routing and Modularization: Policies (e.g., routers in BudgetMem (Zhang et al., 5 Feb 2026)) select budget tiers (Low/Mid/High or similar) at each pipeline stage, learning module- and query-aware cost-performance allocations.
• Explicit Budget Signal Integration: Embedding budget as a token in the prompt or as a structural component in planning state (e.g., “Budget: [b] tokens” in chain-of-thought models (Niu et al., 3 Nov 2025), control tokens at regular intervals (Wen et al., 24 Aug 2025), budget fields in feedback blocks (Liu et al., 21 Nov 2025)).

This yields robust policies that tightly track budget constraints, utilize allocated resources efficiently, and deliver monotonic or Pareto-optimal accuracy/cost frontiers across varying budget settings.

4. Budget-Aware Planning in Specialized Domains

LLMs and Reasoning Agents

Budget-aware planning for LLMs targets both token/compute usage during reasoning and tool usage during agentic execution:

• BudgetMem (Zhang et al., 5 Feb 2026): Reinforcement-learned router controls per-stage memory extraction modules in a QA pipeline, choosing among implementation-, reasoning-, and capacity-based budget tiers, trading off cost against task F1/LLM-judge accuracy.
• BudgetThinker (Wen et al., 24 Aug 2025): Ratio-based control tokens inserted at inference, with curriculum RL for tight budget adherence and high CoT reasoning accuracy.
• BARD (Niu et al., 3 Nov 2025), HBPO (Lyu et al., 21 Jul 2025): Joint learning for reasoning accuracy and precise control over CoT length.
• Tool-use agents (Liu et al., 21 Nov 2025): Budget Tracker plugin inserts explicit budget-state blocks for tool calls (e.g., search/browse), enabling agents to modulate exploration–verification logic and reach higher accuracy for the same or lower external cost.

Cloud Autoscaling and Infrastructure

• Performance-Feedback Autoscaler (PFA) (Ilyushkin et al., 2019): Feedback on resource throughput guides adaptive, budget-respecting provisioning at each interval; avoids the need for runtime estimates and delivers low job slowdown while automatically balancing under/over-provisioning.
• Multi-Stage Edge Server Upgrade (M-ESU) (Wihidayat et al., 18 Dec 2025): MILP and heuristic greedy algorithms allocate deployment/upgrade actions across stages, modeling per-stage budgets, depreciation, and demand growth, yielding up to 21.57% higher satisfaction versus deployment- or upgrade-prioritized baselines.

Multi-Agent Systems

• BAMAS (Yang et al., 26 Nov 2025): Solves an ILP to select heterogeneous LLM agents under budget, then chooses collaboration topology via RL, assigning best LLMs to critic/planner roles for optimal cost/performance frontier.

Sequential/Hierarchical Planning and MDPs

• Capacity- and Budget-Constrained Monotonic MDPs (Vora et al., 2024): Two-step process uses LSAP for capacity grouping and meta-trained PPO for each group, achieving scalable, near-optimal repair schedules as $n$ grows large.

Online, Real-time, and Safety-Critical Domains

• Safety-Constrained Robotic Planning (Cherenson et al., 3 Apr 2025): The gatekeeper + ReRoot architecture achieves online feasibility and safety under dynamic budget renewal and path constraints for UAVs in unknown environments.
• Online Crowdsourcing (Liu et al., 2018): Greedy thresholding algorithms for dynamic worker-task assignment under travel-cost budgets, with provable competitive ratios and robust online matching.

5. Practical Insights and Performance Benchmarks

• Explicit budget-awareness outperforms naive scaling: Agents or planners granted higher budgets alone do not improve unless mechanisms for budget-signal propagation and budget-conditioned planning are integrated (Liu et al., 21 Nov 2025).
• Pareto frontier tracing: Sweeping budget preference parameters or explicit budget levels yields trade-off curves (accuracy–cost, task satisfaction–expenditure) that strictly dominate baselines only when budget signaling and policies are end-to-end integrated (Zhang et al., 5 Feb 2026, Wen et al., 24 Aug 2025).
• Robustness to underlying models and transfer: Budget-aware controllers (e.g., routers, RL policies) demonstrate transfer across LLM backbones; e.g., routing policies trained on LLaMA perform robustly on Qwen without retraining (Zhang et al., 5 Feb 2026).
• Efficiency and scalability: Greedy and partitioning heuristics, when guided by budget-aware gain/cost or assignment/aggregation strategies, close the gap to combinatorial optima with orders-of-magnitude faster computation (Wihidayat et al., 18 Dec 2025, Vora et al., 2024).

6. General Challenges and Extensions

Key challenges remain in the integration of multi-dimensional budgets (e.g., combining token, time, and external API budgets), adaptivity under distribution shift, and representations that generalize budget-control signals beyond simple scalars (e.g., for multimodal or dynamic environments). Advances in meta-learning and reward shaping are promising for robust generalization and balance across efficiency frontiers (Lyu et al., 21 Jul 2025, Vora et al., 2024).

A plausible implication is that as systems become more heterogeneous, interactive, and cost-varying, budget-aware planning will increasingly be required as a first-class modeling layer, not merely an evaluation constraint.

7. Summary Table: Representative Approaches

Domain	Core Method	Budget Signal	Planner/Policy	Empirical Result (selected)	Reference
LLM Reasoning	BudgetMem	Tiered modules	PPO-based router	Strictly better cost–F1 curve	(Zhang et al., 5 Feb 2026)
Tool-use Agents	Budget Tracker + BATS	Prompt block	Prompt-injection, BATS logic	+12pp accuracy at same budget	(Liu et al., 21 Nov 2025)
Cloud Autoscaling	PFA	Per-interval	Feedback loop, throughput	–47% slowdown, runtime <4x faster	(Ilyushkin et al., 2019)
Edge Compute	M-ESU/H	Stage-wise	Gain/cost greedy + MILP	≤1.25% from MILP, +21% satisfaction	(Wihidayat et al., 18 Dec 2025)
Multi-agent LLMs	BAMAS	ILP+RL policy	Workflow + role assignment	–86% cost, parity accuracy	(Yang et al., 26 Nov 2025)
Budgeted MDPs	LSAP+Meta-PPO	Group assign.	2-step, scalable PPO	Linearly scalable, near-optimal	(Vora et al., 2024)
Online Matching	Greedy-OT	Cost thresh.	Learnt/Random-Thresh Greedy	60–70% of OPT, negligible runtime	(Liu et al., 2018)

Budget-aware planning thus constitutes a unifying paradigm for constrained optimization and adaptive resource allocation, enabling rigorous, scalable, and high-performance solutions in learning, reasoning, control, scheduling, and combinatorial domains.