GraphPlanner: Graph Memory-Augmented Agentic Routing for Multi-Agent LLMs

Abstract: LLM routing has achieved promising results in integrating the strengths of diverse models while balancing efficiency and performance. However, to support more realistic and challenging applications, routing must extend into agentic LLM settings, where task planning, multi-round cooperation among heterogeneous agents, and memory utilization are indispensable. To address this gap, we propose GraphPlanner, a heterogeneous graph memory-augmented agentic router for multi-agent LLMs that generates routing workflows for each query and supports both inductive and transductive inference. GraphPlanner formulates workflow generation as a Markov Decision Process (MDP), where at each step it selects both the LLM backbone and the agent role, including Planner, Executor, and Summarizer. By leveraging a heterogeneous graph, denoted as GARNet, to capture interaction memories among queries, agents, and responses, GraphPlanner integrates historical memory and workflow memory into richer state representations. The entire pipeline is optimized with reinforcement learning, jointly improving task-specific performance and computational efficiency. We evaluate GraphPlanner across 14 diverse LLM tasks and demonstrate that: (1) GraphPlanner outperforms strong single-round and multi-round routers, improving accuracy by up to 9.3% while reducing GPU cost from 186.26 GiB to 1.04 GiB; (2) GraphPlanner generalizes robustly to unseen tasks and LLMs, exhibiting strong zero-shot capabilities; and (3) GraphPlanner effectively leverages historical memories, supporting both inductive and transductive inference for more adaptive routing. Our code for GraphPlanner is released at https://github.com/ulab-uiuc/GraphPlanner.

• The paper introduces a graph memory-augmented MDP framework that integrates historical and workflow memories for agentic routing in multi-agent LLM systems.
• It utilizes a heterogeneous graph architecture (GARNet) and explicit role assignments (Planner, Executor, Summarizer) optimized via PPO to improve task accuracy by up to 9.3%.
• Extensive experiments show enhanced efficiency with reduced GPU usage, robust zero-shot generalization, and scalable adaptability to new agent roles.

GraphPlanner: Graph Memory-Augmented Agentic Routing for Multi-Agent LLMs

Motivation and Background

Modern applications of LLMs increasingly demand efficient orchestration across diverse model backbones and specialized agentic roles. While initial routing approaches have been formulated as either single-shot or sequential assignments based on query content and interaction history, these methods fall short in multi-agent agentic LLM systems requiring explicit roles, complex task decomposition, and long-horizon coordination. In agentic settings, routers must address three major challenges: (i) modeling heterogeneous agent collaboration and dependencies, (ii) addressing deferred rewards and credit assignment issues, and (iii) leveraging rich traces of historical multi-agent workflows to improve future routing. Prior routers generally ignore agent specialization, treat LLM calls independently, and underutilize historical interaction context.

Methodology

Agentic Routing via MDP and Heterogeneous Graphs

GraphPlanner formulates routing in a multi-agent LLM system as a Markov Decision Process (MDP), in which routing steps comprise both an agentic role selection (Planner, Executor, Summarizer) and a backbone LLM assignment. The key innovation is the heterogeneous graph-based architecture, GARNet, which exploits both current workflow memory ($G_\mathrm{workflow}$) and a historical memory graph ($G_\mathrm{history}$). In each routing step, the router’s state consists of the current query; the action is a tuple selecting role and LLM. Semantic masks ensure only valid role transitions, balancing workflow progress and role consistency. GraphPlanner integrates intermediate answers, role assignments, and cost/utility trade-offs directly into the graph, producing globally context-aware routing policies.

GARNet uses role hub nodes shared between current workflow and historical graphs, allowing efficient message passing and aggregation of contextual, historical, and cost-aware representations. This shared-node mechanism ensures adaptation over both local (trajectory-specific) and global (historical) dimensions, enabling robust handling of both inductive (no access to histories at inference) and transductive (including histories during inference) generalization. The entire policy is optimized with Proximal Policy Optimization (PPO), jointly maximizing task utility and minimizing computational cost.

Workflow Generation and Agent Roles

The explicit inclusion of agent roles enables hierarchical and parallel workflow construction. The Planner decomposes complex queries; Executors answer sub-queries, optionally incorporating context; the Summarizer aggregates intermediate results. GraphPlanner’s architecture supports dynamic, query-dependent workflow topology, contrasting with both single-step, flat assignments (single-round routers) and strictly linear multi-step approaches (multi-round routers).

Experimental Results

Routing Efficiency and Numerical Performance

GraphPlanner is evaluated on 14 tasks spanning 6 domains (math reasoning, code generation, commonsense reasoning, world knowledge, comprehensive QA, and out-of-domain generalization). Compared to both strong single-round (e.g., RouterKNN, RouterMLP, RouterDC, GraphRouter) and multi-round (Prompt LLM, R2-Reasoner, Router-R1) baselines, GraphPlanner consistently improves average accuracy by up to 9.3% in unconstrained workflow construction settings, while reducing peak GPU memory from 186.26 GiB (Router-R1) to 1.04 GiB and achieving fastest inference latency (1.2 s/query).

For complex reasoning tasks (math, code), GraphPlanner yields particularly pronounced gains (5% absolute accuracy improvement), indicating that the workflow-based agentic framework is most effective when compositionality and inter-agent cooperation are essential. On simpler recognition tasks, gains over baselines are less prominent, supporting the claim that adaptive workflow construction is most impactful for multi-step, high-complexity domains.

Generalization to Unseen Tasks and LLMs

GraphPlanner demonstrates robust zero-shot and few-shot generalization. On benchmarks withheld from training—including LogicGrid, MGSM, CommonGen, and AIME—GraphPlanner surpasses prior routers by large margins (e.g., 78% average accuracy vs. 46%, 58%, 38% for leading baselines). This robustness extends to deployment with unseen backbone models: GraphPlanner dynamically interpolates to new agent-LLM configurations without retraining, supported by GARNet’s history-integrating architecture.

Ablation Studies

Key ablations validate two core premises. First, removal of historical context or downgrade to homogeneous graph networks substantially degrades performance, underscoring the necessity of explicit memory integration and heterogeneity modeling. Second, GARNet outperforms heavier graph transformers and graph attention alternatives, confirming that efficient, tailored role-based message passing is both more effective and computationally preferable in large-scale agentic routing scenarios.

Cost-Accuracy Pareto Analysis

Adjusting the trade-off parameter $\lambda$ within the PPO-optimized reward allows GraphPlanner to flexibly traverse the accuracy-cost Pareto frontier, delivering higher accuracy at reduced compute, or lower cost for equivalent performance, compared to all examined baselines.

Adaptability to New Agentic Roles

GraphPlanner readily extends to additional agentic roles (e.g., Thinker, Verifier), demonstrating both strong RL-based learning when trained on new roles and impressive zero-shot/few-shot generalization when encountering unseen agent role types at test time. This establishes its extensibility toward richer role taxonomies and more expressive multi-agent protocols.

Implications and Future Directions

Practical Implications

GraphPlanner offers an efficient, scalable paradigm for deploying multi-agent LLM-based systems that combine specialization, adaptivity, and historical learning. The reduction in compute and improved routing accuracy will directly impact real-world applications with stringent efficiency requirements or those operating on dynamic, heterogeneous agent/LLM pools (e.g., cloud-based APIs, enterprise AI orchestration).

Theoretical Impact

GraphPlanner's architecture provides a foundation for principled workflow generation in structured multi-agent LLM environments. Beyond backbone/model selection, it frames routing as a structured optimization over role-LLM pairs, leveraging both short- and long-horizon context via graph-based state encoding. This formulation bridges the gap between reinforcement learning for agentic coordination and efficient memory-augmented policy architectures.

Speculation on Future Developments

GraphPlanner’s demonstrated extensibility to new roles and robust zero-shot operation suggests future research could explore (i) richer agent profiles (e.g., Tool-User, Critic, Retriever), (ii) tighter integration with tool-call and API-based ecosystems, and (iii) self-optimizing agent systems capable of lifelong learning over expanding multi-agent histories. Advanced, learned agentic planning protocols leveraging even more complex relational memory architectures could enable scalable, adaptive orchestration of massive agentic LLM swarms.

Conclusion

GraphPlanner advances the field of multi-agent LLM routing by introducing a graph memory-augmented, agentic workflow generation framework, jointly optimizing task-specific utility and efficiency. Its heterogeneous graph architecture, explicit role modeling, and deep integration of historical and workflow memory yield state-of-the-art routing performance across challenging domains, while providing favorable generalization and efficiency profiles. Future work will likely build on these foundations to further scale and enrich agentic coordination in open, dynamic AI ecosystems.

Reference: "GraphPlanner: Graph Memory-Augmented Agentic Routing for Multi-Agent LLMs" (2604.23626)