Agent Context Protocols Enhance Collective Inference

Abstract: AI agents have become increasingly adept at complex tasks such as coding, reasoning, and multimodal understanding. However, building generalist systems requires moving beyond individual agents to collective inference -- a paradigm where multi-agent systems with diverse, task-specialized agents complement one another through structured communication and collaboration. Today, coordination is usually handled with imprecise, ad-hoc natural language, which limits complex interaction and hinders interoperability with domain-specific agents. We introduce Agent context protocols (ACPs): a domain- and agent-agnostic family of structured protocols for agent-agent communication, coordination, and error handling. ACPs combine (i) persistent execution blueprints -- explicit dependency graphs that store intermediate agent outputs -- with (ii) standardized message schemas, enabling robust and fault-tolerant multi-agent collective inference. ACP-powered generalist systems reach state-of-the-art performance: 28.3 % accuracy on AssistantBench for long-horizon web assistance and best-in-class multimodal technical reports, outperforming commercial AI systems in human evaluation. ACPs are highly modular and extensible, allowing practitioners to build top-tier generalist agents quickly.

• The paper introduces Agent Context Protocols (ACPs) to standardize multi-agent communication and coordination for scalable, fault-tolerant collective inference.
• The methodology decomposes complex tasks into DAG-based sub-tasks with structured message formats, enabling efficient error recovery.
• Empirical evaluations demonstrate significant improvements in coordination accuracy and robust performance across benchmark tasks.

Structured Protocols for Multi-Agent Collective Inference

Motivation and Context

Recent advances in LLM-based AI agents have yielded systems proficient in specialized tasks such as coding, complex reasoning, and multimodal data synthesis. However, the construction of robust generalist systems demands seamless collaboration among heterogeneous agents, whose task interdependencies often necessitate sophisticated coordination and fault tolerance—gaps not adequately bridged by ad-hoc natural language communication. The lack of standardized mechanisms for interoperability, error handling, and structured execution in multi-agent settings presents critical barriers to scaling collective inference systems.

Agent Context Protocols: Design and Architecture

To address these challenges, the paper introduces Agent Context Protocols (ACPs), a domain-agnostic, modular schema governing agent-agent communication, coordination, and error resolution. ACPs rest on two primary abstractions: the Execution Blueprint (a DAG encoding sub-task dependencies and status) and structured message formats for inter-agent and agent-tool interactions. Each complex task $T$ is decomposed into atomic sub-tasks $\{\tau_i\}$, with individual agents $A_i$ assigned operations $\mathcal{O}_i$ according to their capabilities; dependencies in $T$ propagate through the Execution Blueprint, which stores both intermediate states and agent outputs.

Agents interact via protocol-governed message types:

• AGENT_REQUEST: Structured data for tool invocation, aggregating LLM-generated and tool-derived inputs with strict validation.
• AGENT_RESPONSE: Standardized output specifications with status codes and downstream variables.
• ASSISTANCE_REQUEST: Context-rich error reports issued on detection of invalid/missing/incomplete data, leveraging descriptive error codes akin to HTTP semantics.

This architecture provides robust mechanisms for tracking progress, diagnosing faults, and dynamic re-planning, all while adhering to the global Execution Blueprint. The persistency of the blueprint enables efficient error isolation and facilitates parallel/serial task execution by specialized agents.

Figure 1: Overview of the ACP-based system workflow. A complex task is decomposed, executed as a DAG, and coordinated via structured messages and fault-tolerant logic.

Empirical Evaluation

Experiments span three axes: benchmarked web assistance (AssistantBench), multimodal report synthesis, and dashboard creation with control ablations evaluating coordination and fault tolerance.

Web Assistance: AssistantBench Performance

On the AssistantBench benchmark, ACP-powered multi-agent systems achieve 28.3% overall accuracy, surpassing both generalist and specialist agents—even those employing more sophisticated base models. When restricted to minimal toolsets, the ACP system maintains competitive performance (24.8% accuracy), demonstrating that the protocol layer per se supports coherent long-horizon reasoning and extensibility without re-training. ACPs drive robust results across all difficulty levels, with 48.5% accuracy on medium and 15.5% on hard tasks. The ability to seamlessly integrate domain-specific tools—via standardized interfaces—unlocks rapid capability expansion and system adaptation.

Multimodal Report Generation

The architecture enables the synthesis of complex multi-agent outputs in domains including Finance, Technology, Healthcare, Automobile, and Real Estate. Coordinated agent workflows generate highly-structured multimodal reports, integrating textual analysis, data visualizations, and curated citations. Human evaluators rated ACP-based reports consistently highest across all assessed dimensions (Coverage, Presentation Quality, Depth, Clarity) compared to Perplexity and Gemini baselines. Notably, ACP-generated documents sustain high presentation and coverage scores, attributed to robust inter-agent communication and context-preserving execution over lengthy workflows.

(Figure 2)

Figure 2: Sample multimodal report segments generated by ACP-based agents, demonstrating integration of text, visuals, and citations in complex documents.

Dashboard Creation: Coordination and Fault Tolerance Ablation

A synthetic dashboard dataset stratified by complexity was used to quantify the marginal gains from task decomposition and protocol-driven coordination. Comparison across Single Agent, No Assistance (multi-agent, no protocol), and full ACP setups revealed that ACP yields substantial improvements per human evaluation—overall score 3.95 vs. 2.94 (No Assistance) and 1.96 (Single Agent). Level 3 (highest complexity) tasks benefit most from coordinated execution, with error handling and dynamic re-routing mechanisms drastically reducing workflow collapse rates and ensuring output consistency. Structured error codes and assistance requests localize faults and facilitate partial execution, supporting high-yield agent collectives in deep workflows.

(Figure 3)

Figure 3: Execution timeline for a complex travel planning task, highlighting parallel and sequential agent activity, and the ACP-based protocol’s error recovery mechanisms.

Comparative Perspective and Related Work

ACP advances on prior agent orchestration frameworks (AutoGen, Magentic-One, MetaGPT, etc.) by formalizing inter-agent dialogue and execution via a single extensible protocol, rather than relying solely on natural language or weakly structured operational roles. Drawing inspiration from single-agent context protocols (e.g., Model Context Protocol), ACPs operationalize persistent blueprints and schema-driven interaction for multi-agent collectives. Related work in collaborative robotics, task grammars, and active inference has explored aspects of interoperability and reasoning; ACP unifies these efforts to tackle long-horizon, domain-diverse, and error-prone environments, establishing a scalable substrate for generalist AI systems.

Implications and Future Directions

The introduction of ACPs sets a new standard for scalable, interpretable, and resilient multi-agent systems. Practically, ACPs provide practitioners a highly modular template for rapid prototyping and deployment of agent collectives, with clear paths for domain specialization and capability expansion via plug-and-play tools. Theoretically, ACPs open research into collective intelligence, scaling behaviors under protocol constraints, and compositional generalization. Scalability to larger populations, incorporation of higher-order reasoning agents for global re-planning, and extension to dynamic, non-stationary environments are promising avenues for further refinement.

Conclusion

Agent Context Protocols present a rigorously structured foundation for robust multi-agent communication, coordination, and error management, enabling efficient, fault-tolerant collective inference. Empirical results validate ACPs as a critical enabler for generalist AI, outperforming existing systems on complex benchmarks and generation tasks. This protocol-driven approach streamlines the composition of reliable, extensible agent teams, pushing the field toward interpretable and scalable collaboration in practical AI deployments.