MCP-Zero: Proactive Toolchain Construction for LLM Agents from Scratch

Abstract: Function-calling has enabled LLMs to act as tool-using agents, but injecting thousands of tool schemas into the prompt is costly and error-prone. We introduce MCP-Zero, a proactive agent framework that lets the LLM itself decide when and which external tools to retrieve, thereby assembling a task-specific toolchain from scratch. The framework is built upon three components: (1) Proactive Tool Request, where the model emits a structured $\left<\operatorname{tool_assistant}\right>$ block that explicitly specifies the desired server and task; (2) Hierarchical Vector Routing, a coarse-to-fine retrieval algorithm that first selects candidate servers and then ranks tools within each server based on the semantic similarity; (3) Iterative Proactive Invocation, enabling multi-round, cross-domain toolchain construction with minimal context overhead, and allowing the model to iteratively revise its request when the returned tools are insufficient. To evaluate our approach we also compile MCP-tools, a retrieval dataset comprising 308 MCP servers and 2,797 tools extracted from the official Model-Context-Protocol repository and normalized into a unified JSON schema. Experiments show that MCP-Zero (i) effectively addresses the context overhead problem of existing methods and accurately selects the correct tool from a pool of nearly 3,000 candidates (248.1k tokens); (ii) reduces token consumption by 98\% on the APIbank while maintaining high accuracy; and (iii) supports multi-turn tool invocation with consistent accuracy across rounds. The code and dataset will be released soon.

• The paper presents an active tool discovery framework that allows LLM agents to autonomously request and retrieve contextually relevant tools.
• It employs hierarchical semantic routing with OpenAI text embeddings to efficiently rank candidates, significantly reducing search complexity.
• The framework achieves remarkable token efficiency and robust scalability, demonstrated by a 98% reduction in token usage and high accuracy in extreme scenarios.

MCP-Zero: Active Tool Discovery for Autonomous LLM Agents

Introduction and Motivation

The MCP-Zero framework addresses a critical bottleneck in the design of autonomous LLM agents: the inefficiency and lack of autonomy in current tool-calling architectures. Existing paradigms either inject comprehensive tool schemas into the prompt—leading to excessive context length and cognitive overload—or rely on static retrieval based on initial user queries, which fails to adapt to evolving task requirements. MCP-Zero proposes a shift from passive tool selection to active tool discovery, restoring decision authority to the agent and enabling dynamic, on-demand capability acquisition.

Framework Architecture

MCP-Zero is built on three synergistic mechanisms:

1. Active Tool Request: The agent autonomously generates structured requests specifying the required server domain and tool operation, rather than relying on user queries or pre-selected tool sets. This ensures semantic alignment between agent needs and tool documentation.
2. Hierarchical Semantic Routing: A two-stage retrieval algorithm first filters candidate servers by semantic similarity, then ranks tools within selected servers. Embeddings are computed using OpenAI text-embedding-3-large, and matching leverages both original and LLM-generated server summaries for improved precision.
3. Iterative Capability Extension: The agent can iteratively refine its tool requests throughout task execution, enabling dynamic construction of cross-domain toolchains and natural fault tolerance. This process continues until suitable tools are found or the agent determines that no external assistance is required.

   Figure 1: MCP-Zero's iterative invocation enables the agent to identify capability gaps and request tools across multiple domains as the task evolves.

This architecture fundamentally reduces context overhead and search complexity, shifting from $O(n)$ exhaustive search over all tools to $O(m + k)$, where $m$ is the number of servers and $k$ is the number of tools per server.

Theoretical Analysis

MCP-Zero's active paradigm is formalized as an information acquisition process, where the agent generates requests to maximize information gain about task completion. The mutual information between the optimal tool set and the request is maximized, reducing uncertainty and focusing attention on relevant subsets of the tool ecosystem. Semantic alignment is improved by operating in the same embedding space as tool documentation, yielding higher retrieval precision compared to user-query-based matching.

Key theoretical advantages include:

• Complexity Reduction: Hierarchical routing reduces the search space from all tools to relevant servers and tools.
• Semantic Consistency: Agent-generated requests are more closely aligned with tool documentation.
• Adaptive Capability: The agent's tool discovery evolves with its understanding of the task, supporting progressive breakdown and cross-domain coordination.

MCP-Tools Dataset

To support evaluation, the authors introduce MCP-tools, a retrieval-oriented dataset comprising 308 servers and 2,797 tools from the official Model Context Protocol repository. The dataset includes both original and LLM-generated summaries, pre-computed embeddings, and a standardized schema for semantic matching. This infrastructure enables reproducible evaluation of tool discovery systems and complements performance-focused frameworks such as MCPBench.

Experimental Evaluation

Needle-in-a-Haystack Experiments

The framework is evaluated under extreme scale conditions, where the agent must retrieve relevant tools from collections ranging from 1 to 2,797 candidates. MCP-Zero demonstrates substantial gains in both accuracy and token efficiency compared to baseline methods.

Figure 2: MCP-Zero achieves strong performance in needle-in-a-haystack tests, outperforming baselines on Claude-3.5-Sonnet and Gemini-2.5-Flash, with GPT-4.1 maintaining high baseline accuracy.

Figure 3: MCP-Zero maintains low average token cost per successful retrieval, even as tool collection size increases.

APIBank Evaluation

On the APIBank dataset, MCP-Zero achieves a 98% reduction in token consumption while maintaining high accuracy across both single-turn and multi-turn scenarios. Notably, MCP-Zero remains robust as the tool pool scales, with accuracy dropping only marginally compared to severe degradation in baseline methods. Query-retrieval baselines stall at 65–72% accuracy, confirming the necessity of agent-authored, semantically aligned requests.

Implementation Guidance

The framework is straightforward to integrate into existing agent systems:

1. Prompt the LLM to actively request tools using a structured output block.
2. Curate a lightweight tool index with semantic descriptions and pre-computed embeddings.
3. Match agent requests to tool documentation via hierarchical semantic routing, feeding top candidates back to the agent for invocation.

A single in-context learning example further improves semantic alignment and retrieval precision, acting as a stylistic and semantic anchor for the agent's requests.

Implications and Future Directions

MCP-Zero establishes active tool discovery as a fundamental design pattern for scalable, autonomous agent systems. By restoring decision autonomy and enabling dynamic capability acquisition, the framework addresses both practical and theoretical limitations of existing architectures. The synergy with systems such as Alita—where agents can not only discover but also synthesize new tools—points toward self-evolving, cost-aware agentic AI.

Future work should explore:

• Enhanced matching algorithms incorporating multi-modal descriptions and usage patterns.
• Packaging MCP-Zero as a dedicated MCP server for meta-discovery.
• Multi-agent orchestration for collaborative tool sharing and dynamic capability extension.

Conclusion

MCP-Zero represents a significant advancement in autonomous agent design, demonstrating that active, iterative tool discovery yields substantial efficiency gains and robust scalability. The framework's theoretical foundations, empirical validation, and supporting dataset provide a comprehensive infrastructure for future research in agentic AI and tool-augmented reasoning.