EvoSkill: Automated Skill Discovery for Multi-Agent Systems

Abstract: Coding agents are increasingly used as general-purpose problem solvers, but their flexibility does not by itself confer the domain expertise needed for specialized tasks. Recent work addresses this through \textit{agent skills}: reusable workflows, and code, that augment agents with domain-specific capabilities. Most skills today are hand-crafted, and existing evolutionary approaches optimize low-level artifacts (e.g. prompts & code) that are tightly coupled to specific models and tasks. We introduce \textbf{EvoSkill}, a self-evolving framework that automatically discovers and refines agent skills through iterative failure analysis. EvoSkill analyzes execution failures, proposes new skills or edits to existing ones, and materializes them into structured, reusable skill folders. A Pareto frontier of agent programs governs selection, retaining only skills that improve held-out validation performance while the underlying model remains frozen. We evaluate EvoSkill on two benchmarks: OfficeQA, a grounded reasoning benchmark over U.S.\ Treasury data, where it improves exact-match accuracy by \textbf{7.3\%} (60.6\% $\to$ 67.9\%); and SealQA, a search-augmented QA benchmark with noisy retrieval, where it yields a \textbf{12.1\%} gain (26.6\% $\to$ 38.7\%). We also investigate the zero-shot transfer capabilties of skills evolved on one task to the other; in particular: skills evolved from SealQA transfers zero-shot to BrowseComp, improving accuracy by \textbf{5.3\%} without modification demonstrating that skill-level optimization produces transferable capabilities beyond the training task.

• The paper proposes a self-evolving loop that uses iterative failure analysis and Pareto frontier-based selection to discover reusable agent skills.
• The methodology integrates executor, proposer, and skill-builder agents that collaboratively evolve interpretable, modular skill folders for specialized tasks.
• Empirical results demonstrate notable performance gains, including a +12.1% accuracy boost on benchmarks and successful zero-shot skill transfer across domains.

EvoSkill: Automated Skill Discovery for Multi-Agent Systems

Motivation and Framework Design

The EvoSkill framework addresses a key limitation in coding agent architectures: the absence of systematic, scalable domain expertise in the form of reusable agent skills. While coding agents leveraging intermediate code representations provide broad task flexibility, their performance on specialized domains remains constrained by a lack of structured, domain-specific workflows. Prior hand-crafted skill development is labor-intensive and not scalable, while evolutionary approaches at the code or prompt level fail to yield composable, transferable artifacts.

EvoSkill introduces a self-evolving loop for skill discovery and refinement, centered on iterative failure analysis and Pareto frontier-based selection. The system maintains a pool of agent programs, each comprising a frozen base model (e.g., Claude Code, Opus 4.5) and a dynamically evolving set of skill folders. The skill induction process operates at a high abstraction level: after task execution, failures are systematically diagnosed, proposals for new skills or skill edits are formulated, and skill folders containing instructions, triggers, and auxiliary scripts are materialized (Figure 1).

Figure 1: Overview of the EvoSkill loop: executor, proposer, and skill-builder collaborate for iterative skill evolution via failure analysis.

The proposal-selection procedure ensures that only skills yielding measurable improvements on held-out validation sets persist, enabling cumulative capability enhancement while preserving interpretability and modularity.

Methodological Details

EvoSkill orchestrates three distinct agent roles:

• Executor Agent: task execution governed by current skill repertoire.
• Proposer Agent: structured failure analysis, skill gap detection, and proposal generation.
• Skill-Builder Agent: synthesizes proposals into structured skill folders conforming to the Agent Skills specification.

Skill folders encapsulate procedural guidance, trigger logic, and optionally executable scripts, enabling fine-grained augmentation of agent workflows. The evolution loop employs round-robin parent selection from a fixed-capacity Pareto frontier, structured batch sampling for failure analysis, cumulative feedback histories to inform proposal refinement, and validation-based program scoring for frontier membership. The underlying model remains frozen throughout, isolating performance improvements to the discovery and composition of skills.

Data and environmental setup employ stratified, category-wise clustering for partitioning benchmarks into disjoint training, validation, and test sets. Agent programs are managed as lightweight git branches, ensuring reproducibility and lineage tracking of evolved skill sets.

Empirical Evaluation

OfficeQA: Structured Reasoning Benchmark

EvoSkill was evaluated on OfficeQA, a domain-specific benchmark requiring grounded reasoning over U.S. Treasury Bulletins. The agent is tasked with extracting and synthesizing quantitative information across dense tabular and textual artifacts.

Experiments spanned varying training splits (5%, 10%, 15%), with held-out validation used for frontier promotion. The strongest configuration—skill-merge, where unique skills from independent runs are combined—achieved 67.9% exact-match accuracy, a 7.3 percentage point improvement over the baseline (60.6%). Gains persisted across all tolerance thresholds, with diminishing returns above 10% training split, indicating potential overfitting.

Figure 2: EvoSkill performance on OfficeQA: skill-merge outperforms baselines and independent runs, providing 67.9\% exact-match accuracy (+7.3\%).

Qualitative skill analysis reveals interpretable, targeted modules such as rigorous data extraction verification protocols and structured quantitative analysis workflows. These directly address recurring failure modes (e.g., adjacent cell misreads, metric confusion), reinforcing the premise that skill-level optimization can produce domain-relevant, composable enhancements.

SealQA: Search-Augmented LLM Reasoning

SealQA poses open-ended fact-seeking tasks under adversarial retrieval, challenging agent robustness against noisy web search results. EvoSkill improved agent accuracy from 26.6% to 38.7% (+12.1\%), with transferable skills such as search-persistence-protocol enforcing exhaustive multi-source verification and completeness checks. The evolved skills on SealQA display qualitative differentiation: search strategy augmentation, rather than document parsing, illustrating the domain agility of the framework.

Zero-Shot Skill Transfer

A critical outcome is the ability of EvoSkill-evolved skills to transfer zero-shot between disjoint benchmarks. The search-persistence-protocol, developed on SealQA, was directly applied to BrowseComp—yielding an absolute accuracy improvement of 5.3 percentage points (43.5% to 48.8%) with no modification. This substantiates EvoSkill's claim that skill-level optimization delivers persistent, transferable, and interpretable modular enhancements, outperforming prior approaches that optimize model-coupled prompts or code.

Related Work Integration

EvoSkill advances modular agent skill discovery beyond Voyager and hand-authored skill paradigms by providing an open-ended, failure-driven framework for reusable, interpretable skill artifact evolution. Techniques such as Feedback Descent, Self-Refine, and evolutionary search (AlphaEvolve, GEPA) have demonstrated the utility of textual feedback and evolutionary optimization, but operate at artifact granularity. EvoSkill shifts the abstraction upwards, evolving agent skill folders (metadata, instructions, and code) tied not to task or model specificity, but to generalizable capability gaps.

Transfer learning in agent systems has traditionally depended on prompt or code-level adaptation, often experiencing degradation across domains. Skill folder abstraction, as demonstrated by EvoSkill, offers a more robust unit for transfer, supported by empirical zero-shot results.

Practical and Theoretical Implications

EvoSkill has direct implications for both practical agent deployment and further research in multi-agent systems:

• Composability and Interpretability: Modular skill folders facilitate direct inspection, reuse, and composition, supporting collaborative skill libraries, rapid adaptation to novel tasks, and maintenance of domain expertise in agentic workflows.
• Scalable Domain Expertization: Skill-level evolution aggregates fine-grained expertise efficiently, mitigating the need for extensive hand-crafting.
• Transferability: Evolved skills persist across model, task, and harness boundaries, laying groundwork for truly reusable AI modules.
• Isolation of Improvements: By keeping the underlying model frozen, EvoSkill enables rigorous attribution of performance gains to skill evolution, not model fine-tuning.

Future research directions include multi-modal skill evolution, broad cross-domain evaluations to characterize domain-general versus domain-specific skill emergence, and systematic modular skill library development and sharing.

Conclusion

EvoSkill establishes an automated, failure-driven paradigm for skill discovery in agent systems, providing a scalable mechanism for reusable, transferable, and interpretable agent capability enhancement. Empirical results across OfficeQA and SealQA underscore substantial performance gains (up to +12.1%), with evidence of skill transferability to BrowseComp. This reframes agent evolution from artifact-level optimization to modular skill induction, supporting agent harnesses with increasingly robust, domain-adaptive competencies. The modular skill abstraction heralds a shift toward collaborative, composable agent expertise, inviting further exploration into richer multi-agent and multi-modal skill ecosystems.