Generative Skill Composition for LLM Agents

Abstract: Recent LLM agents benefit from skills for solving complex tasks. Skills encapsulate modular packages of procedural knowledge and instructions for performing specialized tasks, such as setting up a sandboxed environment, running a test suite, or refactoring a function across multiple files. As skill libraries grow and become reusable across tasks and domains, selecting an appropriate skill composition has emerged as a central bottleneck. Existing approaches fall into two categories. One exposes the agent's reasoning to the entire skill collection; the other performs skill retrieval via embeddings or LLM-based rerankers. Both provide useful insights; however, they miss the structural nature of skill composition, which is a joint decision over which skills, how many, and in what order -- three dimensions that cannot be decoupled. We formalize this as structured skill composition: given a task and a skill library, predict an executable skill plan that jointly specifies the activated subset, count, and execution order. We propose SkillComposer, which instantiates structured skill composition as task-conditioned skill sequence prediction. SkillComposer uses a constrained autoregressive decoder over skill identifiers, so subset, count, and order emerge jointly from a single decoding pass, and dependencies between successive skills are captured naturally. We build a training set of task-composition pairs from a real, human-curated skill library. We then evaluate SkillComposer along two axes: composition quality on a held-out test set, and downstream task success on SkillsBench across two production-grade coding agents. On GPT-5.2-Codex, Gemini-3-Pro-Preview, SkillComposer raises the pass rate by +23.1, +18.2pp over the no-skill baseline, surpassing top-3 retrieval and matching the gold-skill retrieval upper bound at lower prompt-token cost.

• The paper introduces SkillComposer, a method that formulates LLM skill planning as ordered, closed-vocabulary sequence generation for efficient compositional reasoning.
• It employs a transformer-based autoregressive decoder with auxiliary heads and retrieval-augmented decoding to ensure precise skill selection and ordering.
• Experimental results show significant improvements in planning accuracy and downstream execution, enhancing both agent performance and explainability.

Structured Skill Composition for LLM Agents via SkillComposer

Motivation and Problem Formulation

The proliferation of reusable, procedural skills for LLM agents—modular packages encapsulating instructions and resources for subtasks such as test execution, file manipulation, and code refactoring—has shifted the operational bottleneck from skill acquisition to compositional planning. As libraries scale to hundreds or thousands of skills, agents must resolve a coupled decision space: which skills to invoke, how many, and in what linearized order, conditioned on the downstream task and its context. Prevailing interfaces either expose agents to the flat library, relegating composition to implicit plan traces, or operate via embedding-based retrieval or reranking, which yield unordered subsets and sidestep explicit cardinality and sequencing.

The paper introduces a structured skill composition paradigm: skill planning is formalized as a closed-vocabulary, task-conditioned, ordered skill sequence generation problem, capturing subset, count, and order jointly within a single inference pass. This paradigm targets real agentic deployments where policies must be explainable, compositional, and directly mappable to executable skill instructions.

Figure 1: Structured skill composition conceptualized with SkillComposer—addressing the bottleneck from large skill libraries by explicitly predicting ordered, executable skill plans and improving both planning accuracy and task outcomes.

SkillComposer: Model Architecture

SkillComposer frames skill composition as task-conditioned autoregressive sequence decoding over the skill indices, terminated by a STOP symbol. This mechanism ensures the agent explicitly determines the relevant skill subset, its cardinality, and a procedurally correct order for execution, thereby capturing inter-skill dependencies.

The architecture consists of:

• A frozen retrieval-tuned text encoder (Qwen3-Embedding-0.6B) producing a pooled, task-context vector from serialized inputs.
• A small, transformer-based, constrained autoregressive (AR) decoder with cross-attention into skill metadata embeddings, enabling positionally-aware skill selection.
• Two auxiliary heads—one for set membership and one for cardinality—trained jointly to provide dense, factorized supervision for which skills are present and how many are required, supplementing the sparse AR supervision for ordering.
• A retrieval-augmented decoding step, in which decoder logits are fused with a static, task–skill TF-IDF retrieval prior and membership priors, allowing for robust relevance estimation and smoothing over data sparsity in tail skills.

  Figure 2: SkillComposer overview—joint task and skill-context encoding (A), contextual autoregressive decoding with auxiliary heads (B), and retrieval/membership prior fusion for enhanced calibration and accuracy (C).

The closed vocabulary and explicit library-index addressing guarantee that predictions correspond to executable, inspectable units—critical in safety, auditability, and context-limited inference.

Experimental Evaluation

Dataset Construction

The authors construct a dataset of 9,872 task–skill-sequence pairs anchored in a real, human-curated skill library (196 skills) from SkillsBench. Ground truth skill chains are sourced from real agent trajectory logs and supplemented by systematically synthesized tasks that span both single-skill and dependency-graph (data-flow/workflow edge)-grounded multi-skill compositions. Rigorous deduplication and structural validation yield robust supervision for sequence planning.

Skill Prediction Metrics

SkillComposer and baselines are evaluated on both in-distribution synthetic and out-of-distribution real holdout sets, using Set F1 (order-agnostic skill selection), Recall@5, MRR, nDCG@5, Set Exact Match, and cardinality calibration.

Result Highlights

• SkillComposer achieves state-of-the-art skill planning accuracy: On synthetic test tasks, it outperforms all baselines, including standard SFT (600M parameters) and LLM-judge (Gemini-2.5-flash) by +2.8pp and +12.9pp Set F1 respectively, with only 3.9M trainable parameters. On challenging real-task holdouts, SkillComposer degrades gracefully (−11pp), while SFT collapses (−27.5pp), yielding a +19.3pp Set F1 gap over SFT.
• Downstream agent execution performance: Directing both GPT-5.2-Codex and Gemini-3-Pro coding agents, SkillComposer achieved 45.3%/44.0% pass rate, respectively—a +23.1 and +18.2 percentage point improvement over agent-only baselines, and surpassing both retrieval-based (top-3, oracle) and “all skills loaded” approaches at substantially lower context cost.

Figure 3: (a) SkillComposer exhibits robust decoding hyperparameter insensitivity. (b) Compute–accuracy tradeoff shows Pareto-optimality over SFT: $\sim154\times$ fewer trainable parameters and $\sim25\times$ less compute. (c) Inference latency parity with SFT, but with consistently higher accuracy.

• Ablation studies confirm that both retrieval and set-membership priors, as well as factorized auxiliary supervision, are critical; removing any degrades F1 by 2.5–7.1pp.

Analysis and Implications

SkillComposer’s compositional decoding underlines the limitations of both naive retrieval and end-to-end LM plan generation for scalable skill orchestration. First, unordered k-best skill retrieval—even with oracle cardinality—suffers from inability to specify execution order or resolve relevant compositional dependencies. Second, standard SFT, while capable of order-aware text sequence generation, overfits synthetic templates and fails to transfer robustly to human-authored, distribution-shifted tasks.

The joint AR/auxiliary head framework elegantly unifies procedural chaining and subset selection, with the retrieval prior acting as a strong regularizer in data-sparse, heavy-tailed libraries. Importantly, the modular design—using a frozen, retrieval-optimized embedding backbone—offers parameter and context efficiency well-suited for real-world deployment regimes where memory and inference cost per agent are critical.

On a theoretical level, this work advances the structured output learning in compositional planning: closed-vocabulary generation over a discrete, fixed library, rather than unconstrained, free-form text. This structure enables invariance to surface-level phrasing, direct determinism in skill activation, and improved transparency.

Future Directions

Several open directions emerge from this work:

• Generalizing to multimodal settings, expanding from text and code to include visual or tabular contexts, and supporting skills that operate on non-textual input.
• Dynamic library management and online composition, where skill libraries may grow or evolve during agent deployment, requiring continual adaptation of the composer.
• Integration with long-horizon or interactive agents, where compositional plans may be updated or recomposed mid-interaction, necessitating differentiable, extensible planners.
• Scaling to heterogeneous skill/talent spaces in mixed-modality or robotics domains, beyond code and text, where procedural dependencies generalize to hardware or simulation environments.

Conclusion

This work demonstrates that Generative Skill Composition, instantiated as closed-vocabulary sequence generation with SkillComposer, is a robust and efficient solution to the structured planning bottleneck in skill-rich LLM agents. By combining retrieval-tuned embedding, compact AR decoding, and inference-time fusion of lexical priors with auxiliary supervision, SkillComposer consistently yields superior skill chain prediction and downstream agent task success at minimal computational overhead. The findings position specialized, structure-aware composers as a preferable alternative to large end-to-end LMs or retrieval-only selectors for closed skill libraries, with clear implications for real-world agent deployment and ongoing methodological extensions in modular AI.