Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation

Abstract: Frontier LLM agents are automating many business tasks, but their high inference cost makes large-scale deployment unsustainable. Small LLMs (SLMs) offer a cheaper alternative, yet they typically fall short when swapped into a harness designed for a frontier LLM. We show that for many routine business tasks, SLM agents can match LLM performance at 90% lower cost, when paired with an adapted harness that can be automatically discovered by a meta agent. The key insight is that much of the task difficulty is shared across instances and can be lifted from the model into the harness via tailored instructions, tools, and orchestration loops. To study this systematically, we create a framework that maps agent failure modes to harness adaptation strategies, and build a harness optimizer that automatically discovers effective adaptations from failure trajectories. Across seven business-oriented agentic tasks and three SLM families, we found optimized harnesses significantly improve performance on 16 of 21 task-SLM pairs, with seven pairs closing the SLM-LLM performance gap and the best SLM agent recovering 89.7% of LLM performance at 4% of the cost. Our analysis further shows that adaptation works best for tasks with more repetitive workflows and for SLMs with sufficient base capabilities. Together, these results suggest that harness adaptation can expand the practical deployment range of SLM agents in routine business tasks.

• The paper presents an automated harness adaptation framework that uses meta-agents to optimize small language models (SLMs).
• It demonstrates that adapted SLMs can reach 89.7% of LLM performance at only 4% of the cost on diverse enterprise tasks.
• The study systematically maps failure modes to tailored adaptation strategies, resulting in a 25% average reduction in latency.

Harness Adaptation for SLM Agents: Automated Methods for Cost-Effective Business Task Automation

Motivation and Problem Statement

With the rapid proliferation of agentic LLM systems in enterprise settings, cost, latency, and privacy constraints have rendered frontier LLMs impractical for large-scale, routine deployments. Small LLMs (SLMs)—defined as open-weight, locally deployable models with fewer parameters—present a compelling alternative. However, they consistently underperform when inserted into harnesses designed for frontier LLMs, particularly on complex agentic tasks involving tool use, planning, and context management.

The core hypothesis advanced in "Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation" (2607.08938) is that harness adaptation—automatically discovered via meta-agents—enables SLMs to recover a vast majority of LLM performance while delivering extreme reductions in inference cost. Key contributions include a formal framework linking agent failure modes to harness adaptation strategies, a meta-agent-powered harness optimizer, and empirical results demonstrating significant performance gains across a suite of business-relevant tasks.

Failure Mode Characterization and Adaptation Framework

SLM agent failures are systematically categorized by capability-linked modes: tool-use, instruction-following, knowledge, long-context, and planning/reasoning. This capability-indexed taxonomy facilitates direct mapping from observed failures (e.g., invalid tool calls, instruction violations, domain knowledge gaps) to adaptation strategies (e.g., context augmentation, tool-set refinement, loop structural modifications).

Harness adaptation is triaged into three design dimensions:

• Context: Engineering prompts, tool descriptions, and workflow plans to scaffold model behaviors.
• Tooling: Restructuring the tool-action space via custom wrappers, schema tailoring, and tool filtering.
• Agent Loop: Modifying orchestration via hooks, guardrails, and multi-agent architectures.

Automating this process is necessitated by the combinatorial harness design space and the stochastic, intertwined nature of agentic failures. Manual engineering cannot reliably scale or adapt to evolving SLM capabilities.

Figure 1: Mapping agentic failure modes to adaptation strategies, showing how context, tools, and loops bridge specific SLM capability gaps.

Automated Harness Optimization via Meta-Agent Search

Building on recent advances in automated agent design, this work proposes a meta-agent-based optimizer that iteratively samples harnesses, diagnoses failures from execution traces, and proposes targeted adaptations. The search space is instantiated through an SDK exposing editable components such as system prompts, tool schemas, hooks, context condensers, and sub-agent definitions.

The optimization loop has three phases:

1. Sampling and batch evaluation on task instances, logging full trajectories and outcomes.
2. Failure diagnosis with code/context access, informed by design-space documentation and prior search memory.
3. Sanity-checking, validation, and memory updates to facilitate diverse exploration (Figure 2).

Meta-agent proposals consistently resolve failures by externalizing fragile workflows, reducing tool complexity, and introducing runtime safeguards (e.g., anti-loop hooks). Empirically, harness optimization saturates quickly, and the fixed offline cost is amortized after a small number of agent deployments.

Figure 2: Overview of the harness optimization loop orchestrated by the meta-agent.

Figure 3: Example harness adaptation for the budget-approval task—tool hooks, set filtering, and workflow instructions driven by meta-agent diagnosis.

Empirical Validation: Tasks, Models, Quantitative Results

Experiments were conducted across seven agentic tasks (attendance auditing, budget approval, stock alerting, anomaly detection, playwright testing, website management, code refactoring) and three SLM families. Each task was paired with a generic harness and evaluated on accuracy, cost, and latency. Harness adaptation was performed via the meta-agent, with optimized harnesses evaluated against frontier Gemini-3.1-pro agents.

Key results:

• Performance Recovery: Adapted SLMs closed the LLM gap on 7/21 task-model pairs, with the best SLM attaining 89.7% of LLM performance at just 4% of the cost.
• Improvement Scope: Harness optimization improved performance on 16/21 pairs; latency was reduced by an average of 25%.
• Cost Efficiency: Offline optimization cost was negligible relative to deployment savings—recouped after ~13 agent runs.
• Failure Mode Resolution: Most adaptation gains addressed instruction-following (81%) and knowledge (81%) failures via context additions (86%) and tool set management (43%).

Analysis of Adaptation Effectiveness: Diversity and Model Capability

Notably, harness adaptation effectiveness is modulated by task diversity and base SLM capability:

• Task Diversity: More diverse tasks (heterogeneous workflows, complex solution strategies) are harder to optimize. The correlation between task diversity and harness improvement is strongly negative ($\rho = -0.96$). Controlled experiments showed performance dropoff as diversity increased.

  Figure 4: Harness optimization score decreases with task diversity—adaptation is most effective for repetitive workflows.

• SLM Capability: SLMs with higher instruction-following, tool-use, and reasoning capabilities derive substantially greater improvements. Adaptation does not fully compensate for missing model competencies.

  Figure 5: Less capable SLMs are harder to optimize; adaptation gains scale with model quality.

Methodological Insights and Practical Implications

Critical lessons were identified for optimizer design:

• Analysis Quality: Providing the meta-agent with unfiltered trajectory data and the full editable harness facilitates superior adaptation proposals. Frontier LLM meta-agents yield stronger diagnoses than weaker, cheaper alternatives.
• Exploration Diversity: Logging prior attempts and running multiple independent searches are necessary to avoid redundant adaptation and local search stagnation.

For practitioners, this implies:

• Harness adaptation should be considered before defaulting to cost-prohibitive frontier LLMs for routine, repetitive tasks.
• Automated harness optimization is mandatory—manual engineering cannot keep pace with the complexity and diversity of SLM failure patterns.
• Benchmarking and harness compatibility should be central in SLM selection and training.

For researchers, the automated harness adaptation paradigm opens avenues in failure profiling, mixtures-of-harness architectures for diverse tasks, and online adaptation for drifting task distributions.

Relationship to Prior Work

This work builds upon and differentiates itself from earlier engineering efforts targeting SLM limitations through manual tool-schema redesign (Lee et al., 8 Oct 2025), human-engineered heuristics (Srivastava et al., 31 Jan 2026), automated agent design in modular spaces (Hu et al., 2024, Shang et al., 2024, Zhang et al., 6 Feb 2025), and meta-agent self-improvement (Zhang et al., 29 May 2025, Xia et al., 17 Nov 2025, Lee et al., 30 Mar 2026). Prior approaches rarely focus on cost-performance tradeoffs or leverage automated empirical diagnosis for harness design targeting SLMs.

Additionally, the failure mode taxonomy refines previous analyses (Cemri et al., 17 Mar 2025, Zhu et al., 29 Sep 2025), offering a more actionable mapping for adaptation.

Conclusion

This paper demonstrates that automated harness adaptation, orchestrated by a meta-agent, enables SLMs to attain near-parity with LLMs on routine business tasks at a fraction of the cost. The adaptation framework, optimizer design, and task suite provide a rigorous foundation for both cost-effective enterprise deployment and principled agent engineering. Findings are strongest for repetitive workflows and capable SLMs, implicating measurable limits to adaptation. The proposed methods constitute a practical and theoretically robust blueprint for scalable agentic AI driven by harness-centric optimization.