AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning

Abstract: Language agents have achieved considerable performance on various complex question-answering tasks by planning with external tools. Despite the incessant exploration in this field, existing language agent systems still struggle with costly, non-reproducible data reliance and face the challenge of compelling a single model for multiple functions. To this end, we introduce AutoAct, an automatic agent learning framework for QA that does not rely on large-scale annotated data and synthetic planning trajectories from closed-source models (e.g., GPT-4). Given limited data with a tool library, AutoAct first automatically synthesizes planning trajectories without any assistance from humans or strong closed-source models. Then, AutoAct leverages a division-of-labor strategy to automatically differentiate based on the target task information and synthesized trajectories, producing a sub-agent group to complete the task. We conduct comprehensive experiments with different LLMs, which demonstrates that AutoAct yields better or parallel performance compared to various strong baselines. Further analysis demonstrates the effectiveness of the division-of-labor strategy, with the trajectory quality generated by AutoAct generally outperforming that of others. Code will be available at https://github.com/zjunlp/AutoAct.

• The paper introduces a novel framework that uses self-instruction and task-specific sub-agents to generate its own training data, eliminating the need for extensive annotated datasets.
• The paper demonstrates that a Meta-Agent differentiating into specialized sub-agents produces high-quality planning trajectories that rival those of closed-source models.
• Experimental results confirm that AutoAct achieves robust performance across diverse QA tasks, highlighting its efficiency and potential for resource-constrained environments.

Introduction

Language agents are AI systems utilizing LLMs that can perform a variety of complex tasks by interpreting and interacting with external information. These agents have significantly advanced through the ability to understand tasks, generate plans, use external tools, and learn from past experiences. However, many existing agent learning systems depend on extensive annotated datasets and synthetic data generated by proprietary models like GPT-4. Additionally, designing agent frameworks often puts excessive pressure on a single model to master multiple functions, in contrast to the division of labor principle suggested by researcher Simon Mintrom.

AutoAct Framework

To address these issues, researchers have developed AutoAct, an agent learning framework that autonomously learns to plan and complete tasks without relying on large, annotated datasets or proprietary models. Instead, it uses a limited amount of initial data provided by users. The structure of AutoAct is highlighted by its Meta-Agent, which is capable of differentiating into a group of sub-agents, each specializing in specific functions—task decomposition, tool invocation, and self-reflection.

AutoAct begins with a process called "self-instruction," where the Meta-Agent expands a database of task data using a few given examples. Then, equipped with a tool library, it autonomously synthesizes planning trajectories. Finally, it differentiates into sub-agents optimized for specific parts of the planning process, a procedure that is both resource-efficient and adaptive to various task scenarios. A division-of-labor strategy enhances the overall capability of the agent system to address complex tasks.

Comparative Performance

Experimental assessments of AutoAct have shown that it performs comparably or even surpasses several strong baselines across different LLM platforms. One notable result is that the framework, when paired with the Llama-2-13b model, achieved comparable performance to that of the GPT-3.5-Turbo agent. This demonstrates the efficiency and effectiveness of AutoAct as it seeks to elevate the performance of open-source models to that of their closed-source counterparts.

The assessment also extended to multiple agent-learning methodologies and compared them to various prompt-based agents. AutoAct displayed impressive results, often outperforming agent-learning frameworks that emphasize either iterative planning or chain-of-thought reasoning.

Findings and Contributions

The success of AutoAct can be ascribed to several critical features. First, it negates the need for heavily annotated datasets and the use of closed-source model trajectories by automating the generation of its own training data. Second, by dividing labor among a group of specialized agents, it overcomes the limitations of overburdening a single agent with numerous planning tasks. The empirical analysis highlighted that AutoAct excels in generating high-quality planning trajectories and displays robust performance across various tasks.

The study's main contributions lie in proposing an automatic agent learning framework that adheres to the principle of bounded rationality, showing the capability of using different LLMs to achieve outstanding performance, and revealing the effectiveness of a division-of-labor strategy within the agent learning domain.

Conclusion

AutoAct is a new development in the field of agent learning frameworks. Its design respects the limitations of individual agents and harnesses their combined strengths to handle complex tasks more efficiently. By enabling more profound learning capabilities without relying on vast amounts of training data or closed-source models, AutoAct represents a significant step forward in the ongoing development of smarter, more autonomous AI agents.