AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents

Abstract: Current research in LLM-based simulation systems lacks comprehensive solutions for modeling real-world court proceedings, while existing legal LLMs struggle with dynamic courtroom interactions. We present AgentCourt, a comprehensive legal simulation framework that addresses these challenges through adversarial evolution of LLM-based agents. Our AgentCourt introduces a new adversarial evolutionary approach for agents called AdvEvol, which performs dynamic knowledge learning and evolution through structured adversarial interactions in a simulated courtroom program, breaking the limitations of the traditional reliance on static knowledge bases or manual annotations. By simulating 1,000 civil cases, we construct an evolving knowledge base that enhances the agents' legal reasoning abilities. The evolved lawyer agents demonstrated outstanding performance on our newly introduced CourtBench benchmark, achieving a 12.1% improvement in performance compared to the original lawyer agents. Evaluations by professional lawyers confirm the effectiveness of our approach across three critical dimensions: cognitive agility, professional knowledge, and logical rigor. Beyond outperforming specialized legal models in interactive reasoning tasks, our findings emphasize the importance of adversarial learning in legal AI and suggest promising directions for extending simulation-based legal reasoning to broader judicial and regulatory contexts. The project's code is available at: https://github.com/relic-yuexi/AgentCourt

• The paper introduces an adversarial evolutionary framework that refines lawyer agents' legal reasoning through iterative simulation of real-world courtroom dynamics.
• Experimental evaluations over 1,000 civil cases demonstrate marked improvements in legal knowledge and argumentative capabilities using LawBench metrics and expert assessments.
• The research paves the way for innovative legal education and AI-based legal strategy development by open-sourcing a multi-agent courtroom simulation platform.

AgentCourt: Advancing Legal AI through Simulated Court Environments

The paper presents a compelling exploration into using LLMs to simulate court proceedings through a platform named AgentCourt. Designed as a multi-agent system, AgentCourt employs LLMs to drive autonomous agents functioning within a courtroom simulation, each representing various roles such as judges, attorneys, plaintiffs, and defendants. The principal objective of this research is to enhance the arguing and legal processing capabilities of lawyer agents through an iterative, adversarial simulation of court cases.

Core Contributions and Methodology

The authors introduce an adversarial evolutionary framework to refine the capabilities of lawyer agents within the simulation. At the heart of this approach is the autonomous development of courtroom skills by lawyer agents through an iterative adversarial process. The framework does not rely on fixed parameters but dynamically evolves through interactions, mirroring the experiential learning of real-world lawyers over extended periods. This evolutionary mechanism is designed to facilitate skills in legal reasoning, enhancing responsiveness, expertise, and logical coherence. Importantly, this technique enables lawyer agents to autonomously formulate effective defensive strategies by repeatedly engaging in legally adversarial proceedings.

Experimental Evaluation

The study provides substantial quantitative results demonstrating improvement in lawyer agents' performance through extensive evaluation. The researchers simulated 1,000 civil court cases, evaluating agent performance pre- and post-evolution. Automatic assessments using LawBench metrics showcased clear advances in task performance related to legal knowledge memorization, understanding, and application, with evolved agents surpassing their initial capabilities. Furthermore, manual evaluation by experienced legal professionals corroborates these findings, highlighting enhancement in cognitive agility, domain-specific knowledge, and logical rigor of agents post-evolution, thereby underscoring the model's potential to rival or even surpass the skills of conventional AI models such as GPT-4.

Practical and Theoretical Implications

AgentCourt's development carries significant implications for the future of AI in legal contexts. Practically, this system can be employed as an advanced tool for legal education, providing lawyer training and case analysis in a digitized, low-cost environment. Theoretically, this research extends the application of LLMs beyond conventional settings, offering a transformative platform for simulating complex multi-agent interactions such as court proceedings. Additionally, by open-sourcing their dataset and system, the authors aim to catalyze further advancements across the legal AI community.

Future Directions

The research outlined in this paper paves the way for numerous future explorations. Ongoing developments could include enhancing the system's ability to tackle increasingly complex legal scenarios and further refinement of agent roles to better diversify interactions within the simulation. Moreover, integrating more sophisticated linguistic strategies may lead to even more nuanced and realistic simulations of legal dialogue and argumentation.

In summary, AgentCourt represents a significant stride in utilizing AI to simulate legal proceedings. It effectively demonstrates how LLMs can be harnessed to foster expert legal reasoning within a carefully controlled virtual courtroom, presenting a powerful model for both educational and professional legal applications. Through its open-source contribution, this work encourages innovation and critical advancements within the field of legal AI, offering a pathway towards an intelligent, automated, and fair legal system.