VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool

Abstract: Due to the growing complexity of modern Integrated Circuits (ICs), automating hardware design can prevent a significant amount of human error from the engineering process and result in less errors. Verilog is a popular hardware description language for designing and modeling digital systems; thus, Verilog generation is one of the emerging areas of research to facilitate the design process. In this work, we propose VerilogCoder, a system of multiple AI agents for Verilog code generation, to autonomously write Verilog code and fix syntax and functional errors using collaborative Verilog tools (i.e., syntax checker, simulator, and waveform tracer). Firstly, we propose a task planner that utilizes a novel Task and Circuit Relation Graph retrieval method to construct a holistic plan based on module descriptions. To debug and fix functional errors, we develop a novel and efficient abstract syntax tree (AST)-based waveform tracing tool, which is integrated within the autonomous Verilog completion flow. The proposed methodology successfully generates 94.2% syntactically and functionally correct Verilog code, surpassing the state-of-the-art methods by 33.9% on the VerilogEval-Human v2 benchmark.

• The paper presents an autonomous AI system that uses a task-driven circuit relation graph to generate and debug Verilog code.
• It employs a multi-agent framework with distinct code and debug agents, leveraging AST-based waveform tracing for functional verification.
• Experimental results show a 94.2% accuracy in code generation on the VerilogEval-Human v2 benchmark, outperforming previous methods by 33.9%.

VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool

The paper introduces "VerilogCoder", an architecture consisting of autonomous AI agents for generating and debugging Verilog code using a task-driven circuit relation graph (TCRG) and an abstract syntax tree (AST)-based waveform tracing tool. This approach aims to improve the correctness of Verilog code, both syntactically and functionally, surpassing existing state-of-the-art techniques.

Introduction

The complexity of modern integrated circuit (IC) design necessitates sophisticated methods for automated hardware description languages (HDL) processing. Verilog, as a prominent HDL, requires robust tools to minimize the designer's overhead associated with iterative coding and debugging. VerilogCoder deploys LLMs in conjunction with AI agents to automate these tasks, providing a scalable solution that addresses both syntax and functionality concerns.

VerilogCoder Architecture

VerilogCoder's architecture encompasses several modules: task planning, multi-agent reasoning, and debugging. A novel TCRG is employed to aid in task planning, while a comprehensive Verilog toolkit supports the multi-agent strategy. This toolkit includes syntax checkers and a novel AST-based waveform tracing tool.

Task Planning

The task planning phase leverages a TCRG to structure sub-tasks, incorporating detailed signal and transition information. The TCRG-based task planner constructs high-quality plans that incorporate essential circuitry details, which are often omitted by conventional LLM planning strategies.

Figure 1: Flow overview of VerilogCoder illustrating the comprehensive plan generation for Verilog code completion.

Verilog Code Implementation with Multi-AI Agents

Verilog code implementation is driven by task plans generated from the TCRG. The multi-agent framework operates in a synchronized manner, with distinct agents handling code syntax verification and functional debugging.

• Code Agent: Responsible for initial code generation and syntax verification, supported by syntax checker tools.
• Debug Agent: Focuses on functional verification, using the AST-based waveform tracing tool to backtrace mismatches and guide corrections.

  Figure 2: Debug Agent configuration and interactions involving simulator and AST-based waveform tracing for functional validation.

Experimental Results

VerilogCoder achieves a 94.2% accuracy in generating syntactically and functionally correct Verilog code on the VerilogEval-Human v2 benchmark, outperforming previous methods by 33.9%. The AST-based tool significantly contributed to this performance, particularly in tasks requiring intricate waveform analysis.

Implications and Future Work

The introduction of VerilogCoder provides a framework that significantly enhances the efficiency of Verilog code generation in IC design processes. Future work can enhance the system's PPA (Power, Performance, Area) metrics through improved LLM training and memory management for continuous learning from previous coding experiences.

In summary, VerilogCoder represents a substantial advancement in autonomous HDL processing, facilitated by advanced task planning and debugging capabilities, demonstrating notable improvements over existing methods.