ChatCoder: Chat-based Refine Requirement Improves LLMs' Code Generation

Abstract: LLMs have shown good performances in generating code to meet human requirements. However, human requirements expressed in natural languages can be vague, incomplete, and ambiguous, leading LLMs to misunderstand human requirements and make mistakes. Worse, it is difficult for a human user to refine the requirement. To help human users refine their requirements and improve LLMs' code generation performances, we propose ChatCoder: a method to refine the requirements via chatting with LLMs. We design a chat scheme in which the LLMs will guide the human users to refine their expression of requirements to be more precise, unambiguous, and complete than before. Experiments show that ChatCoder has improved existing LLMs' performance by a large margin. Besides, ChatCoder has the advantage over refine-based methods and LLMs fine-tuned via human response.