Scalability of Step-Level Grounded Editing Methods to Complex Data Analysis Scripts

Determine whether interactive steering approaches that let users edit natural-language utterances grounded in each step of AI-generated code (for example, grounded abstraction matching and editable step-by-step explanations for SQL) can be extended to support longer and more complex data analysis scripts.

Background

The paper reviews prior methods for steering AI-assisted programming that allow users to edit natural-language descriptions grounded in each step of AI-generated code, such as grounded abstraction matching and editable step-by-step explanations for SQL. These techniques aim to provide an accessible abstraction level for reading, verifying, and modifying code by manipulating natural-language utterances tied to code steps.

However, the authors argue that such approaches hide the AI’s reasoning and decomposition process, making it difficult for users to understand the rationale behind generated code. While these methods may work for short programs (e.g., spreadsheet formulas or SQL queries), the authors explicitly state uncertainty about extending these approaches to longer and more complex data analysis scripts, motivating the need to investigate their scalability and suitability in such contexts.