The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code

Abstract: As LLMs are transforming software development, the functional quality of generated code has become a central focus, leaving readability, one of critical non-functional attributes, understudied. Given that LLM-generated code still needs human review before adoption, it is important to understand its readability especially compared with human-written code and the role of prompt design in shaping it. We therefore set out to conduct a systematic investigation into the code readability of LLM-generated code. To systematically quantify code readability, We establish a comprehensive readability model that synthesizes textual, structural, program, and visual features of code. Based on the model, we evaluate the readability of code generated by the mainstream LLMs under 5,869 scenarios extracted from large code base including World of Code (WoC) and LeetCode. We find that current LLMs produce code with overall readability comparable to human-written code, but displaying distinct readability issue patterns. We further examine how different prompt dimensions affect the readability of LLM-generated code, and find that function signatures, constraints and style descriptions emerge as the most influential factors, while the overall impact of prompt design remains limited. Our findings indicate that, on one hand, LLM-generated code is at least comparable to human-written code in readability, validating its potential for systematic integration into software workflows from a non-functional perspective; on the other hand, distinct readability issue patterns and limited effectiveness of prompt engineering reveal a latent technical debt, highlighting the need for future research to improve the readability of LLM-generated code and thus ensure long-term maintainability.

• The paper introduces a multi-dimensional readability benchmark comparing LLM-generated and human-written Python code, achieving up to 83.8% AUC.
• It identifies distinct readability defect patterns in AI-generated code, such as excessive complexity and redundant comments, highlighting LLM-specific challenges.
• Prompt engineering showed limited improvements, underscoring the need for alternative strategies to mitigate technical debt in AI-assisted development.

Authoritative Summary: The Readability Spectrum in LLM-Generated Code

Motivation and Research Objectives

Recent advancements in LLMs have transformed code generation in both professional software engineering and algorithmic programming contexts. While most research emphasizes functional correctness and security, this study systematically interrogates the readability of code generated by LLMs, a crucial non-functional attribute affecting maintenance and collaborative workflows. The authors frame their contributions around three central questions: (1) Is LLM-generated code more or less readable than human-written code? (2) Do LLMs exhibit distinct readability issue patterns compared to humans? (3) How do prompt engineering strategies influence code readability?

Evaluation Framework and Methodology

The paper introduces a comprehensive readability benchmark, aggregating 5,869 Python code scenarios from World of Code (WoC) and LeetCode, pre-dating widespread LLM adoption. The authors construct a multidimensional readability model synthesizing textual, structural, program, and visual features (TF, BWF, PF, DF) and validate its predictive capacity via stratified cross-validation, yielding peak metrics of 77.5% accuracy and 83.8% AUC.

The methodological pipeline encompasses prompt engineering across eight distinct dimensions, empirical evaluation of LLMs (GPT-4o, Grok-3, Claude-3.7, DeepSeek-v3, Llama 3), and rigorous statistical analyses (random forest regression, t-tests, permutation importance, ablation studies).

Figure 1: Overview of the methodology integrating data collection, prompt engineering, LLM code generation, and multi-faceted readability assessment.

Quantitative Readability Comparison

LLM-generated code achieves statistically higher readability scores than human-written baselines, with win rates (LLM code outperforming human code) surpassing 50% across all models. For Claude-3.7, the win rate reaches 73.3% overall, with average readability scores consistently exceeding the human baseline. Wilcoxon signed-rank tests confirm the significance of these results ($p < 0.001$, $r = 0.398$ effect size).

However, human-written code exhibits superior consistency, with tighter score distributions and fewer outliers, whereas LLM-generated code fluctuates more dramatically. This fluctuating stability is attributed to the next-token generation dynamics of LLMs as opposed to established human coding habits. Cross-model comparison reveals Claude-3.7 as the top performer, especially for algorithmic tasks, while WoC data (reflecting real-world software contexts) introduces additional structural complexity.

Figure 2: Boxplot of the readability scores comparing LLM-generated and human-written code.

Figure 3: Readability score distributions for code produced by different LLMs, showing the relative strength of Claude-3.7.

Qualitative Analysis of Readability Issue Patterns

The study's thematic analysis documents divergent issue patterns between LLM-generated and human-authored code. Human code is predominantly challenged by Deficient Comments (DC) and Inconsistent Style (IS) within Textual and Structural dimensions, reflecting prioritization of functionality over documentation. In contrast, LLM-generated code suffers most from Excessive Complexity (EC) and Redundant Comments (RC) in Semantic/Lexical and Information-Theoretic domains.

Notably, three novel LLM-specific defect classes were identified:

• Unknown API usage: LLMs import or hallucinate external libraries without adequate context.
• Redundant Variable definitions: Token-by-token generation yields unnecessary variable proliferation.
• Overblanking: Excessive, semantically meaningless blank lines that degrade functional density.

These issues are indicative of fundamental architectural differences between probabilistic code generation and human experience-driven habits.

Prompt Engineering and Its Effects on Code Readability

Random forest regression and multi-pronged statistical analysis establish that function signatures, constraints, and style descriptions are the only prompt dimensions reliably enhancing code readability. Function signatures anchor code structure, constraints and style descriptions provide explicit guidance for readability optimization. However, the overall $R^2$ of these models was low ($<0.3$), evidencing the limited absolute impact of prompt design on readability. Most prompt enhancements yield negligible gains, and only specific guidance within minimal prompts yields statistically significant improvements.

Figure 4: Feature importance of prompt dimensions based on Random Forest regression; function signatures rank highest, followed by constraints and style descriptions.

Implications for Practice and Future Directions

The empirical findings support systematic LLM integration in software workflows from a readability perspective, but concurrently expose latent "technical debt" in AI-generated code. Practically, this mandates development of AI-specific reviewer tools and attribution mechanisms leveraging high-level readability patterns. Architectural AI technical debt must be proactively managed with automated refactoring agents trained to target LLM-specific defects.

The limited efficacy of prompt engineering underscores the need for alternative interventions, including LLM fine-tuning focused on non-functional objectives, post-processing workflows, and collaborative human-AI program synthesis. The comprehensive readability dataset and benchmark established here are positioned to serve as baselines for future multi-dimensional model evaluation and automated readability optimization.

Conclusion

This paper delivers a rigorous, multi-perspective evaluation of LLM-generated code readability across thousands of real and algorithmic tasks. While LLMs achieve quantitative readability parity with humans, they manifest qualitatively distinct and contrasting issue patterns, including several emergent defects unique to AI generation. Prompt engineering only minimally improves readability, necessitating broader strategies for code quality assurance. The results simultaneously validate LLMs' non-functional integration potential and highlight the imperative for targeted research and tooling to address technical debt and sustain maintainability in AI-assisted software development.