Rethinking Code Complexity Through the Lens of Large Language Models

Abstract: Code complexity metrics such as cyclomatic complexity have long been used to assess software quality and maintainability. With the rapid advancement of LLMs on code understanding and generation tasks, an important yet underexplored question arises: do these traditional complexity metrics meaningfully characterize the difficulty LLMs experience when processing code? In this work, we empirically demonstrate that, after controlling for code length, classical metrics exhibit no consistent correlation with LLM performance, revealing a fundamental mismatch with model-perceived difficulty. To address this gap, we propose LM-CC, a novel code complexity metric designed from the perspective of LLMs. The core premise of LM-CC is that LLM-perceived difficulty is driven by the nonlinearity of program semantics. Accordingly, we decompose programs into semantic units based on entropy, organize these units into a compositional hierarchy, and quantify complexity as a principled aggregation of compositional level and branching-induced divergence, capturing cumulative model uncertainty during code processing. Our extensive experiments show that LM-CC not only correlates more strongly with LLM performance than traditional metrics but also that lowering it directly enhances task performance.

• The paper presents LM-CC, a model-aware metric that integrates compositional hierarchy with branching-induced divergence to capture LLM-perceived code difficulty.
• It empirically demonstrates that LM-CC strongly correlates (ranging from -0.92 to -0.97) with LLM performance, outperforming traditional complexity metrics.
• The study shows that semantics-preserving code rewriting guided by LM-CC can enhance LLM-based task performance by up to 20.9%.

Rethinking Code Complexity Through the Lens of LLMs

Introduction

Code complexity metrics, such as cyclomatic complexity, have been essential in software engineering for evaluating software quality and maintainability. This paper challenges the assumption that these traditional metrics adequately characterize the difficulty encountered by LLMs in code processing. It posits that the nonlinearity of program semantics, rather than structural complexity, drives the LLM-perceived difficulty. Consequently, the authors propose LM-CC, a novel metric that integrates compositional hierarchy with branching-induced divergence, thereby capturing model uncertainty.

Revisiting Current Complexity Metrics

The authors conduct empirical studies on traditional code complexity metrics and their correlation with LLM performance across various code-related tasks. They observe that metrics such as cyclomatic complexity, Halstead Complexity, Maintainability Index, and Cognitive Complexity show inconsistent correlations with LLM performance when controlling for code length.

Figure 1: Comparison between Cyclomatic Complexity (CC) and our proposed LM-CC. While CC assigns identical values to code snippets with significantly different cognitive loads for LLMs (top), LM-CC effectively distinguishes them by capturing the model uncertainty on non-linear code semantics (bottom).

Proposed Metric: LM-CC

Hierarchical Semantic Decomposition

To address the identified gap, LM-CC organizes source code into entropy-aligned semantic units, forming a hierarchical semantic decomposition that reflects compositional nesting and branching in programs.

Figure 2: Hierarchical Semantic Decomposition Example. Left: source code with token-entropy annotations, where color-coded regions indicate elevated LLM uncertainty. Right: the induced hierarchical semantic representation, with elements color-aligned to their semantic units of source code.

Feature Extraction and Metric Definition

LM-CC incorporates features that measure compositional level and branching factor to quantify code complexity. It uses a weighted aggregation of TotalBranch and TotalCompLevel, parameterized by a weighting factor $\alpha$, capturing the intricate dependency structures encountered by LLMs during code processing.

Figure 3: Ablation on the weighting factor $\alpha$ in LM-CC. Performance peaks at intermediate $\alpha$ values, while hierarchy-only ($\alpha \to 0$) and branching-only ($\alpha \to 1$) configurations perform substantially worse.

Theoretical Justification

The paper provides theoretical underpinnings indicating that LM-CC outperforms traditional metrics by aligning with the predictive entropy accumulation in hierarchical code structures, a factor neglected by conventional cyclomatic complexity measures.

Experiments

Correlation with Model Performance

Extensive experiments demonstrate that LM-CC exhibits strong partial correlations (ranging from $-0.92$ to $-0.97$) with LLM performance, surpassing traditional metrics in predicting model difficulty.

Causal Impact Through Complexity Reduction

By employing semantics-preserving code rewriting, researchers further verify that reducing LM-CC enhances LLM-based task performance by up to 20.9%.

Implications of LM-CC

LM-CC offers practical utility in refining LLM-centric tasks such as code evaluation, refactoring, reasoning, and training data curation. By guiding semantics-preserving transformations, LM-CC can improve LLM efficiency in understanding and processing complex code structures.

Conclusion

The paper introduces LM-CC, a model-aware complexity metric redefining code complexity assessment for LLMs. Through empirical validation, LM-CC demonstrates its superior alignment with LLM-perceived difficulty over traditional metrics, laying groundwork for improved model performance in code-related tasks. Future applications of LM-CC in AI-driven software development present promising avenues for enhancing code intelligence methodologies.

By incorporating these insights, researchers are better equipped to design LLMs that are robust in handling the intricacies of complex code semantics—ultimately advancing AI-driven programming tools.