Memory Error Detection in Security Testing

Abstract: We study 10 C/C++ projects that have been using a static analysis security testing tool. We analyze the historical scan reports generated by the tool and study how frequently memory-related alerts appeared. We also studied the subsequent developer action on those alerts. We also look at the CVEs published for these projects within the study timeline and investigate how many of them are memory related. Moreover, for one of this project, Linux, we investigate if the involved flaws in the CVE were identified by the studied security tool when they were first introduced in the code. We found memory related alerts to be frequently detected during static analysis security testing. However, based on how actively the project developers are monitoring the tool alerts, these errors can take years to get fixed. For the ten studied projects, we found a median lifespan of 77 days before memory alerts get fixed. We also find that around 40% of the published CVEs for the studied C/C++ projects are related to memory. These memory CVEs have higher CVSS severity ratings and likelihood of having an exploit script public than non-memory CVEs. We also found only 2.5% Linux CVEs were possibly detected during static analysis security testing.

• The paper demonstrates that memory-related alerts account for 35.4% of all static analysis detections in C/C++ projects.
• It reveals that the median lifespan of these alerts is 77 days, with only 39.9% acted upon, underscoring delayed developer response.
• The study shows that nearly 40% of CVEs are memory-related, receiving higher severity scores and 59% featuring exploit scripts.

Memory Error Detection in Security Testing

This essay examines the paper "Memory Error Detection in Security Testing" which studies the frequency and subsequent developer actions concerning memory-related alerts identified by static analysis security testing tools in C/C++ projects. It further analyzes the relationship between these alerts and publicly reported Common Vulnerabilities and Exposures (CVEs), focusing on their severity and exploitability.

Frequency and Developer Response to Memory Alerts

The study evaluates static analysis reports from ten C/C++ projects. Memory-related alerts account for approximately 35.4% of the total alerts detected (median across the projects) (Figure 1). Despite the frequent detection of memory issues, the median lifespan of these alerts is 77 days, indicating delays in developer action. Only about 39.9% of memory alerts are acted upon by developers, with a subset of 12.4% explicitly marked as bugs.

Figure 1: Determining actionability of static analysis alerts.

Actionability of Alerts

The actionable nature of alerts—those fixed through code changes—varies, with a noticeable portion remaining unaddressed or triaged as false positives. The study also highlights that memory alerts are rarely marked as intentional coding practices, suggesting genuine detection of potential security issues.

Lifespan and Fix Complexity

The median lifespan of actionable memory alerts is consistent across projects, with minimal differences in fixing time compared to non-memory alerts. The average complexity of these fixes is low, often requiring changes to a few lines of code and involving minor logical changes within the codebase.

Memory-Related CVEs

About 40% of publicly reported vulnerabilities (CVEs) in the study's context are memory-related, receiving higher CVSS severity ratings compared to non-memory CVEs. The average severity score for memory CVEs is significantly greater in both CVSS2 and CVSS3 ratings. A considerable 59% of memory CVEs have publicly available exploit scripts, emphasizing their critical nature.

Identification by Static Analysis

The paper examines whether these vulnerabilities were detectable when initially introduced into the code. In Linux, as a case study, only 2.5% of the memory-related CVEs showed a possible correlation with static analysis alerts when the code was first committed. This finding underscores limitations in either the static analysis tools’ capabilities or the developers' responsiveness to these alerts.

Implications

The results suggest that while static analysis tools can frequently detect memory-related vulnerabilities, the efficacy of these tools is limited by delayed developer responses and potential oversight. The correlation between memory-related alerts and high-severity CVEs highlights the critical need for prompt and effective handling of such alerts.

The findings advocate for enhanced tool precision to reduce false positives and increased developer engagement with security alerts. Moreover, improving static analysis tools' capability to identify issues when code is first introduced could preemptively mitigate potential vulnerabilities.

Conclusion

The paper highlights the prevalence of memory-related security issues within C/C++ projects and their significant impact on security vulnerabilities. While static analysis tools are pivotal in identifying these issues, the process efficacy is hampered by delayed remedial actions and tool limitations. Future developments should focus on enhancing detection accuracy and encouraging prompt developer intervention to address these critical memory errors effectively.