Validation of Automatically Generated Patches: An Appetizer

Abstract: In the context of test case based automated program repair (APR), the research community call the patches that pass all the test cases but fail to actually fix the bug test case overfitted patches. Currently, overfitted patches has to be manually inspected by the users. Being a labor intensive activity that hinders widespread adoption of APR tools, automatic validation of APR-generated patches has been the topic of research in recent years. In this paper, we point out the limitations of the existing techniques/methodologies that call for further research, and introduce two promising directions toward effective automatic patch validation: (1) motivated by the relative effectiveness of anti-patterns, we propose to use statistical techniques to avoid the uncomputability of applying some of the anti-pattern rules and automate the technique. Our results show that we achieve at least 57% precision. (2) We present a proposal for a semi-automatic technique that helps the programmers in finding properties of the patched methods and stress testing the patches based on those properties so as to filter out overfitted ones as many as possible.