AlphaTrans: A Neuro-Symbolic Compositional Approach for Repository-Level Code Translation and Validation

Abstract: Code translation transforms programs from one programming language (PL) to another. Several rule-based transpilers have been designed to automate code translation between different pairs of PLs. However, the rules can become obsolete as the PLs evolve and cannot generalize to other PLs. Recent studies have explored the automation of code translation using LLMs. One key observation is that such techniques may work well for crafted benchmarks but fail to generalize to the scale and complexity of real-world projects with dependencies, custom types, PL-specific features, etc. We propose AlphaTrans, a neuro-symbolic approach to automate repository-level code translation. AlphaTrans translates both source and test code, and employs multiple levels of validation to ensure the translation preserves the functionality of the source program. To break down the problem for LLMs, AlphaTrans leverages program analysis to decompose the program into fragments and translates them in the reverse call order. We leveraged AlphaTrans to translate ten real-world open-source projects consisting of <836, 8575, 2719> classes, methods, and tests. AlphaTrans breaks down these projects into 17874 fragments and translates the entire repository. 96.40% of the translated fragments are syntactically correct, and AlphaTrans validates the translations' runtime behavior and functional correctness for 27.03% and 25.14% of fragments. On average, the integrated translation and validation take 34 hours to translate a project, showing its scalability in practice. For the incorrect translations, AlphaTrans generates a report including existing translation, stack trace, test errors, or assertion failures. We provided these artifacts to two developers to fix the translation bugs in four projects. They were able to fix the issues in 20.1 hours on average and achieve all passing tests.

• The paper introduces AlphaTrans, a neuro-symbolic technique combining program analysis and LLMs for translating entire code repositories, demonstrated from Java to Python.
• AlphaTrans employs a three-phase methodology covering program decomposition, type mapping, and iterative compositional translation with functional validation techniques.
• Evaluation shows high syntactic correctness (99.1%) and successful type mapping (92%), identifying functional validation tied to test coverage as a primary challenge and area for future research.

Repository-Level Compositional Code Translation and Validation

The paper "Repository-Level Compositional Code Translation and Validation" proposes AlphaTrans, a novel technique for automated code translation at the repository level, focusing on the critical task of translating entire repositories from Java to Python. Unlike traditional methods that focus on rule-based transpilers or crafted benchmarks, AlphaTrans employs a neuro-symbolic approach that integrates program analysis and the capabilities of advanced LLMs. This technique addresses prevalent challenges in code translation, such as complexity, validation, and contextualization of code.

Technical Contributions

AlphaTrans is introduced as a comprehensive answer to several limitations in current code translation approaches. The methodology is built in three distinct phases: program transformation and decomposition, type translation and skeleton construction, and compositional translation and validation.

1. Program Transformation and Decomposition: In this phase, AlphaTrans tackles the complexity associated with language-specific features like method and constructor overloading in Java. Through refactoring, these constructs are transformed into expressions more amenable to Python, facilitating accurate translation. Furthermore, the decomposition of the program into smaller fragments enables more manageable inputs for the LLM, overcoming the limitations of its context window.
2. Type Translation and Skeleton Construction: A crucial part of establishing an executable target language structure is creating a type mapping system. Here, AlphaTrans adapts types from the source language to suitable counterparts in the target language (Python in this case), building a project skeleton that can support iterative inclusion of translated code fragments.
3. Compositional Translation and Validation: This phase utilizes iterative feedback and validation techniques to ensure the translated code is functionally equivalent to the source. By leveraging GraalVM's language interoperability features, AlphaTrans inspects runtime behavior and functional correctness through both source hooks and translated tests.

Results and Evaluation

The authors demonstrate AlphaTrans's efficacy by testing it on ten substantial open-source Java projects, numbering over 8,500 methods and more than 2,700 tests. The translation system achieves a high syntactical correctness rate of 99.1%. Functional correctness is validated at 25.8%, albeit with a test coverage constraint influencing this figure. The adoption of automated type translation yielded successful mappings for approximately 92% of types.

Implications and Future Directions

The results show that while AlphaTrans is highly effective in producing syntactically correct translations, the main challenge lies in functional validation, largely contingent on the quality and coverage of the original test suite. This suggests avenues for future work in improving translation validation through enhanced test generation, possibly leveraging LLMs for this purpose.

Another important implication is the potential scalability of AlphaTrans across multiple language pairs beyond Java and Python, given sufficient adaptation of program decomposition and type translation processes. Additionally, enhancing the LLM's ability to correctly infer equivalent idiomatic expressions between languages remains an open research avenue.

Conclusion

The paper presents AlphaTrans as a robust step forward in repository-level code translation by integrating symbolic methods with the generative capabilities of LLMs. Though challenges in complete functional validation remain, this approach provides effective solutions to many challenges in the field, particularly with respect to syntactical correctness and adaptation to LLMs. Future enhancements focused on validation could further solidify its place as a standard method for comprehensive codebase translation.