Generalizability of the proposed GraphRAG framework beyond the code migration domain

Determine whether the proposed GraphRAG framework—comprising dependency parsing-based knowledge graph construction and a hybrid retrieval strategy that fuses one-hop graph traversal with Reciprocal Rank Fusion—generalizes effectively beyond the ABAP custom code migration domain by rigorously evaluating its performance and applicability in other settings, including public question answering benchmarks such as HotpotQA.

Background

The paper introduces a practical GraphRAG framework tailored for enterprise-scale deployment, emphasizing efficient knowledge graph construction via dependency parsing and a hybrid retrieval approach that combines graph traversal with vector similarity using Reciprocal Rank Fusion. The system is evaluated on enterprise datasets related to ABAP custom code migration, demonstrating notable improvements over dense vector baselines while significantly reducing construction costs compared to LLM-based extraction.

Despite strong results within the code migration domain, the authors explicitly note uncertainty about how well the approach transfers to other domains and tasks. They propose future evaluations on broader public benchmarks, such as HotpotQA, to assess whether the dependency-based construction and hybrid retrieval strategies maintain effectiveness outside the tested enterprise context.