ARCeR: an Agentic RAG for the Automated Definition of Cyber Ranges

Abstract: The growing and evolving landscape of cybersecurity threats necessitates the development of supporting tools and platforms that allow for the creation of realistic IT environments operating within virtual, controlled settings as Cyber Ranges (CRs). CRs can be exploited for analyzing vulnerabilities and experimenting with the effectiveness of devised countermeasures, as well as serving as training environments for building cyber security skills and abilities for IT operators. This paper proposes ARCeR as an innovative solution for the automatic generation and deployment of CRs, starting from user-provided descriptions in a natural language. ARCeR relies on the Agentic RAG paradigm, which allows it to fully exploit state-of-art AI technologies. Experimental results show that ARCeR is able to successfully process prompts even in cases that LLMs or basic RAG systems are not able to cope with. Furthermore, ARCeR is able to target any CR framework provided that specific knowledge is made available to it.

• The paper introduces ARCeR, an innovative agentic RAG framework that automates Cyber Range configuration from natural language descriptions.
• The method leverages advanced LLMs, including Claude 3.7 Sonnet and Sentence Transformer Models, and employs retrieval techniques to supply domain-specific data.
• The evaluation shows that ARCeR consistently achieves perfect accuracy in generating viable cyber configurations, with iterative self-correction addressing syntax issues.

ARCeR: An Agentic RAG for the Automated Definition of Cyber Ranges

Introduction

The paper "ARCeR: an Agentic RAG for the Automated Definition of Cyber Ranges" addresses the evolving landscape of cybersecurity and the need for creating realistic IT environments to train professionals in handling threats. The proposed solution efficiently automates the generation and deployment of Cyber Ranges (CRs) using an Agentic Retrieval-Augmented Generation (RAG) paradigm. This approach capitalizes on advanced AI technologies to generate CRs from natural language descriptions, thereby simplifying the traditionally labor-intensive process of designing and deploying custom scenarios.

Cyber Ranges and Their Significance

Cyber Ranges, as defined by the NIST, are simulated network environments providing a secure and controlled space for cybersecurity training and testing. These environments emulate real-world IT infrastructures and cyber threats, allowing organizations to prepare for various cybersecurity scenarios without jeopardizing actual systems. The paper emphasizes the critical role of CRs in cybersecurity training, making them indispensable tools in both educational and corporate settings.

Agentic RAG for CR Generation

The paper introduces an innovative system based on the Agentic RAG model, which effectively leverages AI for generating CR configuration files from high-level textual inputs. The Agentic RAG framework incorporates advanced LLMs and external tools to dynamically adapt and retrieve necessary information for accurate CR deployment.

Key Features of the Proposed System:

• Versatility: The system supports multiple CR frameworks by merely adapting the reference documents used.
• User-Friendliness: Instructors can use natural language to specify CR characteristics, making the process accessible to varying expertise levels.
• Cost-Effectiveness: Eliminates the need for extensive LLM fine-tuning by using retrieval techniques to enhance pre-trained models with domain-specific knowledge.

Implementation Details

The system utilizes Anthropic's Claude 3.7 Sonnet, coupled with a Sentence Transformer Model for embeddings, to drive the LLM's reasoning core. Its architecture also integrates a RAG subsystem that employs the Maximal Marginal Relevance (MMR) technique to enhance the retrieval of relevant documents from a vector store. This setup ensures a comprehensive supply of pertinent information to the LLM, facilitating precise CR configurations.

Figure 1: Overall approach schema.

Evaluation and Results

The evaluation consisted of a comprehensive analysis of the system's performance in generating CR descriptions via different approaches: a basic LLM, a standalone RAG, and the full Agentic RAG configuration. The Agentic RAG consistently outperformed the others, achieving perfect accuracy in generating viable CR configurations across tested scenarios. The iterative self-correction mechanism through tool-calling was pivotal in addressing syntax errors during configuration file generation.

Figure 2: Performance analysis.

Discussion and Limitations

Despite its effectiveness, the system has constraints, such as its inability to preemptively verify if user requests align with a CR framework's capabilities. Another limitation is its current focus on syntactic validation rather than semantic accuracy, which requires manual checks for potential discrepancies.

The adaptability of the Agentic RAG significantly contributes to the efficiency of CR generation and deployment, offering a promising direction for scalable and flexible cybersecurity training solutions.

Conclusion

The paper successfully demonstrates the applicability of the Agentic RAG paradigm in automating the definition and deployment of Cyber Ranges. With its ability to interface with various CR frameworks and adapt to user specifications, ARCeR signifies a substantive advancement in cybersecurity training methodologies, promising a more accessible and cost-effective approach to CR development. Future research could focus on further refining semantic correction capabilities and extending the system's adaptability to even more diverse CR frameworks.