MASTER: Multi-Agent Security Through Exploration of Roles and Topological Structures -- A Comprehensive Framework

Abstract: LLMs-based Multi-Agent Systems (MAS) exhibit remarkable problem-solving and task planning capabilities across diverse domains due to their specialized agentic roles and collaborative interactions. However, this also amplifies the severity of security risks under MAS attacks. To address this, we introduce MASTER, a novel security research framework for MAS, focusing on diverse Role configurations and Topological structures across various scenarios. MASTER offers an automated construction process for different MAS setups and an information-flow-based interaction paradigm. To tackle MAS security challenges in varied scenarios, we design a scenario-adaptive, extensible attack strategy utilizing role and topological information, which dynamically allocates targeted, domain-specific attack tasks for collaborative agent execution. Our experiments demonstrate that such an attack, leveraging role and topological information, exhibits significant destructive potential across most models. Additionally, we propose corresponding defense strategies, substantially enhancing MAS resilience across diverse scenarios. We anticipate that our framework and findings will provide valuable insights for future research into MAS security challenges.

• The paper introduces MASTER, a framework that enhances LLM-based MAS security by addressing vulnerabilities in role assignments and topologies.
• The methodology employs an automatic MAS constructor and an adaptive attack strategy, significantly increasing the Attack Success Rate over conventional methods.
• Defense strategies, including prompt leakage defense, hierarchical monitoring, and preemptive measures, reduce ASR below 20%, demonstrating robust security improvements.

MASTER: A Comprehensive Multi-Agent Security Framework

The paper introduces MASTER, a pioneering framework designed to enhance the security of LLM-based Multi-Agent Systems (MAS) through exploration of roles and topological structures. MASTER aims to address the inherent security risks posed by the collaborative and heterogeneous nature of MAS in various domains.

Introduction to MASTER

MASTER provides a robust, automated construction process for diverse MAS configurations and introduces a new, scenario-adaptive attack strategy. The framework emphasizes the importance of understanding role assignments and topological structures in MAS, which are often exploited in security breaches. MASTER features five core components: Constructor, Interaction, Attack, Defense, and Evaluation.

Figure 1: MASTER is the first MAS security research framework that comprehensively considers different scenarios of roles and topological structures in MAS.

Methodology

MAS Automatic Constructor

The MAS Automatic Constructor facilitates the creation of MAS with varying roles and topologies. It employs two key phases: Topology Selection and Role Assignment. Topologies such as Chain, Tree, Star, Circle, Hierarchy, and Complete are evaluated by selector agents, and roles are assigned based on task requirements.

Adaptive Attack Strategy

MASTER presents a unique attack strategy divided into three stages: information probing, trait injection, and activation. The probing stage gathers detailed role and topology information, which is then used to craft scenario-specific trait injections. Finally, these traits are activated, leveraging MAS's collaborative structure to amplify security breaches.

Figure 2: Overview of MASTER. MASTER consists of five parts: Constructor, Interaction, Attack, Defense, and Evaluation.

Defense Strategy

Three major defense strategies are incorporated:

1. Prompt Leakage Defense: Detects attempts at prompt leakage during the probing phase.
2. Hierarchical Monitoring: Implements importance-based monitoring, prioritizing agents based on their role and topological position.
3. Scenario-Based Preemptive Defense: Analyses MAS scenarios to configure agents for early defense activation.

Evaluation Metrics

MASTER introduces new metrics to evaluate MAS security:

• Adversarial Role Consistency: Measures the degree to which agents maintain harmful roles post-attack.
• Cooperative Harmful Behavior: Evaluates team efficiency in executing malicious tasks.

Experimental Analysis

The paper provides thorough experiments across various MAS configurations, demonstrating the enhanced attack potential using role and topology knowledge. MASTER's attack strategies significantly increase the Attack Success Rate (ASR) compared to traditional methods.

Figure 3: Results of Different Domain showing ASR, adversarial role consistency, and cooperative behavior.

Defense Evaluation

Defense strategies effectively reduce ASR to below 20% even with heightened attack propagation. These approaches offer a robust countermeasure against the vulnerabilities amplified by role and topology exploitation.

Observations and Future Directions

The study highlights critical vulnerabilities in MAS—especially in data management and hierarchical topologies. Findings suggest the need for continuous research into adaptive security frameworks capable of addressing evolving MAS configurations.

Conclusion

MASTER establishes a comprehensive framework for MAS security, emphasizing the criticality of role and topology considerations. Its contributions offer a solid foundation for future MAS security research, aiming to mitigate the security risks inherent in agent interactions across diverse domains.

This pioneering work lays the groundwork for developing safer, robust MAS by systematically addressing the security challenges posed by their structural complexity and collaborative nature.