GS-MCTS: Group-aware Strategy MCTS

• The paper introduces GS-MCTS, a novel tree search variant that overcomes LLM output randomness and combinatorial search challenges through group-based evaluation.
• It integrates adversarial strategy priors with classical MCTS phases, enabling the efficient generation of multi-turn adversarial attack sequences.
• Empirical benchmarks show GS-MCTS achieves up to 95.2% attack success with an average of 7.4 attempts, outperforming baseline adversarial methods.

Group-aware Strategy-guided Monte Carlo Tree Search (GS-MCTS), as introduced in "Adversarial Attack-Defense Co-Evolution for LLM Safety Alignment via Tree-Group Dual-Aware Search and Optimization" (Li et al., 24 Nov 2025), is a Monte Carlo Tree Search variant tailored for exposing "jailbreak" vulnerabilities in safety-aligned LLMs. By integrating group-aware evaluation and data-driven strategy priors into the standard four-phase MCTS, GS-MCTS efficiently discovers effective multi-turn attack sequences that lead to harmful LLM behavior. The approach is designed to counter two central obstacles in adversarial jailbreak: combinatorial strategy search and high output stochasticity even for fixed prompts. GS-MCTS outputs diverse, high-quality adversarial queries to facilitate the co-evolutionary training of both attack and defense models, and has demonstrated superior attack success in multiple empirical benchmarks.

1. Motivating Problem and Design Objectives

GS-MCTS addresses the challenge of reliably generating adversarial prompts—so-called jailbreaks—that subvert the safety mechanisms of LLMs. Two primary difficulties motivate this design: (1) the large, discrete combinatorial space of prompt-level attack strategies, each usable over multiple turns, and (2) inherent randomness in model generations, which obscures assessment of attack effectiveness. The method aims to (a) efficiently traverse the multi-turn prompt-rewrite search space, (b) mitigate LLM output randomness through group evaluation, and (c) supply diverse, challenging adversarial examples essential for systematic attack-defense co-evolution (Li et al., 24 Nov 2025).

2. Algorithmic Architecture and MCTS Integration

GS-MCTS augments standard Monte Carlo Tree Search by incorporating group-based evaluation and strategy-guided priors. The canonical MCTS four-phase workflow adapts as follows:

• Selection Phase: Traverses the search tree from the root (original malicious query) via a Predictor UCT (PUCT) rule, balancing the empirical mean reward $Q(s,a)$, a learned adversarial prior $P(s,a)$, and classic exploration scaling $\sqrt{\sum_k N(s,k)}/(1+N(s,a))$.
• Expansion Phase: Explores new actions $a^*$ not previously expanded at the current leaf, adding child nodes to represent untried attack strategies.
• Simulation (Evaluation) Phase: Executes explored strategies by generating $G$ parallel queries, passing each through the defense model, and scoring harmfulness via a judge. Node reward is set as the maximum harmfulness across the group, prioritizing the worst-case scenario.
• Backpropagation Phase: Propagates the observed reward upward, incrementing visit counts $N(s,a)$ and updating mean rewards $Q(s,a)$ for all traversed edges.

Group-wise simulation and priors over actions (grounded in attack and defense model confidences) distinguish GS-MCTS from classical approaches, directly addressing combinatorial and stochasticity issues in prompt attacks.

3. Formal Definitions, Notation, and Core Criteria

Let $p \in P$ denote the original input query and $A = \{a_1,\ldots,a_K\}$ the set of $K$ pre-defined strategy templates, such as "Role-playing," "Semantic Ambiguity," or "Logical Reversal." Each node in the search tree is indexed by a state

$s = (p, \hat{q}, \hat{o}, \hat{j}),$

comprising the root query, a group of $G$ modified queries $\hat{q}$, their corresponding LLM responses $\hat{o}$, and the judged groupwise assessments $\hat{j}$ per output.

The reward for an edge $(s,a)$ is defined by

$R(s,a) = \max_{i=1\dots G} j_i^h,$

where $j_i^h$ represents the harmfulness score of the $i^{th}$ generated answer. A node is marked "jailbroken" if $r \geq \eta$ for some threshold $\eta$ (and sufficient co-relevance).

For selection, an adversarial prior $P(s,a)$ is computed for each candidate action by averaging normalized attack confidence (log-probability under the attack model) and defense non-rejection score (one minus the log-probability of a refusal token in the defense model's output). The PUCT selection criterion combines empirical rewards and the prior:

$$a^* = \arg\max_{a \in A} \left[ Q(s,a) + c_p\, P(s,a) \frac{\sqrt{\sum_k N(s,k)}}{1 + N(s,a)} \right]$$

with $c_p>0$ governing exploration vs. exploitation.

Backpropagation updates are executed as:

$$N(s,a) \leftarrow N(s,a)+1, \qquad Q(s,a) \leftarrow \frac{Q(s,a)_{\text{old}} \cdot N_{\text{old}} + r}{N(s,a)}$$

4. Pseudocode Workflow and Implementation Guidelines

The following annotated pseudocode summarizes the GS-MCTS process—for $N_m$ search cycles, group size $G$, and harmfulness threshold $\eta$:

def GS_MCTS(p, N_m, eta, G): initialize_root_node(s0, p) for cycle in range(N_m): # 1. SELECTION s = s0 path = [] while s is fully expanded and not terminal: a_sel = argmax_a(Q(s, a) + c_p * P(s, a) * sqrt(sum N(s, k)) / (1 + N(s, a))) path.append((s, a_sel)) s = child(s, a_sel) # 2. EXPANSION if s is not terminal and has untried actions: a_new = pick_untried_action(s, A) s_prime = expand(s, a_new) path.append((s, a_new)) s = s_prime # 3. SIMULATION / GROUP EVALUATION q_group = M_a.generate_group(p, s, a_new, G) # strategy-guided o_group = M_D.answer_group(q_group) j_h_group = J_a.score_harm(p, q_group, o_group) r = max(j_h_group) if r >= eta and j_c is high: mark_jailbroken(s) # 4. BACKPROPAGATION for (s_prev, a_prev) in path: N[s_prev, a_prev] += 1 Q[s_prev, a_prev] = (Q[s_prev, a_prev] * (N-1) + r) / N[s_prev, a_prev] return best_path_or_first_jailbreak()

Practically, attack model $M_a$ is prompted with $\text{Prompt}_A(p, s, a)$, sampling $G=6$ outputs at temperature $0.9$ for maximal diversity. The defense model $M_D$ computes a refusal probability using the token set $$T_r = \{\mathrm{"I"}, \mathrm{"I'm"}, \mathrm{"As"}, \mathrm{"Sorry"}\}$$. Search depth is capped at $K$ (no repeated strategies beyond $|A|$ turns); cycles are limited to $N_m=50$.

Strategy taxonomy comprises 8 high-level categories (with 40+ sub-patterns), each corresponding to a unique prompt template. Early stopping is triggered on successful jailbreak discovery.

5. Comparative Performance, Ablation, and Convergence

Empirical studies with GS-MCTS in the ACE-Safety framework demonstrate robust attack effectiveness across standard benchmarks. On both MergedHarm (in-distribution) and Malicious-Instruct (out-of-distribution) test sets, GS-MCTS achieves an attack success rate (ASR) of up to $95.2\%$ with an average number of attempts (ANA) of approximately $7.4$, outperforming baselines such as GCG, PAIR, TAP, and MPA.

Ablation analysis reveals:

• Replacing GS-MCTS with random query modification ("w/o GS-MCTS") substantially increases attack success rate under low-resource conditions (ASR-LR rises from $8.8\%$ to over $20\%$).
• Removing the prior probability term $P(s,a)$ ("w/o prior probability") degrades defense robustness.
• Disabling early stopping on the jailbreak threshold results in diminished performance.
• Smaller group sizes ($G=1$) systematically underestimate harmfulness, while $G \geq 6$ yields stable, accurate $Q$-value estimates.
• Most improvements in attack efficacy occur within 2–3 MCTS iterations, with convergence typically reached by iteration 4.

This suggests that GS-MCTS's mix of probabilistic exploration (PUCT), group-based evaluation to mitigate output randomness, and strategy-aware priors is essential for generating effective, diverse adversarial samples and for advancing joint adversarial training (Li et al., 24 Nov 2025).

6. Connections, Implications, and Future Directions

GS-MCTS is a domain-specific, extensible data-driven search procedure that advances automated jailbreak prompt discovery. Its ability to combine prompt strategy compositionality, adversarial priors, and group-wise worst-case evaluation positions it as a strong candidate for adversarial robustness pipelines, particularly in arms-race scenarios such as attack-defense co-evolution. The methodological insights—such as group sampling for robust harmfulness estimation and PUCT-based integration of empirical reward with model priors—potentially generalize to related adversarial domains and reinforcement learning settings.

A plausible implication is that iterative, structured adversarial search algorithms like GS-MCTS will become integral to LLM safety verification and to curricula for robustifying both attack and defense models. Further investigation may address scaling to broader strategy spaces, integrating richer priors, or tailoring group evaluation metrics to evolving threat characteristics.