Boundary Point Jailbreaking of Black-Box LLMs

Abstract: Frontier LLMs are safeguarded against attempts to extract harmful information via adversarial prompts known as "jailbreaks". Recently, defenders have developed classifier-based systems that have survived thousands of hours of human red teaming. We introduce Boundary Point Jailbreaking (BPJ), a new class of automated jailbreak attacks that evade the strongest industry-deployed safeguards. Unlike previous attacks that rely on white/grey-box assumptions (such as classifier scores or gradients) or libraries of existing jailbreaks, BPJ is fully black-box and uses only a single bit of information per query: whether or not the classifier flags the interaction. To achieve this, BPJ addresses the core difficulty in optimising attacks against robust real-world defences: evaluating whether a proposed modification to an attack is an improvement. Instead of directly trying to learn an attack for a target harmful string, BPJ converts the string into a curriculum of intermediate attack targets and then actively selects evaluation points that best detect small changes in attack strength ("boundary points"). We believe BPJ is the first fully automated attack algorithm that succeeds in developing universal jailbreaks against Constitutional Classifiers, as well as the first automated attack algorithm that succeeds against GPT-5's input classifier without relying on human attack seeds. BPJ is difficult to defend against in individual interactions but incurs many flags during optimisation, suggesting that effective defence requires supplementing single-interaction methods with batch-level monitoring.

• The paper introduces BPJ, a novel automated black-box jailbreak that leverages curriculum learning and boundary point selection to bypass LLM safeguards.
• It employs noise interpolation to generate intermediate attack targets, enabling efficient optimization with only binary classifier feedback.
• The approach demonstrates strong transferability across different queries while significantly outperforming baseline methods in speed and query efficiency.

Boundary Point Jailbreaking: Automated Black-Box Attacks on LLM Safeguards

Introduction and Motivation

The paper "Boundary Point Jailbreaking of Black-Box LLMs" (2602.15001) introduces Boundary Point Jailbreaking (BPJ), a fully automated black-box jailbreak attack method targeting classifier-guarded LLMs with state-of-the-art defensive systems. Unlike prior attacks which rely on access to classifier scores, gradients, or curated attack libraries, BPJ exploits only a single bit of information per API query: whether the classifier flags (blocks) the interaction or not. This substantially restricts information available to attackers, reflecting realistic threat scenarios for deployed LLMs safeguarded by robust filters like Anthropic’s Constitutional Classifiers (CC) and OpenAI’s GPT-5 input monitor. The core contribution is a curriculum-based, evolutionary search technique utilizing "boundary points"—evaluation samples highly sensitive to small changes in attack efficacy—to efficiently traverse the decision boundary of powerful classifiers.

BPJ demonstrates the first fully automated black-box attack that reliably constructs universal adversarial prefixes capable of bypassing both CC and GPT-5’s input classifier, achieving policy-violating outputs for long-form biological-misuse queries. Transferability is observed: prefixes optimized on a single instance generalize to unseen malicious queries, a crucial property for scalable jailbreaks. The attack is shown to outperform previous best-of-$n$ and curriculum-only baselines, requiring orders of magnitude fewer queries for successful evasion.

BPJ Algorithmic Framework

BPJ addresses the primary obstacle in optimizing black-box jailbreaks: lack of fine-grained feedback for incremental improvements. The key challenge is that strong classifiers give no actionable signal when candidate attacks are flagged, especially for difficult targets. To resolve this, BPJ introduces two components:

1. Curriculum Learning via Noise Interpolation:

BPJ does not directly optimize for the ultimate harmful target. Instead, a noise interpolation scheme generates a continuum of intermediate targets by randomly masking characters in the intended harmful string with noise (e.g., non-ASCII tokens), controlling difficulty. High-noise levels yield benign, easier-to-bypass samples; low-noise levels approach the original harmful case. Optimization starts at easier levels and progressively moves toward the true target.

Figure 1: BPJ generates a curriculum of attack targets through noise interpolation, calibrating target difficulty for reliable optimization.

2. Boundary Point Selection:

Within each difficulty level, the algorithm seeks evaluation points ("boundary points" or BPs) that distinguish between close candidate attacks—i.e., inputs where some attacks succeed and others fail. Boundary points are precisely the queries near the classifier’s decision boundary, maximizing the information gained per query. Attack candidates are refined using an evolutionary search (token insertion, substitution, deletion), with selection pressure based on BP performance.

Figure 2: Boundary Point Jailbreaking workflow: boundary points are generated at each curriculum level, attacks evolved via mutations, and boundary points refreshed as attacks succeed.

BPJ alternates between attack improvement (mutations, BP-based selection) and BP discovery, advancing to lower-noise levels only when the evolved prefix achieves a threshold success rate at the current level. The process concludes once a prefix reliably evades the classifier on the plaintext harmful target ($n = 0$).

Empirical Evaluation and Results

BPJ is evaluated across three settings:

• Prompted GPT-4.1-nano classifier (for benchmarking)
• Anthropic’s Constitutional Classifiers guarding Claude Sonnet 4.5
• OpenAI’s GPT-5 input monitor

Strong quantitative results are reported:

• Constitutional Classifiers (CC): BPJ increases average harmful rubric score from 0% (baseline) to 25.5% (non-empty responses), and up to 68% with basic elicitation. Best-of-50 queries yield 39.9% (attack-only) and 80.4% (attack+elicitation). Approximate query budget: 660,000 per attack.
• GPT-5 Input Classifier: BPJ elevates average rubric score from 0% to 75.6%, with Max@50 queries reaching 94.3%. Query budget: 800,000.
• Transferability: Prefixes learned on a single question transfer to diverse unseen queries, including those outside the original topic domain.

  Figure 3: BPJ prefixes succeed against both Constitutional Classifiers and GPT-5’s input classifier, achieving high rubric scores on unseen biological misuse questions.

BPJ shows remarkable convergence speed. Against the GPT-4.1-nano prompt classifier, BPJ converges on average 5x faster than curriculum-only attacks and dramatically outpaces Best-of-$n$, which does not converge within the budget.

Figure 4: BPJ surpasses Best-of-N and Curriculum-only methods in both speed and final success probability for cascade classifier evasion.

Transferability is empirically confirmed: as BPJ optimization proceeds against a single question, the best current prefix increasingly bypasses classifiers for other HarmBench questions.

Figure 5: The transferability of BPJ attacks increases over the optimization process, but can be non-monotonic as transfer is ancillary to the optimization objective.

The optimization process exhibits smooth transitions across noise levels, with most time spent at harder (lower-noise) steps.

Figure 6: Noise level decreases steadily during BPJ optimization, marking successful continuation across curriculum targets.

Figure 7: BPJ optimization runs display consistent ladders of noise reduction across multiple target questions.

BPJ displays robust, consistent convergence: across target questions, speed-up over curriculum-only varies from 3x to 8x, with lower convergence variance.

Figure 8: BPJ’s improvement in speed and consistency is uniform across target questions, outperforming curriculum-only and Best-of-N baselines.

Figure 9: The transferability of BPJ attacks correlates with target question difficulty—the more difficult the target, the greater the transfer to unseen queries.

Theoretical Analysis

The paper formalizes BPJ as an evolution-driven search over a noise-interpolated objective, alternating local mutations and quantile selection driven by boundary points. Using Price-type identities and a continuation framework, the authors prove that mean fitness improvement is governed by population fitness variance and alignment between relaxed and base objectives. Noise interpolation creates actionable optimization signal where direct binary objectives would stall, and boundary point selection sharply reduces query complexity by maximizing the information content per classifier query.

In the infinite population limit, local convergence and continuation conditions are established. The curriculum facilitates warm-starting optimization across difficulty levels, provided sufficient alignment (true attacks receive higher relaxed fitness) and equilibrium tracking.

Practical Security Implications

BPJ’s black-box attack model is especially relevant for real-world deployment: it succeeds without privileged access, relying only on flag/non-flag classifier feedback. BPJ is difficult to defend against at a per-interaction level, but optimization typically incurs numerous flags (hundreds of thousands), enabling batch-level anomaly detection. Effective defense requires layered, analytics-driven monitoring beyond single-turn classifiers.

BPJ attacks incur substantial classifier flags, likely triggering bans under production conditions. The study does not address stochastic classifiers or main-model jailbreaks (uses a human-found bypass for model refusals), though BPJ could be adapted to more complex scenarios.

Anthropic reports probe-based classifiers are more resistant to BPJ than text-based; training classifiers on BPJ-generated attack strings improves detection of adversarial prefixes. Randomization across classifiers may also increase robustness.

Related Work and Conceptual Advances

BPJ adapts concepts from hard-label decision-based attacks, notably the Boundary Attack, but extends them via noise interpolation and prefix-based adversarial generation. Prior decision-based and transfer attacks on LLMs either required gradient/score access or surrogate models, or failed to perform at scale against strong classifiers. BPJ’s principled curriculum and boundary-point search yield superior efficiency and transferability.

BPJ operationalizes evolutionary algorithms with curriculum scheduling and active learning, maximizing optimization signal under extremely sparse binary feedback.

Conclusion

BPJ represents a substantial advancement in automated black-box jailbreaking strategies against classifier-guarded LLM deployments, achieving universal prefix attacks with minimal information and strong transferability. The curriculum-based boundary point method yields orders-of-magnitude improvements in convergence efficiency and scalability. For practical defense, robust monitoring at batch-level is essential; simple per-interaction classifiers are insufficient for reliable safeguards. Theoretical analyses formalize necessary optimization signal and objective alignment properties for success.

The broader implication is that as defensive systems improve, attacker methodologies (such as BPJ) will increasingly exploit fundamental limitations of ML-based safeguards. Future work includes countermeasures to BPJ, formal analysis of transferability, adaptation to stochastic or multi-layered security, and optimization under stricter query constraints.