OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Sets

Abstract: Safe completion requires models to provide useful assistance without enabling harm, but this behavior is difficult to evaluate with isolated prompts. We introduce OpenSafeIntent, a benchmark of controlled prompt-sets that vary intent while holding the underlying task fixed. Each datapoint contains benign, dual-use, and malicious variants of the same task. This design lets us evaluate whether models calibrate assistance across intent shifts, rather than merely appearing safe on average. Across a broad model suite, we find that prompt-level safety hides important failures: models often fail to remain safe across matched intent variants, dual-use behavior is brittle under paraphrase, high-level answers on risky topics are not reliably safe, and responses that reframe ambiguous requests into safer tasks are substantially less likely to cross the safety boundary. Our results suggest that safe completion should be evaluated as intent-calibrated behavior over controlled task variants, not as a single safety-helpfulness tradeoff over independent prompts.

• The paper introduces a matched prompt triplet methodology to evaluate intent-calibrated safe completions across benign, dual-use, and malicious prompts, revealing inconsistency in model responses.
• It employs novel evaluation axes, including Mean Safety, Triplet Safety, and utility metrics, to expose weaknesses in current abstraction-based safety strategies.
• The study highlights significant instabilities in response modes and paraphrase-induced safety flips, urging the need for refined calibration techniques in LLMs.

Intent-Calibrated Safety Evaluation: The OpenSafeIntent Benchmark

Motivation and Benchmark Design

OpenSafeIntent targets a core challenge in LLM safety: ensuring models provide calibrated, intent-aware assistance in the presence of dual-use prompts—requests where the underlying task is fixed but user intent can range from benign to malicious. Traditional safety benchmarks often assess models on individual prompts sampled independently, conflating genuinely intent-aware regulation of output with trivial topic or form-based heuristics. OpenSafeIntent addresses this by constructing matched prompt triplets—benign, dual-use, and malicious variants—anchored to a stable harm domain and task type, enabling precise evaluation of a model's safety calibration across intent shifts.

Figure 1: Structure of an OpenSafeIntent prompt-set, varying only intent across a fixed harm domain, task type, and underlying task, with paraphrases to probe consistency.

The benchmark comprises 805 prompts across 115 prompt-sets, leveraging a multi-stage, model-augmented pipeline grounded in PKU-SafeRLHF seeds and controlled via harm-domain and task-type taxonomies. Each dual-use prompt is further paraphrased, supporting assessment of behavioral consistency near the safety boundary.

Evaluation Metrics and Analysis Framework

OpenSafeIntent expands safety measurement beyond prompt-level averages through several novel axes:

• Mean Safety: Average fraction of safe responses across all prompt types, providing a coarse aggregate.
• Triplet Safety: Fraction of prompt-sets where all three variants (benign, dual-use, malicious) receive safe responses, exposing intent-inconsistent failures that Mean Safety can obscure.
• Mean Utility: Safety-gated helpfulness, reflecting the average utility of completions that remain within safety boundaries.
• Worst-Case Utility: Minimum utility across benign and dual-use variants within each prompt-set.

  Figure 2: Mean and Triplet Safety across models; Triplet Safety provides greater discriminative power, unveiling intent-inconsistent failure cases masked by Mean Safety.

Model Performance Characteristics

Key results demonstrate that average safety metrics insufficiently diagnose intent-calibration failures. For instance, several models with similar Mean Safety scores diverge substantially on Triplet Safety, highlighting inconsistent application of safety protocols as prompt intent varies. The best-performing model (GPT-5.4) achieves 0.56 Mean Utility, while the model average is only 0.42, indicating substantial remaining headroom for utility-preserving safe completions. Dual-use prompts, where the risk of ambiguous intent is maximized, drive most utility loss—benign prompt utility shows minimal drop relative to dual-use when safety boundaries are strictly observed.

Stratified analysis reveals that specific task types (e.g., Explanation, Troubleshooting) and harm domains (e.g., Hazardous Agent Use, Privacy and Data Misuse) are disproportionately challenging for safety-consistent calibration.

Figure 3: Stratified Triplet Safety and Mean Utility across task types and harm domains; indirect tasks and nuanced harm domains yield lower scores.

Failure Modes in Safe Completion

Abstraction and Assistance Response Modes

The investigation of dual-use prompt completions identifies four major modes:

• A1: Refusal
• A2: High-level, non-operational response
• A3: Concrete answer after reframing to a safe task
• A4: Direct, unconstrained answer to the risky task

A central finding is the conditional unsafe rate for A2 (abstract answers) remains high (58.14%), contradicting the common presumption that abstraction offers reliable mitigation. Direct answering (A4) remains unsafe as expected (52.06%), whereas reframing to a safe task (A3) consistently yields the lowest unsafe rate (18.65%), underscoring the insufficiency of superficial abstraction as a primary safety mechanism.

Figure 4: Distribution and conditional unsafe rates for assistance response modes, evidencing the failure of high-level responses as a reliable safety strategy.

Local Stability Under Paraphrase

Experimental manipulation of dual-use prompt paraphrases uncovers significant instability: on average, only 53.24% of paraphrase sets are classified as uniformly safe; 21.39% are uniformly unsafe; and 25.37% induce “safety flips” (mixed safe/unsafe completions within a paraphrase set). Furthermore, among safe responses, there remains notable range in utility, demonstrating that even safety-preserving completions can vary widely in task helpfulness as wording shifts.

Figure 5: Distribution of safety behavior across paraphrased dual-use prompt sets, highlighting prevalence of paraphrase-induced safety flips.

Error Taxonomy: Detection Versus Execution

Failure analysis partitions unsafe dual-use completions into two categories:

• Detection Failures: The model fails to recognize that the prompt requires bounded (not fully operational) assistance.
• Policy-Execution Failures: The model recognizes the need for constraint/refusal but fails to maintain boundaries during completion.

(Figure 4, again)

Figure 4: Safety failure modes on dual-use prompts; both detection and execution failures are prevalent and model-dependent.

These failure modes are not strictly correlated: models with similar safety rates may manifest distinct error profiles, suggesting that safety alignment reflects the intersection of multiple, potentially uncoupled capacities.

Implications and Future Directions

The OpenSafeIntent benchmark fundamentally recasts safe completion as a calibration problem, requiring LLMs to modulate their responses across intent shifts without a change in underlying task. The empirical results invalidate strategies relying solely on generic abstraction and expose the instability of current safety mechanisms to paraphrasal variants. Practically, model training must prioritize stable selection of correct response modes—i.e., flexible assistance, reframing, or redirection/refusal as warranted by nuanced intent in context.

Research directions catalyzed by these findings include:

• Designing safety objectives and alignment procedures sensitive to small intent perturbations and local prompt neighborhoods.
• Decoupling detection from execution mechanisms during policy finetuning.
• Development of stochastic or consensus decoding methods to increase stability under paraphrase.

Conclusion

OpenSafeIntent provides a rigorous, triplet-based framework for evaluating intent-calibrated safe completion in LLMs. The benchmark reveals that prompt-level aggregates are insufficient for detecting critical failures, abstraction alone does not guarantee safety, and response stability remains brittle even under minor input reformulations. Future LLM safety research necessitates intent- and context-aware protocols that achieve robust calibration across a spectrum of ambiguous, dual-use scenarios, advancing both the theoretical understanding and practical deployment of safe and effective LLMs (2607.02047).