Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale

Abstract: There is no doubt that safety alignment is an essential step in LLM training. However, conceptually it does not distinguish between various domains and the level of potential harm of a query, which creates significant complications in the fields like cyber security, where a model should not be constrained by its safety circuits to accomplish the goals of legitimate, authorized operations. In this work, we share our findings from a large scale abliteration experiment on 24 open-source LLMs and show that domain-specific abliteration is achievable with standard methodology on the example of a 1T-parameter Kimi K2. Building on recent work showing that refusal in LLMs occupies a multi-dimensional subspace within layers, we find that it is also distributed widely across layers, especially in trillion-parameter MoE architectures, and so we aim to capture the part of it that represents harmful concepts in the cybersecurity domain exclusively. We also investigate the correlation between models' features and the effect of domain-specific abliteration, identifying that the type of safety training and architecture are the most reliable predictors. Finally, we classify the models into 3 abliteration susceptibility tiers and put forward a set of conjectures as to why a particular effect from this intervention might be observed in a given model.

• The paper demonstrates that domain-selective abliteration can reduce cybercrime refusals by up to 93 percentage points while maintaining overall model performance.
• It utilizes per-layer activation extraction and orthogonal projection to isolate and target refusal behaviors across 24 diverse language models.
• Architectural differences and alignment methods, especially DPO-based training, are key predictors of model susceptibility to targeted safety interventions.

Domain-Selective Safety Alignment via Abliteration: An Analytical Perspective

Introduction

The paper "Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale" (2607.02714) investigates the geometric, architectural, and training determinants of domain-specific safety alignment in LLMs, focusing on the cybersecurity domain. Building on the emerging understanding that refusal responses inhabit a multi-dimensional subspace within model activations, the authors systematically evaluate whether it is possible to abliterate (i.e., erase via orthogonal projection) refusal specifically for one harm domain (e.g., cybercrime) without degrading model safety elsewhere. Their large-scale experiment, evaluating 24 open-source LLMs spanning 0.6B to 1T parameters and diverse architectures, elucidates how alignments across harm domains, safety training methods, and model architectures interact to shape susceptibility to such targeted interventions.

Methodological Framework

Central to the experimental protocol is the construction and application of domain-specific refusal directions. The approach involves:

• Extraction of harmful/harmless prompt pairs within cybersecurity, carefully contrasting prompts targeting offensive actions versus benign or educational queries within the same technical space, to isolate the dimension of intent-specific refusal.
• Collection of per-layer, last-token activations, leveraging mean difference in hidden states to define a refusal vector for each layer.
• Application of orthogonal projection at strengths up to 30% of layers, systematically varying the selection strategy between uniform-spread and norm-maximization, and assessing outcomes on both custom red-teaming datasets and standardized scientific benchmarks.
• Quantitative evaluation via automated refusal detection and MMLU for capability preservation.

This pipeline affords a robust cross-model comparison of the magnitude, specificity, and tradeoffs of refusal erasure interventions.

Core Findings: Domain Selectivity and Model Response

Differential Domain Susceptibility. Abliteration effects are sharply domain-heterogeneous. With a cybersecurity-focused direction, the authors succeeded in reducing refusal for cybercrime tasks by up to 93pp in Kimi K2 (1T parameters) while substantially preserving refusal in categories such as explicit content, violence, and privacy (retaining 44–100% refusal in non-target domains). Misinformation, by contrast, often showed greater cross-model reduction, indicating high inter-domain geometric proximity for refusal representations across some harm types.

Figure 1: Per-domain refusal rates at 30% abliteration (left) and change from baseline (right) across all 24 models. Cybersecurity (cybercrime) consistently shows the largest reduction in high-susceptibility models.

Figure 2: Mean refusal reduction per domain at 30% abliteration. High-susceptibility models (n=12) show disproportionate domain targeting.

Capability Retention. Across all models, general capabilities (MMLU) are nearly unchanged, with the maximal decline in any model being 0.028 and median variation close to zero. This underscores that domain-specific abliteration leaves the backbone cognitive utility of the system intact.

Figure 3: MMLU vs. refusal rate trajectories (baseline to 30%). Refusal declines without appreciable loss in MMLU performance.

Non-Monotonic Susceptibility. Susceptibility to domain-specific safety removal does not monotonically increase with model size. Some smaller models (e.g., Qwen3-8B) are highly susceptible, whereas much larger models (e.g., Mistral-Large at 123B, GLM-4.7 at 355B) can be essentially immune under standard interventions.

Figure 4: Maximum refusal reduction across 24 models, grouped by susceptibility class. Models span 0.6B to 1T parameters.

Layer Selection Efficacy. Uniformly spreading abliteration over the majority of model depth (25–95% layers) yields superior refusal reduction compared to norm-based targeting, especially in large-scale and mixture-of-experts (MoE) architectures.

Figure 5: Norm-based vs. uniform spread layer selection across 9 models. Uniform spread consistently achieves equal or greater refusal reduction than norm-based selection.

Architecture and Training as Predictors of Susceptibility

The analysis identifies strong correlates of abliteration susceptibility:

• Safety Training Protocol: DPO-based alignment confers the highest susceptibility to domain erasure (median ~30pp), whereas GRPO and models with undisclosed RL training are far less amenable. PPO+DPO and RLHF exhibit intermediate susceptibility.
• Architecture: MoE architectures exhibit higher overall resistance relative to dense transformers, plausibly due to the richer, more distributed representational geometry within their expert submodules.
• Model Family Effects: Some families (e.g., Phi) are consistently highly susceptible, while others (e.g., Mistral, certain Qwen variants) exhibit strong within-family heterogeneity.

  Figure 6: Susceptibility predictors: MMLU baseline score vs. maximum cybercrime refusal drop. Bubble size indicates parameter count; diamonds denote MoE architectures; color indicates safety training method.

  

  Figure 7: Maximum refusal reduction by safety training method. DPO-based methods are most susceptible; GRPO and undisclosed RL methods confer near-resistance.

  

  Figure 8: Susceptibility by architecture type (left) and model family (right). Dense and MoE show comparable overall distributions, but family-level patterns emerge.

Refusal Geometry: Domain Distinctness in Activation Space

Per-domain direction analysis reveals that refusal subspaces for different harm domains are not geometrically aligned. Specifically, cybersecurity refusal vectors are relatively distinct from those for other domains, suggesting that sufficiently fine-grained dataset construction and direction extraction can selectively target specific refusal behaviors.

Figure 9: Per-domain refusal direction similarity on the scientific benchmark. Cybercrime and copyright show the most distinct refusal directions.

Figure 10: Per-domain refusal direction similarity on the cross-evaluation dataset. Cybersecurity has the most distinct refusal direction.

This geometric result validates the multi-dimensionality hypothesis for refusal representation and enables operational domain selectivity.

Implications and Future Directions

The demonstrated feasibility of domain-focused abliteration, particularly at trillion-parameter scales and in complex models like Kimi K2, challenges the idea that safety alignment necessarily implies global refusal circuits. The strong, persistent selectivity observed indicates that harm-prevention circuits are distributed, multi-faceted, and at least partially separable by domain via linear methods. This has several concrete implications:

• For AI Safety: The multidimensional, non-global nature of learned refusal could be leveraged for targeted safety augmentation but also presents the risk of non-uniform defense against jailbreaks or adversarial interventions. Optimization of safety training to increase geometric overlap across harm categories may be a promising mitigation.
• For Security Applications: Practitioners can adapt refusal mechanisms to better suit domain-specific operational needs, e.g., enabling red-team LLMs to be useful for authorized offensive operations without broadly relaxing guardrails.
• Limitations: Uniform projection was effective only in some architectures and safety training regimes; more sophisticated interventions (e.g., those in RepIt or Heretic) may be required for robust, flexible domain targeting. The specificity of refusal removal is sensitive to the construction of harmful/harmless tasks, limiting generalization beyond the prompt distribution represented.

Conclusion

This study provides systematic empirical evidence that the domain specificity of LLM safety alignment is both achievable and contingent. The effectiveness of abliteration as a tool for targeted refusal removal is tightly linked to the geometry of refusal representations, architecture, and alignment protocol. These findings reinforce the view that safety alignment in LLMs is not a uniform, monolithic construct but a complex, distributed property. Theoretical and practical advances are needed to develop architectures and training schemes that enforce more robust, domain-agnostic refusal while preserving operational domain selectivity where needed. This work substantially informs forthcoming research directions on both alignment robustness and domain-specific language agent deployment strategies.