Refusal Direction Surgery in Transformers

• Refusal direction surgery is a technique that redefines transformer activation vectors to control and modulate a model's refusal behavior.
• It employs methods such as activation addition, directional ablation, and weight surgery to precisely remove or enhance refusal responses without full model retraining.
• Empirical results demonstrate a reduction in harmful refusals from nearly 100% to below 10%, while maintaining core capabilities across languages and modalities.

Refusal direction surgery is a set of inference-time and training-time interventions that modulate or ablate a model’s refusal behavior by acting directly on internal feature representations, typically within the residual stream of decoder-only transformers. Originating from mechanistic findings that refusal is mediated by geometric structures―initially thought to be single vectors, later shown to be multidimensional cones―these methods allow for precise, targeted control of model refusal, enabling both the removal and reinforcement of safety behaviors without retraining or extensive modification of model weights. This paradigm is widely studied across LLMs, compressed models, and even video generators, forming a central axis of contemporary model alignment and jailbreak defense research.

1. Mathematical Foundations of Refusal Directions

The core construct in refusal direction surgery is the refusal direction, a vector or set of vectors in activation space that encodes the distinction between refusal and compliance responses. Let $h \in \mathbb{R}^d$ denote the residual-stream activation at a given layer $\ell$ and token position $t$. A canonical extraction pipeline is as follows:

• Difference-of-means: Compute means over a set of refusals $H_{\mathrm{refuse}}$ and compliances $H_{\mathrm{comply}}$, then take

$$d = \frac{1}{|H_{\mathrm{refuse}}|} \sum_{h \in H_{\mathrm{refuse}}} h - \frac{1}{|H_{\mathrm{comply}}|} \sum_{h \in H_{\mathrm{comply}}} h$$

The unit-norm direction $\hat{d} = d / \|d\|_2$ is used as the refusal direction (Arditi et al., 2024, Chhabra et al., 5 Apr 2025, García-Ferrero et al., 18 Dec 2025).

• Principal Components / SVD: Stack difference vectors across prompt pairs, perform SVD, and use dominant singular vectors to capture multidimensional structure (Abbas et al., 26 Apr 2025, Piras et al., 11 Nov 2025). Self-Organizing Maps (SOMs) yield a set of local refusal directions tiling a concept manifold (Piras et al., 11 Nov 2025).
• Logistic Probes: Train a linear classifier to distinguish refuse and comply activations, using the learned weights as the direction (Joad et al., 2 Feb 2026).

Recent work demonstrates that refusal cannot be reduced to a single line: distinct refusal subtypes occupy separate (though only partially aligned) directions, and the true structure may resemble a low-dimensional cone rather than a strict subspace (Wollschläger et al., 24 Feb 2025, Joad et al., 2 Feb 2026, Piras et al., 11 Nov 2025).

2. Surgical Intervention Methodologies

Refusal direction surgery modifies the internal representations at inference or in model weights:

• Activation addition (induction): For a chosen layer $\ell^\ast$ and all token positions, add $\alpha \, \hat{d}$ to the hidden state:

$$h' = h + \alpha \hat{d} \qquad (\alpha > 0\ \text{induces refusal}, \ \alpha<0\ \text{removes})$$

(Arditi et al., 2024, Zhao et al., 16 Jul 2025, García-Ferrero et al., 18 Dec 2025)

• Directional ablation (removal): Project out the refusal component:

$h' = h - (\hat{d}^\top h) \hat{d}$

This suppresses the model’s ability to refuse (Lermen et al., 2024, Wang et al., 22 May 2025).

• Multi-directional ablation: Given $k$ directions $\{d_i\}$, remove all simultaneously:

$$h' = h - \sum_{i=1}^k \frac{h \cdot d_i}{\|d_i\|^2} d_i$$

Substantially outperforms single-direction methods for nuanced refusal control (Piras et al., 11 Nov 2025).

• Weight surgery: Modify attention/MLP output matrices via rank-1 projection:

$W' = W - \hat{d} \hat{d}^\top W$

This implements directional ablation persistently in model parameters (Arditi et al., 2024, Chhabra et al., 5 Apr 2025).

• Spectral residualization: Decompose the raw refusal vector into “clean” and “polysemantic” components by regression against concept atoms encoding protected skills/styles, subtracting all non-target features (“Ghost Noise” removal) (Cristofano, 13 Jan 2026).

3. Automated Extraction and Selection Pipelines

Refusal directions are extracted and validated via systematic pipelines:

• Dataset construction: Assemble prompts labeled for refusal and compliance across categories (e.g., safety, propaganda, over-refusal) (García-Ferrero et al., 18 Dec 2025, Joad et al., 2 Feb 2026).
• LLM-as-a-judge: Use an auxiliary model to assign categorical refusal scores to completions for robust supervision (García-Ferrero et al., 18 Dec 2025).
• Metric-driven candidate selection: Evaluate candidate directions by ablation/addition and measure the impact on refusal rate, over-refusal, and distributional shift (e.g., KL divergence) (Siu et al., 30 May 2025).
• SVD/SOM manifold sweeping: Instead of a single difference vector, SOMs are trained to find a set of neurons (local mean vectors) whose pairwise difference from the harmless centroid spans multiple refusal submodes (Piras et al., 11 Nov 2025).
• Affine/Nonlinear concept editing: COSMIC introduces affine steering and output-agnostic direction selection, maximizing cosine-similarity alignment between ablated/additions and reference clusters (Siu et al., 30 May 2025).

Empirical selection of optimal layers/positions is essential: refusal signals are typically concentrated in deep or mid-to-late transformer layers, and misplacement yields weak or deleterious effects (García-Ferrero et al., 18 Dec 2025, Zhao et al., 16 Jul 2025).

4. Empirical Findings, Trade-offs, and Limitations

Key experimental results and observations include:

• Effectiveness: Directional ablation reduces refusal rates on harmful prompts from ≈100% to <10% (often <5%) with minimal loss of general model capability (Arditi et al., 2024, Chhabra et al., 5 Apr 2025, Lermen et al., 2024).
• Over-refusal risk: Increasing steering strength can induce high refusal on benign queries. All directions studied produce nearly identical refusal–over-refusal trade-offs: $\alpha$ acts as a general refusal knob, with the specific direction tuning refusal style (Joad et al., 2 Feb 2026).
• Multilingual generalization: Refusal directions transfer seamlessly across languages. Vectors derived in English, Chinese, or Thai enable universal jailbreaks in other languages, highlighting a shared embedding-space axis (Wang et al., 22 May 2025).
• Drift and repair under fine-tuning/compression: Model compression and instruction fine-tuning can cause refusal direction drift, degrading safety. Adding a projection-constrained loss during training, or performing AIRD weight surgery, stabilizes or restores original refusal alignment (Du et al., 8 Sep 2025, Chhabra et al., 5 Apr 2025).
• Ghost Noise and concept entanglement: Naive ablation sometimes suppresses core capabilities (logic, code, math) due to “polysemantic” refusal vectors. Ridge-regularized concept residualization (SRA) mitigates this, yielding near-zero distribution drift and preserved performance (Cristofano, 13 Jan 2026).
• Probabilistic and robustification methods: Probabilistic ablation during training (e.g., DeepRefusal) forces the refusal mechanism to be encoded more robustly, defending against adversarial attacks with negligible capabilities drop (Xie et al., 18 Sep 2025).
• Practical surgery in diffusion/video models: Analogous methods generalize to video generators, where linear or low-rank refusal vectors are subtracted from network weights to “unlearn” specific content classes with minimal collateral effect (Facchiano et al., 9 Jun 2025).

Table: Example Quantitative Impacts of Refusal Direction Surgery (Selected Models/Settings) | Metric | Baseline | After Surgery | Source | |-------------------------------------------|----------|---------------|----------------| | Refusal Rate (harmful, Llama3-8B) | 100% | 2–5% | (Arditi et al., 2024) | | Safety Score (JailbreakBench, Qwen-80B) | 99% | 81–99% | (García-Ferrero et al., 18 Dec 2025) | | KL Drift (Qwen3-VL-4B, standard ablation/SRA) | 2.088/0.044 | | (Cristofano, 13 Jan 2026) | | Compliance Rate (cross-lingual, after ablation) | <10% | 69–96% | (Wang et al., 22 May 2025) |

5. Extensions, Defenses, and Future Directions

The evolution of refusal direction surgery informs both red-teaming and defense development:

• Margin and clustering losses: Training can be enhanced by maximizing separation on the refusal axis in all languages and refusal subtypes (Wang et al., 22 May 2025).
• Subspace and activation monitoring: Active runtime projection monitoring can flag or block low-refusal projections, surfacing attacks (Wang et al., 22 May 2025).
• AlphaSteer/utility-safe mapping: Data-driven, null-space–constrained steering yields high refusal and minimal over-refusal by learning a mapping null on benign activations and aligned only on malicious ones (Sheng et al., 8 Jun 2025).
• Ensemble/multi-cone defenses: Enriching the concept basis to capture distinct refusal modes or cones increases robustness to single-vector ablation (Wollschläger et al., 24 Feb 2025, Piras et al., 11 Nov 2025).
• Interpretability and mechanistic safety: Detailed attribution studies (e.g. via direct feature attribution, attention head hijacking) elucidate how adversarial prompts suppress refusal features and how repairs re-anchor refusal (Arditi et al., 2024, Chhabra et al., 5 Apr 2025).
• Application to other safety “concepts”: The surgery paradigm generalizes beyond refusal to encode or remove various behavioral traits (harmfulness, toxicity, bias) (Zhao et al., 16 Jul 2025, Siu et al., 30 May 2025).

6. Open Challenges and Theoretical Considerations

• High-dimensional and nonlinear geometry: Recent findings demonstrate that refusal is not reducible to a single axis, with concept cones and representational independence considerations complicating the intervention landscape (Wollschläger et al., 24 Feb 2025).
• Polysemantic trade-offs: The entanglement of the refusal direction with protected skills—Ghost Noise—remains a prime concern; identifying atomic concept directions for disentanglement is an emerging direction (Cristofano, 13 Jan 2026).
• Drift and continual learning: Stability under continual parameter changes, fine-tuning, and compression is nontrivial, requiring dynamic constraint or re-anchor strategies (Du et al., 8 Sep 2025, Chhabra et al., 5 Apr 2025).
• Empirical–theoretical gap in adversarial resilience: While probabilistic and multi-directional approaches improve robustness, formal guarantees for complete defense against adaptive attacks are limited (Xie et al., 18 Sep 2025, Piras et al., 11 Nov 2025).

Refusal direction surgery thus constitutes a critical, rapidly evolving toolkit for safety alignment, mechanistic interpretability, and adversarial robustness in foundation models. Its continuing refinement will likely shape future architectures and alignment protocols.