Data-Free Selective Unlearning (DFSU)

Updated 29 January 2026

Data-Free Selective Unlearning (DFSU) is a set of methods that erase specific learned information from models without accessing the original data.
It leverages techniques like nullspace projection, surrogate data synthesis, and sparse masking to selectively remove unwanted knowledge while preserving core performance.
DFSU is crucial for privacy compliance, decontamination, and bias removal, providing provable guarantees with minimal utility loss.

Data-Free Selective Unlearning (DFSU) refers to a class of machine unlearning techniques that remove or erase targeted knowledge—typically about specific datapoints, classes, or personal identifiers—from pretrained models without accessing any raw data from the forget set. DFSU methods operate directly on model parameters, surrogate embeddings, or self-synthesized data and are designed to be both selective (minimally affecting performance on retained knowledge) and efficient (avoiding retraining or iterative fine-tuning). In recent literature, DFSU has emerged as a paradigm for privacy, regulatory compliance, and model decontamination across computer vision, natural language, and multimodal domains (Mishra et al., 16 Dec 2025, Mishra et al., 16 Dec 2025, Shah et al., 2023, Zhou et al., 22 Jan 2026, Xie et al., 18 Sep 2025, Zhang et al., 2024).

1. Formal Problem Statement and Motivation

DFSU aims to remove the influence of a specified "forget set" $F$ (e.g., samples, classes, domains, or tokens) from a trained model $f:\mathcal{X}\to\Delta^C$ , while preserving performance on the complementary "retain set" $R$ (Shah et al., 2023, Mishra et al., 16 Dec 2025, Zhang et al., 2024, Zhou et al., 22 Jan 2026). The forget set may comprise (a) particular class identities, (b) domain-labeled subsets, or (c) sensitive sequences such as PII in LLMs. Crucially, the setting is "data-free"—the algorithm has no access to any original forget examples, and often no real data at all.

Typical desiderata:

Selectivity: $f'$ should match $f$ on $R$ , but for $F$ behave as though the target information was never learned (random or baseline output).
No data access: Only model weights, fixed embeddings, or synthesizable proxies are available.
No retraining: Methods avoid gradient-based fine-tuning over (real or synthetic) forget data.
Provable or controlled unlearning: Guarantees or effective metrics for the erasure of $F$ and preservation of $R$ .

Motivations for DFSU include privacy (e.g., GDPR right to be forgotten), post-hoc model decontamination, bias removal, and compliance in real-world deployments where log or training data is inaccessible (Mishra et al., 16 Dec 2025, Zhou et al., 22 Jan 2026).

2. Methodological Foundations: Nullspace Projections and Surrogate Datasets

At the technical level, DFSU methods fall into several families, governed by how they identify and eliminate information about $F$ :

a. Nullspace Projection in Embedding Spaces:

Closed-form projection onto the orthogonal complement of the subspace spanned by "forget set" representations is a central technique for models with explicit embedding banks (e.g., CLIP) (Mishra et al., 16 Dec 2025, Mishra et al., 16 Dec 2025).

For CLIP, forget-class text embeddings form $T_\text{target}$ ; its orthonormal basis $U$ (from SVD or QR) induces the projector $P = I - \alpha UU^{T}$ , with $\alpha\in[0,1]$ controlling the strength.
Extending this, (Mishra et al., 16 Dec 2025) integrates both class text and synthesized prototype embeddings for domain-specific or global unlearning.
Closing the loop, the Consistent Class Unlearning Projection (CCUP) framework minimizes a three-term Frobenius norm objective over the projection matrix $W$ to balance forgetting and retention (Mishra et al., 16 Dec 2025).

b. Surrogate Data Generation:

Data-free model inversion and self-generation produce pseudo-samples to stand in for missing forget data (Zhou et al., 22 Jan 2026, Xie et al., 18 Sep 2025, Zhang et al., 2024).

In LLMs, model inversion is used to synthesize surrogate PII which is then annotated and targeted by parameter-efficient unlearning (e.g., LoRA) combined with token-level masking (Zhou et al., 22 Jan 2026).
Reveal-and-Release (Xie et al., 18 Sep 2025) uses optimized soft-prompts to elicit the model’s own knowledge about forget information, scoring outputs for relevance and diversity before iterative LoRA-based parameter editing.

c. Sparse Representations and Bottleneck Masking:

Discrete Key-Value Bottleneck (DKVB) architectures allow DFSU by masking codebook entries frequently activated by forget-set representations, thereby preventing activation for $F$ without retraining (Shah et al., 2023).

d. Data-Free Distillation and Filtering:

Generative networks synthesize data that is then filtered and used for student-teacher distillation. The Inhibited Synthetic PostFilter (ISPF) algorithm suppresses forbidden-class signal in synthetic data, then reallocates this mass during distillation to retaining classes to improve utility (Zhang et al., 2024).

3. Algorithmic Descriptions and Computational Complexity

Representative algorithms are characterized by minimal computational overhead:

Approach	Core Step	Computational Cost
CLIP Nullspace Projection (Mishra et al., 16 Dec 2025, Mishra et al., 16 Dec 2025)	SVD/QR + d×d matrix inversion	O(d³) setup, O(d²) per-image inference
Sparse Key-Value Masking (Shah et al., 2023)	Forward pass + key masking	O(#F) forward passes, no backprop
Model Inversion + LoRA (Zhou et al., 22 Jan 2026)	Pseudo-data gen + low-rank update	<10 epochs LoRA, synthetic data only
ISPF Data-Free Distillation (Zhang et al., 2024)	Inhibited generator + post-process filter	<10⁴ synth samples, no real data
Iterative PEM Subtraction (Xie et al., 18 Sep 2025)	NeuralUCB prompt search + LoRA edits	Soft prompt search + modular finetuning

Closed-form SVD/QR methods incur only a one-time matrix decomposition and update (e.g. $d\leq1024$ ), while masking and LoRA/PEM unlearning involve only minor forward passes or lightweight parameter updates. In all cases, per-inference overhead is negligible compared to full model retraining.

4. Quantitative Results and Evaluation Criteria

DFSU evaluation across modalities relies on standard classification and privacy risk metrics:

Accuracy drop on forget/retain sets: Substantial reduction (e.g., $>99\%$ drop) in forget class zero-shot accuracy, with modest loss for retain classes (e.g., 81.45%→75.4% (Mishra et al., 16 Dec 2025)).
Membership Inference Attack (MIA) Score: $\, \mathrm{MIA} = (BF_{\text{forget}}{-}AF_{\text{forget}}) - (BF_{\text{retain}}{-}AF_{\text{retain}})\,$ ; higher indicates more selective forgetting (Mishra et al., 16 Dec 2025, Mishra et al., 16 Dec 2025, Zhang et al., 2024).
Privacy leakage for LLMs: Exact Reconstruction Rate (ERR), Fractional Reconstruction Similarity (FRS), Sample-Level Exposure (S-Exp), Entity-Hit Rate (E-Hit) with results such as ERR $\to0\%$ , E-Hit $\approx1\%$ , PPL penalty $<0.2$ (Zhou et al., 22 Jan 2026).
Utility retention: Measured on retain-classes, held-out datasets, or standard tasks; marginal decline (often $<1\%$ accuracy or $<0.5$ PPL) (Shah et al., 2023, Zhang et al., 2024).
Anamnesis Index (AIN): For "ease of re-learning": ISPF achieves AIN $\approx1$ , indicating unlearned information is not trivially relearned (Zhang et al., 2024).

Below is an excerpt from (Mishra et al., 16 Dec 2025) for CLIP domain-agnostic unlearning:

Domain	Retain BF $\to$ AF	Forget BF $\to$ AF
clipart	92.0%→80.8%	86.6%→3.4%
real	97.3%→91.8%	93.6%→6.1%
sketch	86.3%→81.7%	81.0%→4.2%

Results generally demonstrate near-perfect selective forgetting with minimal collateral utility damage.

5. Privacy, Decontamination, and Theoretical Guarantees

DFSU methods, by construction, avoid direct exposure to the forget set and thus satisfy critical privacy mandates (Mishra et al., 16 Dec 2025, Zhou et al., 22 Jan 2026). By projecting out the subspace corresponding to forget classes or tokens, or by masking discrete representations, these methods induce provable or instrumentally measured unlearning:

For nullspace projections, alignment with erased classes is strictly zero for hard projection, and quantifiably reduced by factor $1-\alpha$ for partial projection (Mishra et al., 16 Dec 2025).
Surrogate synthesis approaches, when coupled with careful parameter editing, can match or exceed oracle (data-dependent) unlearning in terms of leakage risk, as evidenced by exposure metrics and membership inference resistance (Zhou et al., 22 Jan 2026, Zhang et al., 2024).
Masking-based DFSU achieves $>99.5\%$ retention accuracy and $-100\%$ on the forget set, with near-zero additional compute, indicating theoretically sound selectivity under plausible independence assumptions about key activations (Shah et al., 2023).

These guarantees are central for privacy-sensitive use-cases such as GDPR "right to be forgotten," proprietary data removal, and model decontamination following dataset recall or regulatory audit.

6. Methodological Variants and Open Challenges

DFSUs have been successfully instantiated in several paradigms:

Global vs. domain-specific unlearning: Nullspace methods can target all domains or tailored subsets, with empirical success in CLIP across diverse visual domains (Mishra et al., 16 Dec 2025).
Parameter-efficient Modular Editing: LoRA and PEM-based methods enable iterative, scalable unlearning via self-generated or inverted forget data (Xie et al., 18 Sep 2025, Zhou et al., 22 Jan 2026).
Data-free distillation: ISPF and related methods exploit the synthetic data regime, optimizing for both effective forgetting and maximal retention utility, overcoming challenges of over-filtering (Zhang et al., 2024).

Limitations include: dependence on accurate class/domain/tokens to be forgotten, possible imperfect synthesis of domain-specific prototypes in highly abstract or unseen domains, scaling challenges with extremely large numbers of classes, and the white-box requirement for some approaches (Mishra et al., 16 Dec 2025, Zhou et al., 22 Jan 2026). A plausible implication is that future DFSU research will target black-box APIs, incremental (continual) unlearning, and further robustness to ambiguous or emergent classes.

References:

(Mishra et al., 16 Dec 2025, Mishra et al., 16 Dec 2025, Shah et al., 2023, Zhou et al., 22 Jan 2026, Xie et al., 18 Sep 2025, Zhang et al., 2024)