Instruction Forgetting in AI Models

• Instruction forgetting is a phenomenon where models lose their ability to follow previously learned instructions, often quantified by metrics like negative backward transfer (BWT).
• It arises from factors like parameter interference, guidance drift, and prompt misalignment, leading to decoupled instruction adherence despite preserved underlying knowledge.
• Mitigation strategies such as task-specific adapters, gradient projection, and style diversification effectively reduce both catastrophic and pseudo-forgetting in diverse AI architectures.

Instruction forgetting refers to the degradation or loss of a model’s ability to reliably follow, interpret, or respond in accordance with previously learned instructions, occurring as a result of sequential or continual tuning on new instruction-driven tasks. This phenomenon, manifested in LLMs, multimodal LLMs (MLLMs), and reinforcement learning (RL) agents, is a critical subtype of catastrophic forgetting, affecting the alignment between human intent and model output even when underlying knowledge is preserved. It arises across diverse architectures and modalities, with empirical consequences for reliability, robustness, and transferability of modern AI systems.

1. Formal Taxonomy and Mathematical Criteria

Instruction forgetting encompasses both catastrophic forgetting—an outright loss of earlier task performance—and several nuanced forms specific to the instruction or alignment paradigm. The literature distinguishes:

• Catastrophic forgetting: An irreversible drop in accuracy or utility on earlier instruction-following tasks following subsequent fine-tuning. Typically quantified as negative backward transfer (BWT) or task-level accuracy drops; e.g., $$\mathrm{BWT}_T = \frac{1}{T-1}\sum_{i=1}^{T-1}\left(a_{T,i}-a_{i,i}\right)$$ where $a_{j,i}$ is performance on task $i$ after learning task $j$ (Zhang et al., 2023, Luo et al., 2023, Chen et al., 2024, Harmon et al., 20 Oct 2025).
• Pseudo-forgetting: Observed as a performance drop on a previous instruction task without true loss of internal capability, but rather due to failure of the instruction prompt to activate the appropriate reasoning or computation graph. Recovery is possible via prompt modifications, suggesting latent knowledge is retained but not elicited (Sun et al., 2024).
• Dual forgetting in multimodal and CVIT contexts: Simultaneous loss of (a) visual or perceptual competence and (b) instruction-following abilities as MLLMs are sequentially tuned on heterogeneous tasks or instruction templates, often compounded by format drift (Wang et al., 2024, Wu et al., 17 Feb 2025, Zheng et al., 2024).
• Superficial vs. essential forgetting: In MCIT, “superficial” forgetting refers to output format/intent alignment failures (e.g., violating required answer style despite correct content), while “essential” forgetting signifies a true collapse in factual or semantic correctness even when format is preserved (Chen et al., 5 May 2025).

Formally, forgetting is measured at the task or example level. For instance, in large-scale LLM evaluation (Harmon et al., 20 Oct 2025):

Retention Type	Definition	Metric
Retention $(1\to1)$	correct $\to$ correct	Fraction of examples retained
Forgetting $(1\to0)$	correct $\to$ incorrect	$$F = \frac{1}{N}\sum_{i} \mathbf{1}[a^{pre}_i = 1 \wedge a^{post}_i = 0]$$
Backward Transfer $(0\to1)$	incorrect $\to$ correct	$$B = \frac{1}{N}\sum_{i} \mathbf{1}[a^{pre}_i = 0 \wedge a^{post}_i = 1]$$

Chance-adjusted variants subtract spurious changes due to random guessing in MCQ settings (Harmon et al., 20 Oct 2025).

In the multimodal domain, instruction forgetting is often isolated by measuring the drop in “instruction following” (format/intent matching accuracy) against “general knowledge” (semantic content accuracy), with BWT computed independently for each (Chen et al., 2024, Chen et al., 5 May 2025).

2. Underlying Mechanisms and Theoretical Models

Instruction forgetting fundamentally arises from parameter interference, drift in key representations, and shifts in guidance pathways:

• Activation bias and guidance drift: Recent work leveraging causal/mediation analyses of LLM internal representations shows that instruction forgetting is often the result of biased activation: instructions no longer reliably trigger the correct latent computation graph. The original function remains encoded, but the model fails to route input through it (Jiang et al., 16 Feb 2025, Jiang et al., 2024).
• Overwriting of instruction-following knowledge: Sequential fine-tuning with parameter sharing can overwrite those parts of the model encoding the mechanics of following instructions—a distinct subspace from task-specific skill or factual knowledge (Chen et al., 27 Feb 2025).
• Style and format entanglement: In MLLMs, the empirical decomposition of forgetting into superficial and essential phases demonstrates that much of the observed failure is due to loss of adherence to required styles or output formats, not genuine erasure of underlying competence. Only after normalizing for style drift (e.g., through style-diversifying training) can essential knowledge loss be detected (Chen et al., 5 May 2025).
• Low-rank structure and inter-task associations: Example-level forgetting across instruction tasks is empirically low-rank: a small number ($\sim$1–3) of canonical axes explain much of the forgetting matrix, facilitating both inference and mitigation (Jin et al., 2024).
• Task overlap and architectural isolation: Parameter-efficient or architectural approaches such as task-specific adapters or mixture-of-experts models structurally minimize cross-task interference by routing adaptation along isolated submodules (Wu et al., 2024, Chen et al., 2024).
• Pseudo-forgetting as instruction misguidance: If rationale generation becomes decoupled from instruction prompts (low RGD activation), catastrophic drops in raw performance can paradoxically be reversed by minor prompt interventions, revealing that forgetting is not always due to weight drift but sometimes to instructional misalignment (Sun et al., 2024).

3. Empirical Evidence and Diagnostics

Instruction forgetting is robustly documented across model scales, modalities, and continual learning paradigms:

• In LLMs: Instruction forgetting is generally present across model families (1B–7B). Larger models paradoxically exhibit higher normalized forgetting rates due to having higher baseline zero-shot ability, converging to similar post-finetuning performance as smaller counterparts (Luo et al., 2023). Decoder-only architectures (e.g., BLOOMZ) retain more general knowledge than encoder-decoder models (e.g., mT0) (Luo et al., 2023).
• In MLLMs and MCIT: Continual instruction tuning yields severe drops in instruction-following metrics (often BWT $< -30$), while general knowledge declines are more modest ($\sim-12$), confirming decoupling between intent alignment and semantic retention (Chen et al., 2024, Chen et al., 5 May 2025). Adopting mixture-of-expert LoRA or separable adapters significantly reduces such forgetting (Wang et al., 2024).
• Speech-aware LLMs: Introduction of speech-text pretraining pipelines leads to catastrophic drops in instruction adherence (e.g., IFrate drop $\sim-50\%$), even for SLMs inheriting high-quality LLM backbones. Direct speech-text representation alignment is necessary to preserve textual instruction competence (Lu et al., 25 May 2025).
• RL agents: RL experiments confirm that forgetting curves mirror human retention patterns (exponential or power-law decay), but standard spaced-repetition curricula (Leitner/SuperMemo) do not fully mitigate forgetting, owing to asymmetrical inter-task transfer not captured by memory-based scheduling (Speckmann et al., 3 Mar 2025).
• Multimodal continual learning: Visual knowledge degradation is traceable to over-compression of visual feature representations, quantifiable via effective rank or information bottleneck analysis. MDGD regularization and gradient projection approaches retain visual “richness,” preserving both visual and instruction components (Wu et al., 17 Feb 2025).

4. Specialized Mitigation Strategies

A broad array of architectural, algorithmic, and data-centric methods have been developed to counter instruction forgetting:

• Parameter isolation and routing (SwitchCIT, MoELoRA, SMoLoRA): Architectural PEFT designs introduce task-specific adapters or expert mixtures, gated by switch or routing networks, to prevent destructive interference among instructions (Wu et al., 2024, Wang et al., 2024, Chen et al., 2024). Isolation of adaptation allows near-zero forgetting at modest memory overhead (e.g., 0.88% per adapter (Wu et al., 2024)).
• Prompt-guidance and intervention: Pseudo-forgetting can be “revived” by appending partial rationales or task-agnostic chain-of-thought prefixes, short-circuiting guidance-pathway failure (Sun et al., 2024).
• Style diversification (ASD): Introducing answer style diversification (rewriting a significant fraction of training examples in alternate canonical formats) immunizes models against superficial forgetting due to format drift, as each instruction is repeatedly observed in all enforced styles (Chen et al., 5 May 2025). This is particularly effective in MCIT and CVIT.
• Key-parameter regularization (RegLoRA): Selectively regularizing the top-M% key directions in LoRA updates, as identified after each training phase, stabilizes “knowledge-bearing” parameters while enabling efficient adaptation to new instructions (Chen et al., 5 May 2025).
• Instruction vector (IV) and function vector (FV) stabilization: Regularizing the IV- or FV-induced computation graphs or head activations (e.g., via KL-divergence or consistency losses) during fine-tuning preserves the latent pathways responsible for original instruction following, reducing both catastrophic and pseudo-forgetting (Jiang et al., 2024, Jiang et al., 16 Feb 2025).
• Low-rank association and selective replay: Since forgetting exhibits low-rank structure, one can use collaborative-filtering or k-NN matrix completion to predict, for new tasks, which upstream examples are most at risk. Upweighting or replaying those examples arrests forgetting more efficiently than uniform replay (Jin et al., 2024).
• Gradient projection and subspace methods (Fwd-Prompt): Projecting prompt and parameter gradients into pre-trained or residual subspaces (via SVD-based bases) simultaneously minimizes interference with past tasks and encourages reuse of non-conflicting pretrained directions, yielding robust anti-forgetting and positive forward transfer (Zheng et al., 2024).
• Layer-aware task arithmetic (LATA): Disentangling task-specific and instruction-following vectors at the layer level allows surgical unlearning or merging without eroding alignment or utility, as opposed to naïve subtraction which indiscriminately damages both (Chen et al., 27 Feb 2025).

5. Modern Benchmarks and Evaluation Protocols

Several benchmarks and protocols facilitate rigorous evaluation of instruction forgetting:

Benchmark	Focus	Distinctive Features
CITB (Zhang et al., 2023)	Dialog + classification	Long-sequence streams, full template instructions, replay methods
CoIN (Chen et al., 2024)	MLLM, classification + QA	Dual IF+GK metrics, format vs. knowledge loss isolation, diverse datasets
SEFE (Chen et al., 5 May 2025)	MCIT, multimodal metrics	Superficial vs. essential forgetting split, ASD+RegLoRA dual mitigation
Speech-IFEval (Lu et al., 25 May 2025)	SLMs, formatted output	IFrate metric, speech-vs-text disentanglement, adversarial prompt styles
SMoLoRA (Wang et al., 2024)	CVIT, vision+language	Dual forgetting, instruction-fidelity (MIF), generalization to unseen formats/tasks

Across these, metrics such as BWT, MIF (instruction-format fidelity), TA (truth-alignment), KC (knowledge-capability), and IV/FV cosine similarity are consistently employed to provide multifaceted analysis.

6. Open Challenges, Practical Recommendations, and Future Directions

Despite notable advances, instruction forgetting remains an active research area. Key insights include:

• Mitigation requires explicit modeling of instruction semantics and intent alignment, not just task-level data or label retention. Rich, diversified instruction formatting (Chen et al., 5 May 2025), and joint reasoning-guided replay (Sun et al., 2024), outperform vanilla replay or regularization.
• Architectural isolation via adapters or expert modules is highly effective at scale; minimal memory overhead grants scalability to long task sequences (Wu et al., 2024, Wang et al., 2024). However, there remains a trade-off with cross-task transfer: current PEFT methods gain stability by sacrificing genericity.
• **Low-rank structure and prompt-guidance phenomena suggest targeted interventions—both at the replay (selective rehearsal) and training objective (representation regularization) levels—achieve superior anti-forgetting performance than global penalties (Jin et al., 2024, Jiang et al., 16 Feb 2025, Jiang et al., 2024).
• Early broad instruction tuning ("self-instruct" stage) buffers catastrophic forgetting and markedly reduces loss under subsequent sequence tuning (Luo et al., 2023). For both LLMs and MLLMs, incorporating a generic instruction task buffer throughout downstream adaptation is advised.
• Distinguishing superficial from essential forgetting is critical for both diagnosis and method development in multimodal and instruction-rich regimes (Chen et al., 5 May 2025, Wang et al., 2024). Evaluations must track both format compliance and semantic accuracy separately.
• In RL and more general multi-task regimes, curriculum design should account for asymmetric inter-task transfer; naive retention-based or spaced-repetition scheduling is insufficient (Speckmann et al., 3 Mar 2025).

Future research is expected to focus on:

• Dynamic subspace learning and adaptive gradient projection.
• Modular architectures that permit parameter-sharing across similar instructions while isolating divergent ones.
• Automated curriculum and benchmark design emphasizing instruction and intent diversity.
• Retrieval-augmented and rehearsal-efficient approaches that minimize the risk of irretrievable knowledge erosion.

Instruction forgetting thus sits at the intersection of continual learning, alignment, parameter-efficient adaptation, and representation learning, and its principled mitigation is essential for sustaining robust and versatile AI systems (Zhang et al., 2023, Chen et al., 5 May 2025, Harmon et al., 20 Oct 2025).