Multimodal Knowledge Edit-Scoped Generalization for Online Recursive MLLM Editing

Abstract: Online multimodal knowledge editing requires injecting a continual stream of visual-textual corrections into multimodal LLMs (MLLMs) with bounded overhead and minimal disruption to unrelated behaviors. Existing editors mainly emphasize edit reliability and long-horizon stability, but rarely control the semantic boundary of each edit. Our pilot analyses of post-edit behaviors and internal neuronal activities reveal a scope gap behind reliable edits: instance-level success neither guarantees transfer to valid cross-modal variants nor prevents leakage to unrelated inputs, while edit-related cross-modal responses concentrate in deeper semantic layers. Therefore, we formulate Edit-Scoped Generalization, reframing online MLLM editing from merely correcting an instance to controlling the propagation boundary of each edit. To this end, we propose ScopeEdit, a scope-aware online editor that decomposes each update into a modality-local absorption branch and an evidence-gated shared generalization branch. The local branch supports stable edit absorption, whereas the shared branch enables cross-modal propagation only when visual and textual evidence are sufficiently aligned. Both branches perform scope-separated write geometries in orthogonal low-rank spaces and maintain branch-wise preconditioners via Sherman--Morrison recursions, yielding constant per-edit overhead. Extensive experiments across diverse benchmarks, long-horizon edit streams, MLLM backbones, real-world VLKEB scenarios, and complex vision-language architectures show that ScopeEdit consistently improves the trade-off between in-scope cross-modal transfer and out-of-scope locality, while preserving edit reliability, stability and online efficiency. Our code is available at https://github.com/lab-klc/ScopeEdit.

• The paper introduces edit-scoped generalization to control in-scope cross-modal propagation while preserving out-of-scope locality.
• The framework, ScopeEdit, employs dual branches—modality-local absorption and evidence-gated shared generalization—for efficient, history-aware weight updates.
• Empirical results demonstrate over 92% reliability, minimal representation drift, and constant per-edit computational cost across diverse MLLM architectures.

Scope-Aware Online Editing for Multimodal LLMs: Edit-Scoped Generalization and the ScopeEdit Framework

Introduction and Motivation

Multimodal LLMs (MLLMs) have emerged as unified agents for knowledge-intensive reasoning over visual-textual inputs, yet their knowledge remains static, necessitating continual updates as facts change or new knowledge arises. Online multimodal knowledge editing aims to inject a stream of factual corrections into MLLMs efficiently without full retraining or side effects on unrelated knowledge. Existing editors are designed for edit reliability and long-horizon stability, but lack semantic granularity: they cannot control the propagation scope of edits, resulting in under-generalization (missed in-scope variants) and over-generalization (leakage to unrelated regions).

Figure 1: Motivation and conceptualization of edit-scoped generalization for online MLLM editing, depicting scope errors when generalization is not explicitly controlled.

Pilot analyses demonstrate that even when an edit is reliable on its intended instance, it can fail on cross-modal variants (28.60%) or introduce behavioral drift outside scope (7.20%). Further, analysis reveals that cross-modal generalization is not a network-wide phenomenon, but instead emerges in deeper semantic layers—suggesting targeted architectural opportunities for edit-scoped control.

Problem Formulation: Edit-Scoped Generalization

This work formalizes the task of edit-scoped generalization in online MLLM editing, reframing the goal from point-wise correction to explicit propagation control. Edit-scoped generalization consists of two requirements:

• In-scope cross-modal generalization: Edits must transfer to all valid, semantically equivalent variants, regardless of whether they are visual or textual.
• Out-of-scope locality preservation: Edits must remain suppressed on semantically irrelevant inputs to prevent undesired behavioral changes.

Pilot experiments confirm that instance-level success is neither a necessary nor sufficient condition for semantic generality and locality. The distribution of edit-related activity across layers also supports a late-layer focus for editing (see Figure 2).

Figure 2: Empirical analysis showing frequent scope errors even for reliable edits. Edit responses concentrate in late layers, motivating architectural design for scope-aware editing control.

ScopeEdit: A Scope-Aware Online Editor

To address the edit-scoped generalization challenge, the paper introduces ScopeEdit, an online editing framework that decomposes weight updates into two functional branches:

• Modality-local absorption branch: Handles edits that should remain local. Updates are absorbed in a subspace aligned with modality-specific information, supporting reliable and stable editing while preserving locality.
• Evidence-gated shared generalization branch: Activated only when both modalities show aligned, strong semantic evidence. Enables in-scope cross-modal propagation.

Each branch operates in a distinct, orthogonal low-rank subspace to prevent entanglement of local and shared effects. Both maintain independent recursive preconditioners (via Sherman–Morrison recursion) for efficient, history-aware online updates. This formulation yields constant per-edit computational and memory cost, independent of edit stream length.

Key architectural contributions include:

• Scope-sensitive low-rank write geometry: Orthogonal decomposition of write spaces for absorption and generalization branches.
• Cross-modal evidence gating: Shared updates are modulated by quantifiable semantic alignment (directional agreement and bilateral support) between visual and textual keys.
• Branch-wise recursive preconditioners: History-aware, low-overhead online adaptation using efficient second-order update statistics.

Empirical Results and Analysis

Comprehensive experiments target five core questions, encompassing edit-scoped efficacy, component ablations, long-horizon stability, computational efficiency, and architectural generality:

Edit-Scoped Generalization Trade-off

ScopeEdit demonstrates state-of-the-art performance on both reliability and the stricter in-scope/out-of-scope trade-off on E-VQA and E-IC benchmarks with BLIP2-OPT and LLaVA-v1.5. After 100 sequential edits, ScopeEdit maintains reliability >92% and multimodal generalization rates substantially above prior art, while locality metrics remain robust.

Ablations and Architectural Sensitivity

Removal of the scope-aware decomposition, orthogonal split, or cross-modal gating consistently degrades edit-scoped performance, particularly over long horizons. Orthogonal write-basis initialization outperforms other strategies in head-to-head comparisons.

Figure 3: Sensitivity analysis showing that adequate rank allocation for low-rank subspaces is required to balance plasticity and stability in online editing.

Stability and Parametric Entanglement

Under long-horizon settings, standard editors such as MEND exhibit severe representation drift and weight-update entanglement, leading to catastrophic forgetting. ScopeEdit, by contrast, demonstrates stable representations and disentangled edit streams, as visualized via hidden-state and weight-overlap analyses.

Figure 4: Hidden-state drift for various editors over 100 sequential edits; ScopeEdit exhibits minimal representation drift, preserving pre-edit distribution alignment.

Figure 5: Pairwise weight-entanglement metrics showing that ScopeEdit maintains low coordinate overlap and cosine similarity, indicating stable, disentangled parameter updates.

Efficiency

ScopeEdit achieves near-constant per-edit time (<1s per edit) and flat, non-growing memory consumption, independent of sequence length, outperforming dense or memory-based methods by large margins.

Figure 6: Edit latency and memory usage over time for various editors; ScopeEdit achieves constant cost per edit.

Cross-Architecture and Scenario Extension

Pilot layer-wise analyses generalize across major MLLM families including BLIP2-OPT, Qwen2-VL, and Qwen3-VL (Figure 7), confirming that late-layer concentration is an architectural invariant.

Figure 7: Consistent late-layer amplification of modality-fused token energy across diverse MLLM architectures, supporting generality of the edit-scoped approach.

Further, ScopeEdit maintains strong edit-scoped and portability outcomes in real-world scenarios (e.g., VLKEB benchmark) and under complex module-editing regimes.

A targeted case study (Figure 8) illustrates that ScopeEdit correctly propagates an edit to all in-scope variants while protecting out-of-scope regions—a property not satisfied by leading baselines.

Figure 8: Case study showing that ScopeEdit generalizes edits correctly within scope and preserves locality, outperforming a strong baseline (M-ORE).

Implications and Directions for Future Research

The explicit formalization of edit-scoped generalization, along with ScopeEdit's modular architecture, advances online knowledge editing toward practical, safe, and efficient continual adaptation. The framework's compositional and orthogonal design provides a blueprint for editing mechanisms that are robust to edit-collapse, inter-edit interference, and context leakage. These findings align with open problems in lifelong adaptation, trustworthiness, and minimization of unintended model side effects (2607.01978).

Potential future directions include automatic discovery of edit scopes, uncertainty-aware propagation controls, more expressive or adaptive cross-modal gating, and integration with open-world, dynamic knowledge environments. Theoretically, the modular parametric separation recalls classic stability-plasticity decompositions, now instantiated in modern MLLMs for controlled online knowledge management.

Conclusion

This work establishes edit-scoped generalization as the central tenet for practical online knowledge editing in MLLMs, and presents ScopeEdit as an efficient, principled solution. The explicit scope decomposition via orthogonal low-rank branches, gated cross-modal generalization, and per-branch recursive preconditioning collectively enable sustainable long-horizon editing without compromising model reliability, generality, or locality. Systematic analyses across architectures and benchmarks confirm both the necessity and generality of the edit-scoped paradigm, laying the foundation for robust lifelong knowledge editing in intelligent multimodal systems.