Multi-Level Change Interpretation

• Multi-Level Change Interpretation is a framework that integrates fine-grained pixel analysis with semantic captioning to interpret temporal changes in data.
• It employs Siamese multi-scale feature extraction and BI³ layers to achieve precise spatial localization and context-rich natural language descriptions.
• The methodology leverages axiomatic counterfactual and Shapley value approaches to attribute model output changes robustly and transparently.

Multi-Level Change Interpretation (MCI) refers to a class of unified models and analytical frameworks that simultaneously interpret changes at multiple levels of granularity—such as pixel-wise detection and semantic-level captioning—across paired data snapshots. In remote sensing, MCI architectures integrate vision and language modalities to jointly detect and describe changes observed in bi-temporal image data, providing both precise spatial localization and high-level human-readable explanations. In explainable machine learning, MCI encompasses counterfactual attribution methods that distribute observed output changes among evolving inputs and mechanisms, ensuring rigor via axiomatic justification. MCI methodologies have achieved state-of-the-art results in remote sensing image change interpretation (RSICI) tasks, and offer robust, axiomatically grounded explanations for unit-level changes in complex modeled systems (Brock et al., 8 Jan 2026, Liu et al., 2024, Budhathoki et al., 2022).

1. Conceptual Foundations and Motivation

The core motivation for Multi-Level Change Interpretation is to address the limitations of traditional single-task change analysis frameworks. Pixel-level change detection (e.g., binary mask segmentation) delivers accurate spatial delineation of alterations over time, but lacks context or reasoning regarding the nature and drivers of these changes. Semantic change captioning, by contrast, generates natural-language descriptions that contextualize changes but often lacks visual precision or grounding. MCI “bridges” these capabilities by sharing a multi-scale backbone for both tasks, allowing the extraction of fine boundary information and high-level semantic understanding in a unified fashion (Brock et al., 8 Jan 2026, Liu et al., 2024).

In machine learning model interpretation, MCI methods provide an axiomatic framework to attribute observed changes in a statistical unit’s output to contributions from input modifications and mechanism/drift, operating at user-specified levels of granularity (Budhathoki et al., 2022).

2. Architectures and Computational Frameworks

MCI architectures for remote sensing image change interpretation typically employ a Siamese design, where two parallel backbones extract multiscale features from temporally aligned image pairs:

• Siamese Multi-Scale Feature Extractor: Based on SegFormer-B1 or related vision transformers; four levels of feature maps are extracted from each input image.
• Bi-Temporal Iterative Interaction (BI³) Layers: At each scale, BI³ modules facilitate inter-image feature exchange via Local Perception Enhancement (LPE) and Global Difference Fusion Attention (GDFA), promoting discriminative encoding of temporal differences (Liu et al., 2024).
• Pixel-Level Change Detection Branch: Aggregates multi-scale fused features (using convolutional bi-temporal fusion and upsampling) and outputs a binary change mask.
• Semantic Change Captioning Branch: Operates primarily on the top-level features, applies additional BI³ processing, projects features into token embeddings, and uses a transformer decoder to produce descriptive captions.
• LLM Orchestration (Vision-Language Agent): Wraps the MCI backbone within a LLM agent that interprets user queries, invokes pixel-level and captioning submodules, optionally integrates domain-specific post-processing (e.g., area quantification), and delivers output in natural language (Brock et al., 8 Jan 2026).

In explainable ML, the computational framework relies on counterfactual reasoning and Shapley value theory to attribute output changes to input and mechanism variations at both group and feature levels (Budhathoki et al., 2022).

3. Multi-Task Learning Objectives and Loss Formulation

MCI models are optimized via multi-task objectives that couple detection and captioning losses. Formally, let $N$ be the number of pixels, $T$ the length of the generated caption.

• Pixel-level change detection loss: Binary cross-entropy over all pixels,

$$\mathcal{L}_{\mathrm{det}} = -\frac{1}{N}\sum_{i=1}^N [ y_i \log \hat{p}_i + (1 - y_i)\log(1 - \hat{p}_i) ]$$

• Captioning loss: Token-level cross-entropy,

$$\mathcal{L}_{\mathrm{cap}} = - \sum_{t=1}^T \log P(w_t\,|\,w_1,...,w_{t-1}, V)$$

• Overall objective: Weighted combination

$$\mathcal{L} = \lambda_{\mathrm{det}}\mathcal{L}_{\mathrm{det}} + \lambda_{\mathrm{cap}}\mathcal{L}_{\mathrm{cap}}$$

with $\lambda_{\mathrm{det}}, \lambda_{\mathrm{cap}}$ chosen to balance scales, often set to 1 post-normalization (Brock et al., 8 Jan 2026, Liu et al., 2024).

In attribution-based MCI, the target is to decompose $$\Delta y = y^{(2)} - y^{(1)} = f^{(2)}(x^{(2)}) - f^{(1)}(x^{(1)})$$ into additively consistent contributions from input and/or mechanism changes using axiomatic counterfactual Shapley methods (Budhathoki et al., 2022).

4. Representative Datasets and Evaluation Metrics

The development and assessment of MCI approaches rely on diverse, large-scale datasets and rigorous quantitative metrics. Representative datasets include:

Dataset	Domain	Size	Labels	Caption Type
Forest-Change	Forest/RSICI	334	Binary change mask	Human + rule-based
LEVIR-MCI	Urban/RSICI	10,077	Pixel-level mask	5 human per image
LEVIR-MCI-Trees	Subset (tree)	2,305	Pixel-level mask	“Tree”-keyword filtered

Detection performance is reported via mean Intersection over Union (mIoU), and per-class IoU for “change” and “no-change.” Captioning quality is assessed by BLEU-1/4, METEOR, ROUGE-L, and CIDEr-D (Brock et al., 8 Jan 2026, Liu et al., 2024).

In attribution-based MCI, simulations use mean absolute error (MAE) between true and estimated attributions, with real-world case studies (e.g., wage growth drivers) for empirical verification (Budhathoki et al., 2022).

5. Experimental Results and Ablative Findings

On LEVIR-MCI and Forest-Change, MCI models demonstrate state-of-the-art performance for both joint detection and captioning:

Dataset	mIoU (MCI)	BLEU-4 (MCI)	CIDEr-D (MCI)
Forest-Change	67.10	40.17	38.79
LEVIR-MCI-Trees	88.13	34.41	48.69
LEVIR-MCI (full)	86.43	65.95	140.29

Ablation studies confirm that omitting low-level features degrades boundary precision (IoU drop of 3–5%) and removing high-level features diminishes caption quality (BLEU-4 decrease of 4–6 points) (Brock et al., 8 Jan 2026). The BI³ layer, particularly the integration of LPE and GDFA, is critical to discriminative change representation (Liu et al., 2024).

In attribution-based MCI, simulations show low variance and high accuracy in linear settings, while fine-grained Shapley attributions are computationally feasible (exact for $d \leq 30$ features, scalable through sampling), and match domain intuition in real-world datasets (Budhathoki et al., 2022).

6. Practical Applications and Interactive Workflows

MCI systems are deployed in interactive vision-language agent frameworks. In remote sensing, the end-to-end workflow is:

1. User issues a complex, domain-specific query.
2. LLM agent parses and orchestrates necessary operations (e.g., MCI invocation for mask and caption, post-processing with statistical tools).
3. System returns annotated images and detailed, context-aware explanations, enhancing interpretability for practitioners (Brock et al., 8 Jan 2026, Liu et al., 2024).

In model explanation, MCI methods enable practitioners to attribute changes in a target variable (e.g., individual wage) either to input feature changes or model mechanism drift, at any desired granularity, with theoretical guarantees of dummy (zero attribution for non-changed causes) and efficiency (sum to total change) axioms (Budhathoki et al., 2022).

7. Limitations, Open Problems, and Future Directions

Major limitations of current MCI models include:

• Under-detection of small, fragmented change patches.
• Captioning quality is partially reliant on rule-based templates; domain-adaptive vocabulary expansion and more expressive, generative captioning remain open issues (Brock et al., 8 Jan 2026).
• Quantitative change estimation, such as area or directionality, is insufficiently addressed by purely supervised MCI schemes.
• Efficient, scalable computation of fine-grained Shapley attributions remains a challenge for high-dimensional feature spaces in attribution-based MCI (Budhathoki et al., 2022).

Future research may explore domain-transferable semantics via LLM refinement, improved multi-scale feature fusion for complex scenes, and broader application of MCI principles across diverse scientific and explainable AI contexts. Robust, open-source datasets and workflows released by recent efforts are expected to drive further advances in both vision-language RSICI and interpretable ML attribution (Brock et al., 8 Jan 2026, Liu et al., 2024, Budhathoki et al., 2022).