eXIAA: eXplainable Injections for Adversarial Attack

Abstract: Post-hoc explainability methods are a subset of Machine Learning (ML) that aim to provide a reason for why a model behaves in a certain way. In this paper, we show a new black-box model-agnostic adversarial attack for post-hoc explainable Artificial Intelligence (XAI), particularly in the image domain. The goal of the attack is to modify the original explanations while being undetected by the human eye and maintain the same predicted class. In contrast to previous methods, we do not require any access to the model or its weights, but only to the model's computed predictions and explanations. Additionally, the attack is accomplished in a single step while significantly changing the provided explanations, as demonstrated by empirical evaluation. The low requirements of our method expose a critical vulnerability in current explainability methods, raising concerns about their reliability in safety-critical applications. We systematically generate attacks based on the explanations generated by post-hoc explainability methods (saliency maps, integrated gradients, and DeepLIFT SHAP) for pretrained ResNet-18 and ViT-B16 on ImageNet. The results show that our attacks could lead to dramatically different explanations without changing the predictive probabilities. We validate the effectiveness of our attack, compute the induced change based on the explanation with mean absolute difference, and verify the closeness of the original image and the corrupted one with the Structural Similarity Index Measure (SSIM).

• The paper introduces eXIAA, a model-agnostic adversarial attack that injects targeted attribution perturbations to significantly alter explanation maps without changing prediction outcomes.
• It employs a three-phase methodology—attack image selection based on the running-up class, feature extraction, and α-weighted injection—to achieve high explanation disruption with minimal perceptual change.
• Empirical results on ResNet-18 and ViT-B16 using ImageNet and CIFAR-10 highlight the method’s effectiveness, exposing vulnerabilities in current post-hoc XAI methods, especially in transformer architectures.

eXIAA: Model-Agnostic Adversarial Attacks on Post-hoc XAI via Targeted Attribution Injections

Introduction and Motivation

The paper introduces eXIAA, a systematic black-box adversarial attack that targets post-hoc explainability methods in computer vision models. Notably, eXIAA disrupts the explanation of model predictions without access to model internals—requiring only prediction outcomes and corresponding XAI attributions. Unlike prior adversarial attacks on explanations that demand white-box access and iterative optimization, eXIAA executes in a single injection step. The threat model is significant: explanations can be manipulated in a way that is nearly visually imperceptible and leaves predicted classes unchanged, thus exposing the fragility of current XAI methods in high-stakes domains such as healthcare and autonomous systems.

Attack Methodology

Technical Pipeline

The eXIAA pipeline consists of three phases: Attack Image Selection, Feature Extraction, and Attribution Injection. The process is illustrated in detail as follows.

Figure 1: Adversarial attack pipeline leveraging post-hoc explanations and targeted attribution injection from a running-up class image.

1. Attack Image Selection: Given a source image $x$ and its classified label $y^* = \arg\max_j f_j(x)$, the attack identifies the "running-up" class $y_r$ with the highest predictive confidence after $y^*$. An attack image $\bar{x}$ from $y_r$, with maximal $f_{y_r}(\bar{x})$, is selected.
2. Feature Extraction: Utilizing the same XAI method $E(\cdot)$ (saliency maps, integrated gradients, or DeepLIFT SHAP), one extracts the top $k$ positive attribution indices $\mathcal{I}_k$ from $\bar{x}$. This process targets prominent, highly informative pixels as determined by the feature attribution.
3. Feature Injection: The selected features are injected into the source image $x$ at the corresponding coordinates using an $\alpha$-weighted blend:

$$\hat{x}_{w,h,c} = \text{clip}\big[(1-\alpha)x_{w,h,c} + \alpha\bar{x}_{w,h,c}\big] \quad \forall (w,h,c) \in \mathcal{I}_k$$

The remaining pixels retain their original values. $\alpha$ is a hyperparameter controlling the perturbation magnitude, typically set low for minimal perceptual distortion.

Quantitative Evaluation

eXIAA was validated using pre-trained ResNet-18 and ViT-B16 architectures on ImageNet and CIFAR-10, attacking explanations with saliency, integrated gradients, and DeepLIFT SHAP.

Manipulation of Explanations

Figure 2: Percentage change in explanations as a function of the top-k injected features and injection strength $\alpha$ for various classifier/XAI method pairs.

eXIAA achieved large absolute differences in explanation attributions without modifying the predicted label. Notably, transformer architectures (ViT-B16) exhibited higher vulnerability, producing more drastic explanation changes for equivalent parameter settings.

Perceptual Indistinguishability

Figure 3: SSIM scores measuring structural similarity between original and corrupted images across injection strengths and attribution sparsity.

Injected perturbations maintained high SSIM values (typically > 0.97), confirming negligible perceptual differences.

Preservation of Class Predictions

Figure 4: Absolute change in confidence for the predicted class between original and adversarial samples.

Even at higher $\alpha$ and top-k values, the average confidence drop remained below 10%, often $<5\%$ for practical settings, indicating that the attack preserves the model’s prediction.

Empirical Insights: Running-Up Class Strategy

eXIAA’s rationale for selecting the running-up class was corroborated.

Figure 5: Attacks using the running-up class yield greater explanation manipulation versus random class selection in CIFAR-10.

Injected features from the most-confused class proved most disruptive to the original explanation, consistent with model behavior under uncertainty.

Moreover, selecting attack images with highest class confidence maximizes attribution strength:

Figure 6: Attack efficacy is greatest with top-confidence images from the running-up class compared to low-confidence images.

Discussion

Performance and Trade-offs

• Numerical Findings:
  • For DeepLIFT SHAP, explanation change for ViT-B16 exceeded 180% with confidence drops $<13\%$ at $\alpha = 0.15$, top $k = 0.8$.
  • Results were consistent across saliency and integrated gradients, with transformer models more susceptible to explanation manipulation.
• Scaling:

The pipeline scales linearly in class count; thus, direct analogs in ImageNet are feasible only with class subsampling (as shown for CIFAR-10).

Attack Limitations and Mitigation

• The attack exploits direct correlations between high-confidence predictions and attribution strength in XAI methods: alternative blending algorithms or local pixel-aware mixing could enhance imperceptibility further.
• Multi-label scenarios require new strategies (e.g., selecting common running-up classes or multi-class attributions).
• Defense prospects include fine-tuning models on adversarial samples or modifying XAI computation pipelines for robustness to targeted attribution injections.

Implications

eXIAA highlights that current post-hoc XAI algorithms are susceptible to robust black-box adversarial manipulation, undermining their reliability in practice. In domains where explanations support critical human decisions, explanation integrity must be established alongside prediction accuracy. Efforts to enhance XAI robustness, either algorithmically or architecturally, are a necessary research direction for trustworthy AI.

Conclusion

eXIAA delivers a potent, model-agnostic adversarial attack capable of dramatically altering the explanations output by leading XAI feature attribution methods without modifying model outputs or perceptual characteristics. The method’s simplicity and efficacy expose systemic vulnerabilities in current interpretability frameworks. This work underscores the necessity of further research into explanation robustness, especially for safety-critical applications, and provides a rigorous empirical methodology for testing and improving XAI systems.