SP-LIME: Submodular Pick for Explaining Models

• The paper introduces SP-LIME which aggregates local explanations into a concise, globally representative set using submodular optimization.
• The methodology optimizes feature coverage by selecting diverse explanations, minimizing redundancy and providing clearer insights into model decisions.
• Empirical evaluations show improved efficiency and clarity in model explanations, particularly in image classification tasks like underwater SONAR analysis.

Submodular Pick for Explaining Models (SP-LIME) is a principled method for selecting a concise, globally informative set of model explanations by optimizing a submodular coverage criterion over local explanations generated by Local Interpretable Model-agnostic Explanations (LIME). Unlike LIME, which produces a local explanation for a single prediction, SP-LIME aims to summarize model behavior on an entire dataset by greedily choosing a limited subset of instances whose explanations collectively maximize coverage of important, diverse model reasoning patterns. This approach supports enhanced interpretability for complex black-box models, particularly in domains where understanding model reasoning is critical, such as underwater SONAR image classification and other high-security applications (Natarajan et al., 2024, Ribeiro et al., 2016).

1. Local and Global Explanation: LIME versus SP-LIME

Standard LIME generates post-hoc, instance-wise explanations by fitting interpretable local models (typically sparse linear) around a sample $x$ through local perturbations and weighting, yielding a set of feature importances that describe how the model behaves in the vicinity of $x$ (Ribeiro et al., 2016). While effective for assessing individual predictions, this local focus does not characterize the model’s decision boundaries or global feature usage.

SP-LIME extends LIME by constructing explanations for multiple data points, then selecting a subset $S$ (of cardinality budget $B$) such that the union of their explanations maximizes a submodular coverage function. The formal selection objective is: $\max_{S\subseteq P,\;|S|\leq B} f(S)$ where $P$ is the pool of candidate instances, $B$ is the explanation budget, and $f(S)$ quantifies the global coverage across interpretable features. This process avoids redundancy and ensures that important, diverse model behaviors are represented in the explanation set (Natarajan et al., 2024, Ribeiro et al., 2016).

2. Submodular Optimization Formulation

SP-LIME formulates the explanation selection as a monotone submodular maximization problem subject to a cardinality constraint:

• Candidate Pool: $P = \{1, \ldots, n\}$ indexes $n$ candidate instances, each with a local LIME explanation.
• Coverage Function: $f: 2^{P} \rightarrow \mathbb{R}$ is a non-negative, monotonic, submodular set function.
• Optimization Problem: Select

$S^* = \arg\max_{S\subseteq P,\;|S|\leq B}\; f(S)$

where

$f(S) = \sum_{j=1}^d \max_{i\in S} |w_{i,j}|$

for local explanation weight vectors $|w_i| \in \mathbb{R}^d$ (absolute importance of feature $j$ in explanation for instance $i$) (Natarajan et al., 2024, Ribeiro et al., 2016).

The greedy algorithm, which at each step adds the candidate with maximal marginal gain, guarantees a $(1-1/e)$ approximation to the optimal coverage (Ribeiro et al., 2016).

3. Feature Coverage and Redundancy

The submodular coverage function $f(S)$ at the core of SP-LIME encourages the selected instance explanations to collectively emphasize important features (as measured by high absolute explanation weights) yet discourages redundant coverage. For each interpretable feature $j$, only the largest weight across $S$ is counted, incentivizing selection of diverse explanations that together span essential components of model reasoning. In cases where an interpretable feature (e.g., a super-pixel segment in images) is important to multiple explanations, it is efficiently “covered” via the highest local attribution in $S$. This methodology enables the user to observe a compact, representative view of model behavior rather than multiple similar or overlapping explanations (Ribeiro et al., 2016).

4. Image-Specific Instantiation: Super-pixels and Segmentation Algorithms

In image classification tasks, interpretable features are constructed via super-pixel segmentation. SP-LIME implementations for images utilize popular segmentation methods:

• Quickshift: Mode-seeking density-based clustering over color–position domain.
• SLIC (Simple Linear Iterative Clustering): K-means–like clustering in $(L^*,a^*,b^*,x,y)$ space. Segments are treated as binary interpretable features (on/off), with LIME perturbing images by alternately masking super-pixels and recording model prediction responses. For each segment, the absolute explanation weight is extracted from a sparse local linear model. SP-LIME then constructs the feature-importance coverage matrix for a pool of images and applies greedy submodular maximization to select explanations (Natarajan et al., 2024).

A sketch of the process:

• Segment image into super-pixels ($d$ features).
• For each segment, produce $N$ perturbed images by masking segments.
• For each perturbed image, compute prediction with the black-box classifier.
• Fit sparse local linear model and extract absolute segment weights.
• Aggregate explanation weight vectors, then maximize submodular coverage over the pool (Natarajan et al., 2024).

5. Algorithmic Pipeline and Complexity

The computational steps in SP-LIME include:

• Generating $N$ perturbed samples per image and evaluating the model.
• Fitting a (sparse) weighted linear model per instance in $O(Nd)$.
• Constructing the $n \times d$ explanation weight matrix.
• For the selection phase, the greedy maximization requires $O(B n d)$ operations: at each of $B$ steps, evaluating the marginal gain over all candidates and features. The resultant solution achieves at least $(1-1/e)$ of the optimum (Ribeiro et al., 2016).

In the context of underwater SONAR image analysis, typical parameter settings include:

• Quickshift with kernel size 2, max distance 100, ratio 0.1
• $B = 1$ (can be generalized to larger budgets)
• $N = 300$ perturbation samples per image
• $d = 10$ top segments per image Wall-clock time for a single SP-LIME explanation: 150 seconds (17 seconds for sampling, remainder for fitting and selection). Runtime scales linearly with number of samples and increases mildly with the number of features (Natarajan et al., 2024).

6. Empirical Evaluation and Interpretability Outcomes

Empirical studies on a class-balanced, stratified underwater SONAR dataset (6 classes, DenseNet121 backbone) yield a test accuracy of 98.21%. Qualitative evaluation of SP-LIME explanations demonstrates that, compared to standard LIME, SP-LIME produces tighter masks, avoids extraneous regions (e.g., shadows in the “Plane” class), and preserves coverage of object-relevant image areas. Under the experimental setting (10 features, 500 samples), SP-LIME exhibits lower runtime (20 s) than LIME (31 s) and Quickshift yields slightly more precise region masks than SLIC (20.00 s vs. 20.47 s).

For well-represented classes (plane, ship, seafloor), masks from SP-LIME focus on object bodies, providing a strong "trust signal." Classes with insufficient real data (fish, mine, human) display less accurate explanations, reflecting overfitting to augmented samples and sparse class representation. The study does not report quantitative faithfulness or fidelity metrics, instead relying on visual and runtime analysis (Natarajan et al., 2024).

7. Strengths, Limitations, and Prospects

SP-LIME offers substantial benefits:

• Condenses numerous local explanations into a select, globally informative subset, maximizing feature coverage and minimizing redundancy.
• Enhances scalability and interpretability with super-pixel segmentation.
• Enables crisp, human-comprehensible visual explanations on robust classes in under 30 seconds.

Documented limitations include:

• Difficulty in producing credible explanations for classes with few true samples or reliance on augmented data.
• Absence of quantitative explanation fidelity evaluation in some domains, such as underwater SONAR.

A plausible implication is that SP-LIME’s global selection procedure can be sensitive to the distribution, quality, and diversity of the input data pool; enriched real-world datasets and advanced augmentation techniques (e.g., ESRGAN) may further enhance explanation fidelity. Future directions involve integrating such approaches to improve representativeness and trustworthiness across all class categories (Natarajan et al., 2024, Ribeiro et al., 2016).