AI Explanation Design: Frameworks & Methods

• AI-DEC is a multidisciplinary field that develops systematic frameworks and taxonomies to design, evaluate, and deploy user-tailored AI explanations.
• It combines algorithmic techniques with interactive design to provide both local and global insights that enhance accountability and trust.
• Practical applications span domains like healthcare and legal decision support, emphasizing human-centered evaluation and adaptive presentation methods.

AI Explanation Design (AI-DEC) is a multifaceted research area at the intersection of explainable artificial intelligence (XAI), human-computer interaction, decision support, and broader sociotechnical concerns. It encompasses systematic frameworks, formal methodologies, and empirically grounded principles for constructing, evaluating, and deploying explanations that make AI models accountable, trustworthy, and actionable for both technical and non-technical stakeholders.

1. Conceptual Foundations and Taxonomies

AI-DEC draws from several decades of work in explanation sciences, XAI, and cognitive systems engineering. A comprehensive taxonomy of explanation types is central. These include:

• Feature-based (e.g., saliency maps, SHAP/LIME attributions)
• Example-based (e.g., prototypes, counterfactuals, case comparisons)
• Rule-based (e.g., decision rules, logical paths, tree paths)
• Contextual and performance-derived (e.g., confidence scores, dataset-level statistics)
• Contrastive/causal and narrative forms (e.g., “Why this not that?” “What-if?” explanations)

Key taxonomies distinguish between local explanations (instance- or sample-specific rationales) and global explanations (system- or cohort-level summaries or mechanisms) (Mamalakis et al., 2024, Jin et al., 2023). The Explanation Ontology (EO) formalizes these types as ontological classes, enabling run-time and design-time mapping from user needs to explanation artifacts via logical rules and SPARQL queries (Chari et al., 2020).

A pivotal insight is that explanation forms must be mapped to user goals, context, and system capability vectors; for example, EO’s mapping function

$$f: (\textsf{UserAttribute} \times \textsf{SystemCapability}) \to \textsf{ExplanationType}$$

formalizes this alignment.

2. Frameworks and Methodologies

AI-DEC frameworks operationalize explanation requirements via multidimensional models. Cornerstone frameworks include:

• Four-class Necessity System (for healthcare AI): self-explainable, semi-explainable, non-explainable, and new-patterns discovery. Each is defined by the robustness of evaluation protocols, inter-observer variability (measured by Cohen’s $\kappa$ or DSC), and representation dimensionality (2D tabular/low, 3D/time-series/medium, multimodal/high). These determine whether explanations are unnecessary, local-only, or mandate additional global/statistical alignments (Mamalakis et al., 2024).
• Explanation Card Designs: AI-DEC as a card-based co-design method partitions explanation design along four critical dimensions: Content (what), Modality (how), Frequency (when), and Direction (interaction flow). Each can be instantiated and recombined via deck selection to match domain and user need (Lee et al., 2024).
• Self-Explanation Scorecard: A level-based ordinal framework for evaluating whether a system supports only surface visualizations, or also exposes successes, mechanisms, reasoning, errors, comparisons, and root-cause diagnostics. This metric is used for both formative and summative system evaluation (Mueller et al., 2021).

Formalisms are generally modular, supporting adaptability to domain (e.g., healthcare, finance, high-stakes decision-making) and user type (expert clinicians, operators, laypersons, auditors). Theories are anchored in cognitive science constructs of mental models, abduction, and conversational collaboration (Mueller et al., 2019, Mueller et al., 2021).

3. Explanation Generation and Presentation Techniques

Explanation generation leverages a diverse array of algorithmic and visual techniques:

• Local attribution methods: LIME, SHAP, Grad-CAM, LRP, with outputs formatted for sample-specific interpretability (Mamalakis et al., 2024, Speckmann et al., 26 Jun 2025). Output forms include bar charts, heatmaps, counterfactual side-by-side views, or narrative rule summaries.
• Global aggregation: Cohort-wide PCA on explanation maps, or statistical alignment with ground-truth models (e.g., shape model alignment for new marker discovery).
• Selective and adaptive presentation: Techniques such as X-Selector minimize information overload and maximize user-AI alignment by optimizing, for each trial, the subset of explanations that steer the user toward the AI’s suggestion, parameterized by learned user-behavioral models (Fukuchi et al., 2024).
• Interactive UIs and explanation pipelines: Modular interfaces (IXAII) let users switch between input introspection, “Why,” “Why Not,” “What If,” and “When” queries, across multiple visualization formats and user roles (Speckmann et al., 26 Jun 2025). Co-design methods employ decks that let users select explanation cards according to their operational context (Lee et al., 2024).
• Behavioral cue integration: In collaborative and UX tasks, explanations integrate rule-based panels (e.g., Nielsen heuristics) and multi-modal features (prosody, sentiment, events), optimizing timing (sync/async) for agency and trust calibration (Fan et al., 2021).

An emphasis is placed on the modular combination of explanation forms, layered “progressive disclosure” of detail, and the decoupling of explanation content from its presentation modality.

4. Human Factors and Evaluation Metrics

AI-DEC prioritizes human-centered effectiveness over mere algorithmic explainability. Evaluation metrics are multi-level:

• Explanation goodness: Fidelity, completeness, non-overwhelm, reversibility.
• Performance: Task accuracy, response time, debugging efficiency, joint human-AI outcome.
• Mental model alignment: User’s post-task knowledge, prediction alignment, concept mapping scores.
• Trust calibration: Agreement and switch metrics, correlation of user trust to model confidence, reduction in automation bias or algorithm aversion (Zhang et al., 2020).
• Satisfaction and usability: Likert-scale ratings on understanding, transparency, and cognitive load (Schleibaum et al., 2024, Speckmann et al., 26 Jun 2025).
• Goal achievement: Calibration of trust, detection of bias, improvement of outcome, stakeholder communication, and safety verification (Jin et al., 2023).

Human-subject studies and iterative user-centered evaluation are required for empirical validation, as postulated in both the literature meta-reviews and applied method papers (Mueller et al., 2019, Shin et al., 2021).

5. Domain-Specific Patterns and Case Studies

AI-DEC has been tested and refined in high-stakes applications:

• Healthcare: The four-class explanation necessity system formalizes when/what explanations are needed, with concrete case studies demonstrating practical application (e.g., Alzheimer’s classification—semi-explainable/local-only, Grad-CAM heatmaps; pulmonary hypertension—new-patterns discovery, local+global+statistical alignment) (Mamalakis et al., 2024).
• Human-AI collaborative workflows: UX evaluation tasks highlight differential needs for explanation granularity and synchronization, surfacing domain-specific principles for effective evaluator-AI collaboration (Fan et al., 2021).
• Decision support and fairness: Evidence-based timeline explanations for judicial and risk contexts provide stepwise transparency across events and support procedural fairness (Ferreira et al., 2020).
• Critical tasks with end-users: Studies emphasize the tailoring of feature-based, example-based, and rule-based explanation forms to user roles and explanation goals (trust, safety, bias, learning, reportability) (Jin et al., 2023).

A frequent pattern is the need to support both expert and novice users with appropriately layered or role-configured explanation content and modalities (Speckmann et al., 26 Jun 2025, Lee et al., 2024).

6. Design Principles, Guidelines, and Open Challenges

A synthesis of the literature yields robust, empirically-grounded design principles for AI-DEC:

• Align explanations with domain semantics and user expertise: Use domain-specific terminology, granularity control, and allow interactive querying and feedback loops.
• Support contrastive and causal/narrative explanation types: Present “why not,” “what if,” and side-by-side case comparisons.
• Optimize for trust calibration, agency, and cognitive load minimization: Use confidence quantification, adaptive/just-in-time explanation timing, scannable and progressive disclosure UIs.
• Prioritize modularity and extensibility: Architect explanation pipelines to permit new XAI modules, visualization methods, and user role configurations with minimal rework.
• Embed explanation design in co-design workflows with concrete strategy cards or ontological templates: Bridge between user needs and AI system capabilities (Lee et al., 2024, Chari et al., 2020).

Challenges remain in quantifying procedural fairness, scaling narrative and evidence-based explanations to complex, automated settings, and dynamically calibrating explanation delivery to match evolving tasks, contexts, and regulatory requirements (Ferreira et al., 2020, Chari et al., 2020, Mueller et al., 2021). Future research directions include empirical behavioral grounding of explanation strategies and integration with formal reasoning and compliance mechanisms.

---

References:

(Mamalakis et al., 2024, Fan et al., 2021, Shin et al., 2021, Ferreira et al., 2020, Ferreira et al., 2021, Fukuchi et al., 2024, Zhang et al., 2020, Lee et al., 2024, Schleibaum et al., 2024, Chari et al., 2020, Mueller et al., 2019, Speckmann et al., 26 Jun 2025, Mueller et al., 2021, Jin et al., 2023)