Position: Explainable AI is Causality in Disguise

Abstract: The demand for Explainable AI (XAI) has triggered an explosion of methods, producing a landscape so fragmented that we now rely on surveys of surveys. Yet, fundamental challenges persist: conflicting metrics, failed sanity checks, and unresolved debates over robustness and fairness. The only consensus on how to achieve explainability is a lack of one. This has led many to point to the absence of a ground truth for defining ``the'' correct explanation as the main culprit. This position paper posits that the persistent discord in XAI arises not from an absent ground truth but from a ground truth that exists, albeit as an elusive and challenging target: the causal model that governs the relevant system. By reframing XAI queries about data, models, or decisions as causal inquiries, we prove the necessity and sufficiency of causal models for XAI. We contend that without this causal grounding, XAI remains unmoored. Ultimately, we encourage the community to converge around advanced concept and causal discovery to escape this entrenched uncertainty.

• The paper demonstrates that Structural Causal Models are both necessary and sufficient for robust and reliable XAI explanations by aligning causal queries with Pearl’s hierarchy.
• The paper formalizes XAI questions into observational, interventional, and counterfactual queries, exposing the limitations of purely statistical and heuristic-based methods.
• The paper advocates for integrating causal discovery and human-in-the-loop refinement to overcome computational challenges and improve model interpretability.

Explainable AI is Causality in Disguise: A Formal Perspective

Introduction

The field of Explainable AI (XAI) is characterized by an abundance of methods, definitions, and evaluative criteria, yet remains decisively fragmented. "Position: Explainable AI is Causality in Disguise" (2603.28597) posits that this lack of consensus is not due to the nonexistence of ground truth explanations. Rather, the ground truth exists but is deeply embedded in the causal structure of the system under investigation. The central thesis is that XAI fundamentally reduces to questions of causality: the only rigorous explanations are those anchored in a causal model—specifically, a Structural Causal Model (SCM). The paper claims and formally establishes that causal models are both necessary and sufficient for exhaustive, reliable explainability, thereby shifting the community’s focus from proliferating ad hoc XAI heuristics to engaging directly with the causal inference problem.

Causal Foundations of XAI

The conceptual contribution is a rigorous alignment of XAI question taxonomies with layers of Pearl’s causal hierarchy: observational, interventional, and counterfactual. The author structures XAI queries into three classes, each mapped to a causal query type:

• Data-based (What?): Queries concerning the generative factors and distributions underlying observed data, mapped to observational queries ($P(X,Y)$).
• Model-based (How?): Queries about the internal mechanisms or transformations within a model, aligned with interventional queries ($P(Y \mid do(X))$).
• Decision-based (Why/Why not?): Case-specific causal reasoning, explicitly requiring counterfactual analysis ($Y_{X=x'}(u)$).

This alignment is nontrivial: by demonstrating that the informational content required to answer XAI questions strictly expands from association to intervention to counterfactuals, the paper exposes the fundamental inadequacy of any approach restricted to purely statistical or correlational analysis.

Formal Results: Necessity and Sufficiency

The technical core comprises two main theorems:

• Sufficiency: Access to the true underlying SCM is sufficient for providing accurate and complete answers to all core XAI questions. The SCM deterministically induces the observational, interventional, and counterfactual distributions necessary for explanation.
• Necessity: For any system, unless the assumed model precisely matches the true SCM, there will exist XAI queries (from observational to counterfactual) for which explanations produced from the alternative model are systematically incorrect. The minimal separating family of XAI queries effectively identifies any model misspecification at a causal level.

The proofs rest on the separation property: for any two distinct SCMs, some query from the suite ($Q1$–$Q6$) will yield divergent results. Thus, only causally correct models guarantee universally valid explanations.

Critique of Existing XAI Methodology

The implications are substantial and sharply critical of the current landscape:

• Lack of Causal Grounding: Most extant XAI methods, including popular feature attribution and saliency approaches, implicitly or explicitly neglect environment or model causality, limiting their validity and robustness—especially in distribution shift or intervention scenarios.
• Axiomatic Framework Limitations: While axiomatic approaches attempt to define XAI in terms of desirable statistical properties, foundational work demonstrates that these fail basic sanity checks without causal underpinning.
• Heuristic Proliferation: The author asserts that the perpetuation of non-causal XAI techniques and evaluation paradigms should be viewed as evasion: without a causal target, claims of “explainability” are not falsifiable.

Pathways Toward Causal XAI

The operational challenge is then reframed as one of causal discovery and abstraction:

• Concept Discovery: Identifying stakeholder-aligned, semantically meaningful variables at the right level of causal abstraction is essential. The author advocates for integrating approaches such as concept bottleneck models, TCAV, and neuro-symbolic frameworks to bridge the gap between uninterpretable low-level features and actionable, causally meaningful abstractions.
• Relation Discovery: Causal structure learning and representation learning are required to uncover directed relationships among identified variables. Doing so at scale necessitates both algorithmic innovation (e.g., scalable constraint or score-based discovery, approximate and online learning) and interactive refinement (human-in-the-loop for resolving underdetermined causal directions and abstractions).

The paper argues that, in high-dimensional or opaque models, approximate causal models—refined with user interaction—may provide actionable robustness, with partial but causally principled explanations that outperform any exclusively statistical heuristic.

Robustness, Limitations, and Open Challenges

Several limitations are acknowledged:

• Computational Complexity: Identifying true SCMs is NP-hard with known intractability for large-scale systems. Theoretical guidance is proposed via $\epsilon$-approximate models and sensitivity analysis.
• Identifiability Barriers: Without randomized interventions or completeness of observational data, only Markov equivalence classes may be identifiable—structural ambiguity that only interventional or counterfactual queries can resolve.
• Human Mental Models: While SCMs provide formal rigor, their expressiveness for capturing intuitive human reasoning and concept formation is debatable. The paper notes that future research in causal abstraction and neuro-symbolic modeling may be critical for aligning computational explanations with human cognition.

Implications and Future Directions

The paper’s position—that progress in XAI is tantamount to progress in causal discovery and representation learning—demands a paradigm shift in both methodology and evaluation:

• Practical XAI methods must, at a minimum, make explicit the causal assumptions they encode.
• Evaluation protocols should focus on falsifiable, interventionist criteria rather than agreement with ad hoc human expectations.
• Research investment should be redirected from incremental improvements to correlation-based heuristics toward the explicit development of scalable, stakeholder-aligned causal discovery engines.

The field's trajectory is thus coupled to advances in approximate causal inference, causal representation learning, and interactive causal systems.

Conclusion

"Position: Explainable AI is Causality in Disguise" (2603.28597) reconceptualizes XAI as a fundamentally causal endeavor, providing formal proofs that only causal models can guarantee reliable, robust, and actionable explanations across the full taxonomy of explainability queries. The work’s principal implication is that progress in XAI is limited by progress in causal modeling and discovery. Future directions hinge on multi-level causal abstraction, scalable concept and relation discovery, and stakeholder-in-the-loop model refinement. While major challenges remain, the causal formalism articulated provides a rigorous foundation for unifying and advancing the field of XAI.