HOLOGRAPH: Active Causal Discovery via Sheaf-Theoretic Alignment of Large Language Model Priors

Abstract: Causal discovery from observational data remains fundamentally limited by identifiability constraints. Recent work has explored leveraging LLMs as sources of prior causal knowledge, but existing approaches rely on heuristic integration that lacks theoretical grounding. We introduce HOLOGRAPH, a framework that formalizes LLM-guided causal discovery through sheaf theory--representing local causal beliefs as sections of a presheaf over variable subsets. Our key insight is that coherent global causal structure corresponds to the existence of a global section, while topological obstructions manifest as non-vanishing sheaf cohomology. We propose the Algebraic Latent Projection to handle hidden confounders and Natural Gradient Descent on the belief manifold for principled optimization. Experiments on synthetic and real-world benchmarks demonstrate that HOLOGRAPH provides rigorous mathematical foundations while achieving competitive performance on causal discovery tasks with 50-100 variables. Our sheaf-theoretic analysis reveals that while Identity, Transitivity, and Gluing axioms are satisfied to numerical precision (<10^{-6}), the Locality axiom fails for larger graphs, suggesting fundamental non-local coupling in latent variable projections. Code is available at https://github.com/hyunjun1121/holograph.

• The paper introduces a sheaf-theoretic framework that formalizes LLM-derived causal priors as presheaf sections to guide active causal discovery.
• It employs Algebraic Latent Projection and natural gradient descent, demonstrating reduced SHD and improved identification on both synthetic and real-world datasets.
• Empirical results reveal performance gains and expose non-locality challenges from latent confounders, highlighting directions for future causal inference research.

HOLOGRAPH: Sheaf-Theoretic Causal Discovery with LLM Priors and Algebraic Latent Projections

Overview

The HOLOGRAPH framework introduces a formal, sheaf-theoretic approach for integrating LLM-based causal priors into causal structure discovery. The method addresses both the lack of rigorous coherence treatment in existing heuristic integrations of LLM outputs and the complexity introduced by latent confounding structures. HOLOGRAPH formalizes local causal information as presheaf sections, synthesizes these into a global causal model by minimizing descent condition violations, and incorporates an algebraic projection machinery to handle the non-locality induced by unobserved variables.

Sheaf-Theoretic Framework for Causal Integration

Central to HOLOGRAPH is the representation of local LLM-derived causal beliefs as sections of a presheaf over the variable subsets, specifically as linear SEM parameters $(W, M)$. The use of ADMGs rather than DAGs allows explicit modeling of bidirected edges (latent confounders), capturing a richer causal semantics.

The Algebraic Latent Projection mechanism is used to define restriction morphisms between different variable sets, ensuring that hidden variables are analytically marginalized according to their contribution to the observed subgraph. The absorption matrix encodes the cumulative effect of indirect, confounder-mediated paths, as justified through a convergent power series requiring spectral radius constraints on the hidden submatrix.

Critical for coherence is the Frobenius descent loss, which penalizes disagreement between the projections of overlapping local beliefs. This is further regularized with spectral penalties and a continuous acyclicity constraint based on the NOTEARS objective, ensuring tractable optimization in the space of acyclic structures.

Optimization and Query Selection

HOLOGRAPH employs natural gradient descent with Tikhonov regularization to account for the Riemannian geometry of the belief manifold, achieving faster and more stable convergence compared to standard SGD, as confirmed in ablation studies.

An active expected free energy (EFE) strategy is used for querying the LLM. Unlike random or fixed queries, the EFE-based approach strongly prioritizes high-uncertainty or structurally contentious edges, accelerating the reduction of epistemic uncertainty and focusing the computational budget on impactful interventions.

Empirical Results and Robustness

Across synthetic (Erdős-Rényi, scale-free) and real-world datasets (Sachs protein signaling), HOLOGRAPH demonstrates improvements in SHD, F1, and SID metrics compared to NOTEARS and DEMOCRITUS. For instance, on ER-50, HOLOGRAPH achieves an SHD reduction from 236 (NOTEARS) and 218 (DEMOCRITUS) to 193, and shows clear dominance in identifying a larger set of interventional frontiers due to its ADMG backend.

Ablation results underscore the necessity of natural gradients, LLM priors, and informative query selection. For example, removing LLM guidance increases loss by approximately 19×, and omitting natural gradients yields up to 9× worse performance. The framework is especially effective in domains where rich external prior knowledge is available (e.g., the Sachs network).

Theoretical Analysis: Sheaf Axioms and Non-locality

A systematic empirical verification of sheaf axioms reveals that HOLOGRAPH achieves numerical satisfaction (<10⁻⁶ error) of identity, transitivity, and gluing axioms, but observes order-unity violations of the locality axiom that scale with problem size. This non-locality is traced to the effect of latent confounders, substantiating that the presheaf of ADMGs under algebraic projection is not a sheaf in the classical sense. The result implies that local consistency cannot guarantee global coherence in the presence of hidden variables—a fundamental challenge for any compositional learning framework in causal discovery.

Implications and Future Directions

HOLOGRAPH's formal integration of LLM priors within a sheaf-theoretic optimization construct has both practical and theoretical implications:

• Enhanced Causal Identification: The explicit modeling of bidirected edges and the corresponding identification frontier expansion enables discovery of causal relations that remain indistinguishable under DAG-alone hypotheses.
• Non-local Coupling Detection: The observed locality axiom violations signal the limitations of any approach based solely on local restrictions and highlight the need for novel treatments of causal non-locality, potentially via non-commutative sheaf cohomology.
• Active Learning Efficiency: EFE-driven querying demonstrates substantial gains in efficient resource allocation for LLM-in-the-loop discovery.
• Limitations: Performance degradation is observed in larger synthetic graphs and with increased hidden confounder complexity, mainly due to search stochasticity and random initialization sensitivity.

Speculatively, the analogy drawn between causal non-locality in latent confounding and quantum mechanical non-locality suggests that advanced sheaf-theoretic or cohomological frameworks developed in quantum foundations may be fruitfully adapted for causal inference.

Conclusion

HOLOGRAPH provides a mathematically rigorous, empirically validated framework for active, LLM-guided causal discovery. By operationalizing LLM priors as sections in a presheaf of SEMs and enforcing global consistency via descent conditions, the method advances both causal model identifiability and the theoretical understanding of the limits of local reasoning in causal inference. The systematic failure of the locality axiom in the presence of latent confounders flags a significant barrier for classical sheaf-theoretic factorization, motivating further research into non-local and cohomological approaches for causal discovery. Potential future extensions include non-commutative cohomology for capturing obstructions, hybrid constraint-based methods, and deeper integration with intervention and counterfactual computation.

Reference: "HOLOGRAPH: Active Causal Discovery via Sheaf-Theoretic Alignment of LLM Priors" (2512.24478)