Identifiable Functional Model Classes (IFMOC)

• IFMOCs are rigorously defined statistical models that enforce identifiability by applying strict functional constraints on causal and latent variable recovery.
• They bridge causal discovery, independent component analysis, and unsupervised learning by formalizing restrictions that ensure unique identification of causal graphs.
• Empirical methods like LCUBE demonstrate that IFMOC-based algorithms can outperform traditional approaches, especially in nonlinear and non-faithful settings.

An Identifiable Functional Model Class (IFMOC) is a rigorously defined class of statistical models in which the functional structure and associated distributional properties enable identification of underlying causal structure, or latent variables, from observed data. IFMOCs represent a principled framework bridging causal discovery, independent component analysis (ICA), and unsupervised learning, by formalizing the function class constraints that guarantee identifiability. These constraints make it possible to recover the correct causal graph, direction of causal influence, or latent factors—often beyond classical limitations such as non-identifiability of Markov equivalence classes or generic nonlinear ICA.

1. Definition and Theoretical Foundations

In the context of causal discovery, an IFMOC is defined as a class of structural or functional models over a given function class $\mathcal{F}$, further constrained by an identifiability condition on bivariate submodels. Concretely, for a set of random variables $X_1, \dots, X_d$ corresponding to vertices $V = \{1,\dots,d\}$ of a DAG $G=(V,E)$, a functional model class expresses each $X_i$ as

$X_i = f_i(\mathrm{PA}_i, N_i)$

where $\mathrm{PA}_i \subset V\setminus\{i\}$ are the parents of $i$, each $f_i \in \mathcal{F}_{|\mathrm{PA}_i|+1}$, and $\{N_i\}$ are mutually independent noise variables.

A $(\mathcal{B},\mathcal{F})$–IFMOC requires, in addition to standard functional model structure, that for every parent–child relationship $(X_j,X_i)$:

• Marginal-fixing closure: After fixing all other parents and residual noise, the resulting bivariate model remains within $\mathcal{F}_2$.
• Local bivariate identifiability: There exists a conditioning where the relevant bivariate triple $(f_i, p_{X_j|\cdots}, p_{N_i})$ belongs to a bivariate identifiable set $\mathcal{B}$. This implies the directionality of the additive-noise (or related) model is generically unique (Peters et al., 2012).

In the nonlinear ICA setting, an IFMOC specifies a function class $\mathcal{F}$ (such as the set of conformal maps or orthogonal coordinate transformations (OCTs)) for the mixing function $f:\mathbb{R}^n \rightarrow \mathbb{R}^n$, often along with an admissible source law family $\mathcal{P}$ and symmetry group $\mathcal{S}$ (e.g., signed permutations, scalings, translations). Identifiability requires that if two different models $f, f' \in \mathcal{F}$ yield indistinguishable observed distributions, they must be related by a trivial symmetry from $\mathcal{S}$, barring degenerate cases (Buchholz et al., 2022).

2. Identifiability Results for IFMOCs

The principal result for IFMOCs is that, under the respective assumptions (e.g., nonlinear additive noise, constraints on the function class, and independence of noise), the entire causal graph or the latent structure is uniquely determined by the joint distribution.

For functional causal models, if data are generated by a $(\mathcal{B},\mathcal{F})$–IFMOC, then no other functional model within the same IFMOC induces the same distribution but with a different DAG. This result establishes identifiability of the complete causal graph, not merely the Markov equivalence class (Peters et al., 2012).

In nonlinear ICA, restriction to certain function classes, such as conformal maps (where $J_f^{\mathsf{T}} J_f = \lambda^2 I$ everywhere), ensures full unconditional identifiability: any two models giving the same observation law must be related by an affine transformation with signed-permutation matrices and scaling (Buchholz et al., 2022).

For bivariate causal direction, identifiability is proven for MDL-based scores using dense function classes ($H \subset C([a,b])$): under mild "low noise" conditions and code-length penalties increasing in parameter count, the score in the causal direction strictly outperforms the anti-causal direction unless $f$ is linear (Hlavackova-Schindler et al., 30 Aug 2025).

3. Classes of IFMOCs and Function Class Constraints

IFMOC construction depends crucially on the choice and properties of the underlying function class. Central concepts include:

• Density: $H$ is dense in $C([a,b])$ if for any continuous function $f$ on $[a,b]$ and any $\epsilon>0$, there exists $h\in H$ with $||f-h||_\infty < \epsilon$. Examples are polynomials, cubic splines with variable knots (Hlavackova-Schindler et al., 30 Aug 2025).
• Differential structure: In ICA, identifiability can be enforced by function class constraints such as conformality or orthogonal coordinate transformations. For instance, the class $\mathcal{F}_\mathrm{conf}$ of conformal maps is characterized by $J_f^\mathsf{T} J_f = \lambda^2 I$ for an invertible $f$ with a scalar field $\lambda$ (Buchholz et al., 2022).

These structural constraints prevent the existence of spurious or nontrivial transformations that would otherwise yield non-identifiable models. In general, the "rigidity" of the Jacobian imposed by these classes is what underpins identifiability.

4. Methodological Implications and Algorithms

The IFMOC paradigm admits practical algorithms for causal discovery and latent variable identification:

• In causal discovery, for each candidate DAG, a suite of regressions is performed to estimate functions $f_i$ from parent sets, followed by tests of independence of residuals from the regressors. Only DAGs for which all such tests succeed are retained; if none or more than one, the algorithm outputs "I do not know" (Peters et al., 2012).
• For bivariate cases, a minimum-description-length (MDL) causal score is computed for each direction (e.g., $X\to Y$ vs $Y\to X$), utilizing function classes dense in the space of continuous functions. LCUBE, a concrete instantiation using cubic regression splines, minimizes a two-part MDL code, operationalizing the IFMOC framework for empirical causal direction recovery (Hlavackova-Schindler et al., 30 Aug 2025).

A representative skeleton for the causal graph recovery procedure is:

For each candidate DAG: For each variable X_i: 1. Regress X_i on its parent set. 2. Test independence of regression residuals from regressors. 3. If any test fails, discard the DAG. Return all DAGs passing all tests, or "I do not know" if ambiguous.

This approach scales polynomially in the number of variables for sparse graphs, and the IFMOC assumption is explicitly testable via observed data (Peters et al., 2012).

5. Empirical Demonstrations and Performance

Empirical validation of IFMOC-based methods demonstrates superior or comparable performance to existing methods:

• On 13 benchmark datasets—including synthetic additive-noise models (AN, AN-s, LS, etc.) and the real-world Tübingen cause-effect pairs—LCUBE achieves the highest Area Under the Direction-ROC Curve (AUDRC) of 87% on Tübingen, 100% on several synthetic sets, averaged 91.5% over 10 common benchmarks, and above-average precision throughout (Hlavackova-Schindler et al., 30 Aug 2025).
• Causal DAG recovery under IFMOC assumptions shows high rates of exact recovery, often outperforming PC-style (Markov+faithfulness) methods, especially on nonlinear or non-faithful settings, and robustly avoids misleading outputs when IFMOC assumptions fail (Peters et al., 2012).
• In nonlinear ICA, conformal and OCT-based classes yield full identifiability (for generic settings), and empirical OCT-regularized flows have demonstrated efficacy in unsupervised representation learning (Buchholz et al., 2022).

A summary of the LCUBE empirical performance is presented below:

Dataset Group	Metric	LCUBE Value
Tübingen cause-effect pairs	AUDRC	87%
Synthetic AN/AN-s/LS/LS-s/MN-U sets	AUDRC/Acc.	100%
10 common benchmarks	Avg. AUDRC	91.5%
All 13 datasets	Relative	Above-average
Runtime (GPU)	Time	Minutes

6. Comparisons and Distinctions with Classical Methods

The IFMOC approach diverges sharply from conditional independence-based methods:

• The Markov condition is implied by any functional model, and thus is a baseline assumption for both approaches.
• Faithfulness is much stronger than required by IFMOCs; IFMOCs instead demand only a restricted “non-cancellation” within bivariate submodels. Thus, IFMOC-based inference is valid under strictly weaker and more testable assumptions (Peters et al., 2012).
• Testability: IFMOC assumptions can be assessed by regression and independence of residuals, whereas faithfulness is untestable from finite data and crucially required for correctness of PC-like algorithms.
• IFMOCs guarantee full identifiability of the causal structure or latent mixing, not merely equivalence classes, under their stated functional/independence conditions.
• Graceful failure: When IFMOC conditions fail, IFMOC-based methods produce an explicit "I do not know" response rather than erroneous structural claims.

7. Implications and Future Directions

IFMOCs provide a unifying and testable framework for overcoming generic non-identifiability in nonlinear causal discovery and ICA. The key insight is that suitable constraints—dense function classes in the regression/MDL paradigm, or rigid Jacobian structure in ICA—are both sufficient and necessary to preclude spurious equivalences and ensure model identifiability. IFMOC-based algorithmic pipelines, including LCUBE and functional model-based causal graph recovery, are efficient, interpretable, and have demonstrated high empirical efficacy. These developments inform future research on principled regularization of model classes, theoretical foundations of identifiability, and scalable algorithms for structure discovery across diverse domains (Hlavackova-Schindler et al., 30 Aug 2025, Buchholz et al., 2022, Peters et al., 2012).