Hammersley-Clifford Theorem

• Hammersley-Clifford theorem is a foundational result that equates conditional independence in strictly positive distributions with clique-wise Gibbs factorization on undirected graphs.
• It underpins Markov random fields by showing that local, pairwise, and global Markov properties are equivalent under the theorem's framework.
• Generalizations extend its scope to toric models, context-specific independences, and structured zeros, aiding advanced model selection and structure discovery.

The Hammersley–Clifford theorem occupies a central role in the theory of discrete graphical models, establishing a fundamental equivalence between certain conditional independence properties and the factorization of strictly positive probability distributions on undirected graphs. This theorem provides the technical foundation for Markov random fields (MRFs), facilitating their representation as Gibbs distributions. Modern developments have extended its framework to encompass algebraic, combinatorial, and context-specific independence structures, as well as to situations where the strict positivity condition is violated. Below, a comprehensive exposition is provided, synthesizing classical results, generalizations, and contemporary research directions.

1. Classical Hammersley–Clifford Theorem

Let $G=(V,E)$ be an undirected graph where $V=\{1,2,\dots,n\}$ denotes variables $X=(X_1, \dots, X_n)$ with joint state space $\mathcal X = \prod_{i \in V} \mathcal X_i$. A probability distribution $P$ on $\mathcal X$ is called strictly positive if $P(x) > 0$ for all $x \in \mathcal X$.

Key Markov Properties

• Pairwise Markov: For each non-edge $\{i,j\} \notin E$, $X_i \perp X_j \mid X_{V \setminus \{i,j\}}$.
• Global Markov (Separation): For disjoint $A,B,C \subset V$, if every path from $A$ to $B$ passes through $C$, then $X_A \perp X_B \mid X_C$.
• Local Markov: For every $i$, $$X_i \perp X_{V \setminus (\{i\} \cup N(i))} \mid X_{N(i)}$$, with $N(i)$ the neighbors of $i$.

HC‐Factorization (Gibbs Representation)

Given the set $\mathrm{Cliques}(G)$ of maximal cliques,

$$P(x) = \frac{1}{Z} \prod_{C \in \mathrm{Cliques}(G)} \psi_C(x_C), \qquad Z = \sum_{x \in \mathcal X} \prod_{C \in \mathrm{Cliques}(G)} \psi_C(x_C)$$

where each $\psi_C: \mathcal{X}_C \to (0,\infty)$ is a potential depending only on variables in $C$.

Main Statement

Theorem (Hammersley–Clifford): For strictly positive $P$, the following are equivalent:

• (a) $P$ satisfies the pairwise Markov property for $G$.
• (b) $P$ satisfies the global (separation) Markov property for $G$.
• (c) $P$ admits the HC-factorization over the cliques of $G$.

Hence, for positive probability distributions, local, pairwise, and global Markov properties coincide and are equivalent to clique-wise factorization (Geiger et al., 2012, Edera et al., 2013, Kovács et al., 2013).

2. Algebraic and Toric Frameworks for Factorization

Geiger, Meek, and Sturmfels introduced a comprehensive algebraic perspective, encoding discrete exponential families—including log-linear and undirected graphical models—as toric statistical models.

Toric Models and Ideals

• Model Matrix: Let $A$ be a nonnegative integer $d \times m$ matrix (e.g., encoding clique memberships). The monomial parametrization is

$$\phi_A: \mathbb{R}_{>0}^d \to \mathbb{R}_{>0}^m, \quad p_j = \prod_{i=1}^d t_i^{a_{ij}}$$

• Toric Ideal: Defined as the kernel of the ring homomorphism induced by $A$ at the polynomial level, encapsulating all binomial constraints needed for factorization.

Generalized Factorization Theorem

A probability vector $P \in \mathbb{R}_{\ge0}^m$ factors according to $A$ if and only if:

• (a) All binomials in the toric ideal $I_A$ vanish at $P$.
• (b) The support of $P$ is nice: that is, for every coordinate outside the support, its column support in $A$ is not contained in the union of column supports indexed by the support.

If these hold, then $P$ is in the topological closure of the image of $\phi_A$, hence a limit of factorable distributions (Geiger et al., 2012).

3. Generalizations Beyond Strict Positivity

Classical HC requires $P>0$ everywhere. Several directions relax positivity:

a. Lattice Supported Distributions

Natural Distributive Lattices: For binary variables, if the support of $P$ is a distributive sublattice $L \subset 2^{[m]}$ (closed under union and intersection, containing ∅ and [m]), and $P$ satisfies all pairwise Markov constraints, then $P$ admits the same structural factorization as in the classical case.

This establishes that HC-equivalence may hold far beyond the strictly positive regime, provided that zeros in $P$ are structured (e.g., forming a lattice). The resulting parametrizations coincide with those induced by Hibi ideals, yielding an intersection between graphical model theory and combinatorial commutative algebra (Kahle et al., 2024).

b. Structural Zeros and Mouissouris's Example

There exist distributions with structured zeros (not full support) that retain the equivalence of the Markov properties and allow a modified factorization (e.g., as a ratio of higher-order marginals) (Kovács et al., 2013).

4. Extensions: Context-Specific Independences and Configuration Folding

a. Context-Specific Hammersley–Clifford Theorem

Many independence properties relevant in practice are context-specific: a conditional independence holds only for some assignments of the conditioning set.

• CSI (Context-Specific Independence): $I(X_a, X_b \mid X_U, x_W)$ asserts conditional independence between $X_a$ and $X_b$ given $X_U$, but only holds when $X_W = x_W$.

A generalized HC theorem states: if, for every context $x_W$, the reduced model $I_{x_W}$ is graph-isomorph and is an I-map for $P(X \setminus X_W \mid x_W)$, then the joint can be factorized as a product over context-restricted conditional distributions, each itself being a clique factorization (Edera et al., 2013). This enables finer-grained factorizations and captures parameter sparsity and local structure unachievable with global graphical models.

b. Bipartite and Dismantlable Graphs—Strong Configuration Folding

On bipartite graphs, the notion of a "safe symbol" (enabling the classical proof) can be replaced by a folding procedure—strong config-folding—on the alphabet. The theorem asserts that if all MRFs on $X$ are Gibbs, the same holds after folding or unfolding configurations, thereby broadening HC applicability to shifts without global safe symbols (e.g., dismantlable graphs, homomorphism shifts to $C_4$) (Chandgotia, 2014, Chandgotia et al., 2013).

5. Proof Sketches, Algorithmic Perspectives, and Examples

a. Proof Overview

• Classical Case: Relies on conditional independence properties, Möbius inversion on clique indicator functions, and the safe symbol property to glue local potentials.
• Algebraic Case: Necessity/sufficiency of binomial constraints from toric ideals, support niceness, and toric geometry (normality, closure under monomial maps).
• Context-Specific and Folding Approaches: Reduction to conventional HC on each context or folded space, then reconstruction of global factorization.

b. Algorithmic Recovery and Structure Discovery

The exact Markov graph structure can be algorithmically recovered from a fully specified $P$ via (supermodular) information content and the computation of conditional independences. This approach applies even when some entries of $P$ are exactly zero, illustrating that the classical requirement for positivity is sufficient but not always necessary (Kovács et al., 2013).

c. Illustrative Examples

• No Three-Way Interaction Model: Factorization over pairs without triple interactions is encoded via toric ideals with binomial generators, e.g., quadratics of the form $p_{000}p_{111} - p_{001}p_{110}$ (Geiger et al., 2012).
• Cycle Graphs and Structured Zeros: For the 4-cycle and lattice-supported examples, one can explicitly write down the parametrization and verify the vanishing of binomial constraints, with applications in design of experiments and contingency-table analysis (Kahle et al., 2024, Kovács et al., 2013).

6. Implications and Open Directions

• The algebraic and combinatorial frameworks enable application of Gröbner basis methods, dimension counts, and computational algebra for model selection and understanding.
• HC generalizations accommodate sparsity-inducing distributions, context-local modeling, and structure learning with incomplete support.
• Open problems include the characterization of minimal generators for graphical model ideals in general, the identification of all situations where positivity can be relaxed, and further developments in configuration folding and cocycle theory for non-bipartite or non-discrete settings (Geiger et al., 2012, Chandgotia, 2014, Chandgotia et al., 2013).

---

Selected References:

Main Result	arXiv id	Key Contribution
Classical and algebraic HC theorem	(Geiger et al., 2012)	Generalization via toric models, support niceness
Context-specific independence extension	(Edera et al., 2013)	CSI-aware factorization theorem
Lattice-supported (structural zeros) generalization	(Kahle et al., 2024)	HC on natural distributive lattice supports
Pairwise Markov structure recovery, supermodularity	(Kovács et al., 2013)	Algorithmic graph discovery under relaxed positivity
Bipartite/folding extension, safe-symbol removals	(Chandgotia, 2014)	Strong config-folding and generalization to folds/unfolds
Markov cocycle formalism, multi-dimensional limits	(Chandgotia et al., 2013)	Non-Gibbsian examples, cocycle parametrization

Each reference provides further details on specific structural, algebraic, and algorithmic aspects of Hammersley–Clifford theory and its far-reaching generalizations.