Probabilistic PDG (PPDG): Framework & Applications

• Probabilistic PDG (PPDG) is a general probabilistic graphical model that extends traditional Bayesian networks by incorporating both structural and quantitative confidence in dependencies.
• The framework enables modular construction and efficient polynomial-time inference via convex optimization and clique-tree decomposition techniques.
• PPDGs have been applied in program analysis, fault localization, and loss design for machine learning, effectively reconciling conflicting or heterogeneous information.

A Probabilistic Program Dependence Graph (PPDG), also widely referenced as a Probabilistic Dependency Graph (PDG) or Probabilistic PDG, is a general and expressive class of probabilistic graphical models designed to represent, reason about, and compute with uncertain, inconsistent, and modular knowledge. PPDGs encompass existing mainstream models such as Bayesian Networks and factor graphs, but allow fine-grained control over structural and parametric trust in each conditional dependence. The framework provides both a versatile representational syntax and a principled, tractable inference and optimization machinery, supporting inconsistent, conflicting, or heterogeneous probabilistic information. PPDGs have found applications in areas such as program analysis, model-based fault localization, structured prediction, and loss design for machine learning.

1. Formal Structure and Semantics

A PPDG is defined as a tuple

$$M = (V, E, \mathrm{Val}, \{p_L\}_{L\in E}, \{\alpha_L\}_{L\in E}, \{\beta_L\}_{L\in E})$$

where:

• V: set of nodes, each representing a random variable.
• E: set of directed (possibly hyper-)edges, each $L: X \to Y$ labeled by variable(s).
• $\mathrm{Val}(X)$: value set of variable $X$.
• $p_L(Y|X)$: local conditional probability distribution for edge $L$; in hyperarcs, both source and target may contain multiple variables.
• $\alpha_L \ge 0$: structural confidence, encoding prior belief in the relevance or causal structure enforced by edge $L$.
• $\beta_L \ge 0$: quantitative confidence or observational weight in the accuracy of $p_L$.

A PPDG thereby expresses that $Y$ depends on $X$ via the conditional $p_L$, with weights encoding the reliability or strength of this dependency.

Three key semantics are provided (Richardson et al., 2020, Richardson, 2022, Richardson et al., 2023):

• Set-of-distributions: the set of global joint distributions whose marginals are compatible with all local cpds; may be empty if the local beliefs are inconsistent.
• Scoring-function: for any candidate joint $μ$, define incompatibility by aggregating edge-wise KL divergences between $μ$'s marginals and $p_L$, weighted by $\beta_L$, plus an “information deficiency” measuring deviation from structural patterns (via entropic terms weighted by $\alpha_L$).
• Unique-distribution: for strictly positive weights and under mild convexity conditions, the loss functional has a unique minimizer $μ^*$, which is designated as the PPDG's global probabilistic prediction.

These semantics extend those of Bayesian networks—recovered as the special case where all local compatibilities are enforced with infinite confidence and edges encode the BN’s local Markov dependencies.

2. Construction and Learning

PPDG model construction proceeds by specifying nodes, the dependency structure, and associated local cpds:

• Program analysis application (Askarunisa et al., 2012): The base graph is a Program Dependence Graph (PDG) constructed from control and data dependence analysis of a program's Control Flow Graph (CFG). Each node corresponds to a statement or predicate. Edges represent syntactic dependencies.
• Statistical dependency estimation: These structural edges are augmented by annotating each node with a discrete state space (e.g., predicate outcome, variable value bins), and by empirically estimating conditional probability tables (CPTs) from execution traces. Frequency estimation from observed trace data yields conditional probabilities for each node given its parents' states.
• General machine learning context (Richardson, 2022): Edges correspond to features, labels, latent variables, and priors; confidences represent trust or prior/regularization strength. Construction is modular: new beliefs (in the form of cpds) can be incorporated as edges, and confidences updated as data or context demands.

Parameter learning in PPDGs is naturally cast as a joint optimization of the global loss functional, alternating between estimating the joint $μ$ and updating local cpds to minimize penalized KL-divergence to marginals observed in data (Richardson et al., 2020).

3. Inference Algorithms and Computational Tractability

PPDG inference consists of answering queries about marginals or conditional probabilities under the global joint $μ^*$, as determined by the model. The core mathematical problem is to minimize the convex loss functional:

$$\mathcal L_\gamma(\mu) = \sum_{L\in E}\,\beta_L\,\mathbb{E}_{x\sim\mu(X)}\Bigl[D_{\mathrm{KL}}(\mu(Y|x) \,\Vert\, p_L(Y|x))\Bigl] + \gamma\,\Bigl(\sum_{L\in E}\alpha_L H_\mu(Y|X) - H(\mu)\Bigr)$$

where $H(\mu)$ and $H_\mu(Y|X)$ denote joint and conditional entropies. For small enough $\gamma>0$ (controlling the trade-off between observational and structural penalties), this program is strictly convex.

Polynomial-time inference is achieved via convex optimization over the space of joint distributions, implemented efficiently for bounded-treewidth networks using clique-tree decompositions and exponential-cone programming. Interior-point algorithms provide solutions with time complexity polynomial in both the number of arcs and the treewidth-dependent size of clusters. For modestly-sized or sparse PPDGs, inference scales similarly to established graphical model algorithms (Richardson et al., 2023).

4. Modeling Expressiveness and Relation to BNs and Factor Graphs

PPDGs strictly generalize both Bayesian Networks (BNs) and factor graphs:

• BN correspondence: Any BN can be translated to a PPDG by encoding each parent set and its corresponding cpd as an edge with high confidence, potentially using aggregate variables for hyperarcs. The distinguished joint in the PPDG semantics coincides with the BN’s joint assignment (Richardson et al., 2020).
• Factor graph/exponential family embedding: Factor graphs with positive weights are encoded by aggregate variables and edges whose cpds and weighting correspond to the factors and exponents, with the free energy (log partition function) emerging as the PPDG's inconsistency score at optimum (Richardson, 2022).
• Beyond BNs and factor graphs: PPDGs allow arbitrary mixtures of causal (structural) and statistical (observational) edges, multiedges (conflicting evidence), and confidence tuning. Their loss-based semantics automatically accommodate inconsistencies and blend competing beliefs, which is unrepresentable in a single BN or factor graph.

The following table outlines the relation:

Model Type	Encoded as PPDG by	PPDG Capabilities Beyond
Bayesian Net	High-confidence cpds/pa-edges	Multiple/conflicting cpds per link, soft structural beliefs
Factor Graph	Aggregate nodes, weighted cpds	Arbitrary edge weights α, β; directed dependencies
General PPDG	Free mixture, conflict handling	N/A

5. Applications: Model-Based Fault Localization and Loss Design

• Fault localization: PPDG-based fault localization constructs a program's PDG, annotates it with empirical cpds from passing traces, and, during a failing execution, ranks program locations by the minimal conditional probability observed (RankCP algorithm). The lowest-probability nodes are flagged as most suspicious. Empirically, this approach outperformed spectrum-based bug isolation in number of faults correctly localized over a set of mutation-seeded Java programs, with strong scalability due to linear-time algorithms for both parameter learning and evaluation (Askarunisa et al., 2012).
• Loss design/variational inference: Any standard ML loss—including cross-entropy, negative log-likelihood, regularized objectives, and evidence lower bound (ELBO) for latent variable models—can be realized as a PPDG inconsistency score for appropriate edge structure and confidences. This yields a direct link from modeling assumptions to optimization objectives, obviating ad hoc loss choice (Richardson, 2022).

6. Modularity, Expressiveness, and Inconsistency Handling

PPDGs retain modularity under edge/nodes addition or deletion and allow direct representation of inconsistent knowledge:

• Union of knowledge sources (e.g., combining structured expert beliefs with noisy data) is achieved simply by graph union.
• Conflicting cpds (on the same edge or variable) are blended by the convex minimization, with the degree of resulting inconsistency quantified by the minimum global loss.
• Incremental updates to edge structure or weights immediately yield revised joint beliefs, facilitating explainability and revision without restructuring.
• Information-theoretic inequalities (e.g., data-processing inequality) follow immediately by monotonicity of inconsistency with respect to edge addition, providing a graphical, compositional visual language for reasoning about divergence and entropy (Richardson, 2022).

7. Practical Algorithms and Empirical Performance

Recent work presents tractable, scalable algorithms for PPDG inference and learning over discrete variables:

• Clique-tree (junction tree) convex program formulation: Exponentially large joint distributions are summarized by cluster-wise marginals, with constraints ensuring running-intersection consistencies.
• Interior-point optimization: Self-dual homogeneous barrier methods solve the resulting convex program in polynomial time, with complexity determined by treewidth and model size. Off-the-shelf solvers (e.g., MOSEK, ECOS via CVXPy) are used efficiently (Richardson et al., 2023).
• Empirical results: PPDG inference matches or outperforms belief propagation in tree-structured BN cases and significantly outpaces generic SGD solvers on moderate-size networks. Solutions smoothly interpolate between maximizing fit to local cpds and maximizing global entropy, yielding graceful behavior under inconsistency.

8. Summary and Outlook

The PPDG formalism and tractable inference framework provide a compositional, loss-centric, and modular approach to probabilistic graphical modeling. They enable fine-grained reconciliation of conflicting evidence and flexible adaptation to both causal and statistical knowledge, strictly generalizing BNs and factor graphs. Representative applications include model-based program analysis, unification of loss design in machine learning, and robust reasoning under uncertainty and partial information (Richardson et al., 2020, Richardson, 2022, Richardson et al., 2023, Askarunisa et al., 2012).

PPDGs are thus positioned as a canonical framework for principled probabilistic inference and optimization in the presence of inconsistency, modularity, and heterogeneous domain knowledge.