Perceptual Counterfactual Geodesics (PCG)

• PCG is a method that defines counterfactual explanations as geodesics in latent spaces using Riemannian metrics, ensuring examples remain on-manifold and semantically coherent.
• It leverages both data-manifold and robust perceptual metrics to optimize smooth, minimal-change trajectories while suppressing adversarial artifacts.
• Empirical evaluations show PCG outperforms Euclidean baselines in realism, perceptual closeness, and validity across both vision and tabular data tasks.

Perceptual Counterfactual Geodesics (PCG) are a class of methods for generating counterfactual explanations by tracing minimal-length paths—geodesics—in the latent space of generative models. These geodesics are defined not by the standard Euclidean metric, but by a Riemannian metric that encodes either data-manifold geometry, classifier sensitivity, or, in the case of images, perceptually aligned robust features. This construction is designed to produce counterfactual examples that remain on the data manifold, are semantically meaningful, and exhibit smooth, realistic transitions between classes or attribute configurations (Pegios et al., 2024, Zaher et al., 26 Jan 2026).

1. Geometric Framework and Motivation

Perceptual Counterfactual Geodesics address limitations of conventional latent-space counterfactual methods, which often employ a flat (Euclidean) latent metric. Standard approaches can result in off-manifold, semantically incoherent, or even adversarial counterfactuals, due to the nonlinear nature of generative decoders and the mismatch between pixel/latent space distances and semantic perceptual similarity. PCG defines counterfactuals as geodesic curves in latent space $Z \subset \mathbb{R}^d$, with geodesic distance determined by a pullback Riemannian metric that reflects either the local geometry of the data manifold, decision boundary, or robust perceptual features (Pegios et al., 2024, Zaher et al., 26 Jan 2026).

Let $g: Z \to X \subset \mathbb{R}^D$ be a generator/decoder with image $\mathcal{M} = g(Z)$ approximating the data manifold. For image models, a robust classifier $f: X \to Y$ and a set of robust feature extractors $\{h_k\}_{k=1}^K$ support the definition of a Riemannian metric on either $X$ or $Z$.

2. Riemannian Metric Construction

Two principal metric constructions have been studied in PCG literature:

a. Data-Manifold and Classifier Boundary-Pullback Metric

For tabular or generic generative models, (Pegios et al., 2024) introduces a metric:

$$G(z) = J_g(z)^\top J_g(z) + \lambda\, \nabla_z[f(g(z))]\, \nabla_z[f(g(z))]^\top$$

where $J_g(z)$ is the Jacobian of the generator, and $\nabla_z[f(g(z))]$ is the latent-gradient of the classifier score. The first term penalizes directions pulling off the data manifold; the second term penalizes latent directions that cause sharp classifier changes, thus smoothing transitions near decision boundaries.

b. Robust Perceptual Metric

In the vision domain, (Zaher et al., 26 Jan 2026) uses robust, perceptually aligned features:

$$G_R(x) = \sum_{k=1}^K W_k\, J_{h_k}(x)^\top J_{h_k}(x)$$

where $h_k$ are features from selected layers of a robust (adversarially trained) backbone, and $W_k$ scale layer contributions. Pulling $G_R$ back through $g$ yields:

$M(z) = J_g(z)^\top G_R(g(z)) J_g(z)$

This metric heavily penalizes directions that alter the image in ways robust features regard as imperceptible or adversarial, while promoting smooth semantic transformations (Zaher et al., 26 Jan 2026).

3. Geodesic Formulation and Optimization

Let $\gamma : [0,1] \to Z$ be a path in latent space from the original seed $z_0$ to the counterfactual $z_T$. Under metric $M(z)$ (or $G(z)$), the length of $\gamma$ is:

$$L[\gamma] = \int_0^1 \sqrt{ \dot\gamma(t)^\top M(\gamma(t)) \dot\gamma(t) }\, dt$$

The geodesic equation arises as the Euler–Lagrange equations for stationary curves of the associated energy functional:

$$E[\gamma] = \frac{1}{2} \int_0^1 \dot\gamma(t)^\top M(\gamma(t)) \dot\gamma(t)\, dt$$

In practice, these ODEs are not solved directly, but instead the path is discretized into $T+1$ waypoints $\{z_i\}$, and the discrete energy/length is minimized with respect to the interior waypoints.

Algorithmically, optimization proceeds by initializing $\{z_i\}$ via linear interpolation and updating intermediate points using gradient descent or Adam, recalculating the metric at each segment midpoint. In robust perceptual PCG, optimization is typically staged: an initial phase minimizes the perceptual geodesic energy between input and a target-class exemplar; a second phase refines the path, pulling the endpoint to satisfy the classifier constraint, with robust re-anchoring procedures to avoid drift or collapse. The endpoint $z_T$ is dynamically re-anchored during optimization to ensure both proximity to the seed and class-flipping validity (Zaher et al., 26 Jan 2026, Pegios et al., 2024).

4. Empirical Evaluation and Metrics

PCG methods have been quantitatively benchmarked against Euclidean and flat-feature baselines across domains.

For tabular data (Pegios et al., 2024):

• Realism/fidelity to data manifold: local Euclidean distance $\mathcal{L}_D$ to the nearest training sample, with PCG achieving $\mathcal{L}_D \approx 0.04$ versus $0.20$ for Euclidean SGD.
• Closeness to origin: $\ell_0$, $\ell_1$, $\ell_2$, and $\ell_\infty$ norms in $X$.
• Validity: flip ratio indicating class change success ($\sim 0.95-0.99$).
• Immutable-feature violations: PCG $<5\%$, Euclidean SGD $>10\%$.

For vision tasks (Zaher et al., 26 Jan 2026) (AFHQ, FFHQ, PlantVillage):

• Robust feature distances: PCG achieves lowest $L_R$ (e.g., AFHQ: $0.31\pm0.02$ versus $1.75\pm0.06$ for next-best).
• Distributional realism: FID and R-FID, with PCG attaining $8.3$ (FID) and $9.1$ (R-FID) versus $12.7$–$23.5$ and $28.3$–$50.1$ for baselines.
• Perceptual closeness: PCG has lowest LPIPS ($0.24$) and R-LPIPS ($0.17$) compared to $0.59$–$0.93$ and $0.53$–$0.79$ for other methods.
• Sparsity and faithfulness: COUT of $0.43$, mean semantic margin (SM) $0.74$ (positive, indicating on-manifold validity) versus $<0.05$ for baselines.
• Manifold alignment (MAS): PCG $\sim 0.87$, higher than others.
• Trajectory smoothness: Average LPIPS-step of $\sim 0.07$ (vs. $\sim0.48$–$0.51$ for linear/spherical interpolation).

Qualitatively, PCG counterfactuals minimize semantic changes necessary to achieve the target, preserve identity and structure, and suppress both adversarial artifacts and semantic drift.

5. Methodological Distinctions and Implementation

Distinct from vanilla latent optimization, PCG strictly incorporates geometric and perceptual constraints via explicit Riemannian metrics:

• Metric selection: Explicit use of generator Jacobian and (for robust PCG) robust feature Jacobians, rather than non-robust classifier or Euclidean metrics.
• Path discretization: Optimization of all path waypoints, as opposed to simple linear interpolation.
• Constraint enforcement: Use of multi-phase alternating loss terms, plus re-anchoring of endpoints to enforce trade-off between class validity and minimal semantically meaningful change.
• Gradient calculations: All updates require computation of metric gradients at segment midpoints, and, for robust PCG, feature differences on robust backbones.

This architecture avoids common pitfalls such as off-manifold adversarial collapse, semantic drift, or failure to generate actionable explanations, by ensuring geometric and semantic alignment across the counterfactual path.

6. Limitations, Comparative Analysis, and Interpretations

PCG research highlights that the choice of Riemannian metric is critical: flat or feature-agnostic metrics result in adversarial, misaligned, or unrealistic counterfactuals, and naive trajectories can mislead end-users by presenting unactionable or implausible perturbations. The use of robust perceptual metrics appears to align generated counterfactuals with human intuition, as well as produce paths that are smooth and semantically valid throughout (Zaher et al., 26 Jan 2026).

A plausible implication is that PCG constitutes a general geometric framework adaptable to any domain where the semantics of valid counterfactual changes can be embedded in an appropriate differentiable metric. However, reliance on differentiable robust feature extractors, and the computational cost of high-dimensional Jacobian evaluations and tangent-space projections, remain practical constraints.

Comparison tables from (Zaher et al., 26 Jan 2026, Pegios et al., 2024) show PCG methods outperforming Euclidean and feature-space baselines across manifold realism, perceptual closeness, and sparse faithfulness, both quantitatively and qualitatively.

7. Impact and Applications

Perceptual Counterfactual Geodesics provide a principled, geometry-aware methodology for actionable, human-meaningful counterfactual explanation in deep models. The on-manifold, smooth transformations enable reliable inspection and model debugging, highlight classifier decision structures, and reveal hidden failure modes that remain undetected under non-perceptual metrics. Demonstrated applications include tabular counterfactual explanation for recidivism/credit models and attribute- or class-level transformations in high-resolution vision models.

By leveraging Riemannian geometry and robust perceptual alignment, PCG sets a benchmark for generating counterfactuals that respect the underlying data distribution and perceptual semantics, and has broad implications for the interpretability and trustworthiness of complex generative and discriminative architectures (Pegios et al., 2024, Zaher et al., 26 Jan 2026).