Stereo Consistency Metrics

• Stereo consistency metrics are quantitative measures that assess the geometric, perceptual, or structural agreement between views in stereo imaging, crucial for tasks like matching and reconstruction.
• Recent advances incorporate edge, gradient, and feature-based approaches to overcome the limitations of traditional pixel-wise metrics, improving robustness under real-world degradations.
• Novel methods such as SIoU and MEt3R achieve high correlation with human perception, enhancing the evaluation of stereo effect in both generative and multi-view applications.

Stereo consistency metrics are quantitative measures designed to assess the geometric, perceptual, or structural agreement between two or more views in a stereo imaging setup. These metrics are fundamental for evaluating stereo matching, depth/disparity estimation, multi-view reconstruction, and generative tasks such as monocular-to-stereo synthesis or text-to-stereo diffusion. Developments in the field have revealed standard pixel-based metrics to be poorly aligned with stereo perception or robustness under real-world degradations, spurring the proposal of novel, task-adaptive metrics grounded in edge, feature, geometric, or semantic consistency.

1. Foundational Classes of Stereo Consistency Metrics

Early stereo consistency metrics focused on pixel-wise photometric agreement such as RMSE, PSNR, or SSIM, which operate under the assumption that correct correspondences exhibit intensity constancy after compensating for disparity. These metrics, however, are uniformly local and are invariant to small global pixel shifts, making them insensitive to disparities precisely at object boundaries—where human stereopsis is most acute (Yu et al., 28 Mar 2025).

To overcome this, gradient and edge-based metrics were introduced, such as the scaled gradient-feature (SGF) cost, which regularizes over local orientation and gradient magnitude to achieve robustness against illumination changes and high selectivity at true disparities (&&&1&&&). Feature-space metrics, such as those based on learned deep feature maps, have also been proposed to overcome limitations arising from photometric or resolution asymmetries (Chen et al., 2022), leveraging representations that are both agnostic to unknown degradations and optimized for stereo matching discrimination.

In multi-view or non-rectified stereo settings, geometric approaches predominate, measuring multi-view ray or tangent-space consistency, as in dynamic multi-view filtering (Yan et al., 2020) or tangent-space consistency from dense normal/azimuth fields (Cao et al., 2023). More recently, perceptual and semantic metrics—such as MEt3R—exploit pretrained 3D geometry or semantic networks to measure global correspondence in a depth-free, proxy-aided manner (Behrens et al., 11 Dec 2025).

2. Metrics for Classical and Learning-Based Stereo

2.1 Edge and Parallax-Driven Consistency: SIoU

The Stereo Intersection-over-Union (SIoU) metric, introduced in "Mono2Stereo" (Yu et al., 28 Mar 2025), combines binary edge overlap and parallax-induced difference detection:

$$\mathrm{SIoU}(L,R,G) = \alpha\,\mathrm{IoU}(E_g, E_r) + (1-\alpha)\,\mathrm{IoU}(B_{g\ell}, B_{r\ell})$$

where $E_g$, $E_r$ are edge masks (via Canny detector) on the generated and ground-truth right images, and $B_{g\ell}$, $B_{r\ell}$ are binarized absolute difference masks between each right view (generated or ground-truth) and the left reference after thresholding. $\alpha$ controls the weight (empirically $\alpha=0.75$ is optimal). SIoU is highly correlated with human judgments of "stereo effect" (Spearman $\rho=0.84$ across 1100 pairs). It is lightweight, training-free, and directly relates to object-boundary parallax—the psychophysical driver of depth perception in stereo (Yu et al., 28 Mar 2025).

2.2 Dynamic Geometric Consistency

In "D$^2$HC-RMVSNet," dynamic consistency checking is formulated over continuous confidence scores computed from pixel and depth reprojection error:

$c_{ij}(\mathbf{p}) = \exp(-\xi_p - \lambda \xi_d)$

where $\xi_p$ is reprojection error and $\xi_d$ is relative depth error. Pixel $\mathbf{p}$ is accepted if the sum over all neighbors $$C_\text{geo}(\mathbf{p}) = \sum_{j\neq i} c_{ij}(\mathbf{p}) \geq \tau$$ for a single global threshold $\tau$ (Yan et al., 2020). This scheme replaces brittle hard thresholds or view-count criteria, improving mean F-score in point cloud benchmarks (+1.65 over baseline).

2.3 Gradient-Based Dissimilarity Metrics

SGF (scaled gradient-feature cost) defines a loss based on normalized, regularized gradient orientation and magnitude:

$$e_{sgf}(u_l, u_r) = 1 - \frac{\nabla_\theta I_r(u_r) \cdot \nabla_\epsilon I_l(u_l)}{\max(\|\nabla_\epsilon I_l(u_l)\|^2, \|\nabla_\theta I_r(u_r)\|^2, \tau)}$$

SGF provides illumination invariance, unique minima at correct disparities, and superior robustness relative to photometric and orientation-only costs, reducing average disparity error across multiple stereo pipelines (Quenzel et al., 2020).

2.4 Feature-Metric Consistency

In the context of resolution-asymmetric stereo or unknown degradations, feature-metric consistency evaluates the $\ell_1$ and SSIM difference not in image space but in a learned feature space $F = \Phi_F(I;\theta_F)$ derived from the stereo matching network itself:

$$L_{fm} = \|F_L - F_{r\uparrow \rightarrow L}\|_1 + \alpha (1-\mathrm{SSIM}(F_L, F_{r\uparrow \rightarrow L}))$$

where $F_L$ is the left-view feature, $F_{r\uparrow \rightarrow L}$ is the (right, warped) feature. This loss remains robust under unknown blur, compression, or mismatch, and a "self-boosting" iterative schedule further optimizes the feature extractor to maximize stereo consistency (Chen et al., 2022).

3. Advanced Semantic and Proxy-Based Metrics

3.1 Multi-View Feature Consistency: MEt3R

MEt3R (“Measuring multi-view consistency in generated images”), as implemented in "StereoSpace" (Behrens et al., 11 Dec 2025), uses a pretrained geometric matcher (MASt3R) plus semantic features (DINO+FeatUp) to compare generated stereo pairs in a canonical 3D space:

$$\mathrm{MEt3R}(I_1, I_2) = 1 - \frac{1}{2}\bigl( \overline{S(I_1, I_2)} + \overline{S(I_2, I_1)} \bigr)$$

with $S(I_1, I_2)$ the spatial average of the cosine similarity between feature re-projections. MEt3R is depth-free, requires no ground-truth geometry at test time, and penalizes both geometric and semantic inconsistencies. On standard benchmarks, it correlates more strongly with subjective stereo plausibility than photometric metrics and is robust on multi-layer, transparent, or non-Lambertian scenes (Behrens et al., 11 Dec 2025).

3.2 Stereo Consistency Rewards for Generative Models

"Text2Stereo" introduces proxy-based rewards for guiding stereo diffusion models:

• $R_s$: Pearson correlation between monocular ($d^m$) and stereo ($d^s$) disparity maps, promoting geometric plausibility.
• $R_c$: penalization of negative disparity values to enforce parallel-axis stereo convergence.
• $R_p$: prompt-image alignment via a CLIP-based text-image score.

Reward-based fine-tuning, leveraging these metrics in concert, markedly increases geometric consistency in generated pairs as measured by $R_s$ (see: $0.414\to0.949$) and results in improved multi-view 3D reconstructions (Garg et al., 27 May 2025).

4. Task-Specific and Application-Oriented Consistency Measures

4.1 Event-Based Stereo Consistency

For event-camera stereo, metrics such as census-based left–right loss and classical disparity MAE/RMSE are dominant:

$$L_\text{census} = \sqrt{\,\|f_\text{census}(I_\text{left}) - f_\text{census}(\hat{I}_\text{left})\|^2 + \epsilon^2 }$$

Here, $f_\text{census}$ denotes the census transform. MAE, RMSE, and outlier rates (1PE/2PE) quantify overall disparity accuracy. Empirical evidence shows census loss tightens left–right event-frame structure and reduces prediction error (Jiang et al., 2024).

4.2 Azimuth and Tangent-Space Consistency

Tangent-space consistency (TSC) applies in settings where only azimuth maps (not full reflectance or depth) are available, on challenging surfaces:

$T(x)\,n(x) = 0$

with $T(x)$ composed of tangent vectors from all views and $n(x)$ the surface normal. TSC is critical for neural-SDF fitting using only azimuth cues, facilitating accurate geometry on textureless/specular surfaces unreachable by classical photo-consistency (Cao et al., 2023).

5. Comparative Analysis and Benchmarking

Empirical comparisons on standard benchmarks consistently show that metrics focusing explicitly on edge overlap, disparity agreement, or semantically meaningful correspondence show higher correlation with human-perceived stereo quality than global pixel-wise or solely photometric metrics.

Notably, SIoU demonstrates Spearman $\rho=0.84$ and Kendall $\tau=0.73$ to subjective stereo effect scores—markedly superior to RMSE, SSIM, or PSNR ($\rho$ and $\tau$ in [0.19, 0.26]) (Yu et al., 28 Mar 2025). MEt3R is able to distinguish between geometric plausibility even in scenes with reflection, translucency, or multiple disparities, in contrast to failures of warping-based or depth-reliant errors that require precise ground-truth (Behrens et al., 11 Dec 2025). Metrics integrating gradient or feature-level matching (SGF, feature-metric) outperform pixel-space metrics in adverse conditions: on Middlebury, a reduction in mean error from 7.20px (SAD) to 3.29px (SGF) is typical (Quenzel et al., 2020).

Metric	Human Judgment Correlation	GT Depth Needed	Robustness Class
SIoU	High ($\rho=0.84$)	Yes	Edges, Disparity
MEt3R	High (shown qualitatively)	No	Semantic Consistency
SGF	Not measured (perceptual), but lowest mean error	No	Gradient/Illumination
Feature-metric	N/A (measures error directly)	No	Degradations/Aliasing
Photometric RMSE/PSNR/SSIM	Low	No	Only color biases

Combined use of these metrics—e.g., reporting SIoU alongside SSIM or MEt3R with iSQoE—enables nuanced diagnosis of model failures, distinguishing between perceptual realism and true binocular stereo plausibility.

6. Limitations, Practical Recommendations, and Open Issues

Most highly reliable measures (SIoU, census/LR-loss, geometric checks) require access to ground-truth stereo pairs or a trusted external matcher. Depth-free and semantic-driven metrics (e.g., MEt3R) are dependent on the consistency and reliability of large pretrained models; in adverse conditions or with severe artifacts, measured consistency may degrade or become less interpretable.

Thresholds and weighting factors (e.g., SIoU's $\alpha$, $\tau$) should be calibrated on validation sets with human judgments when possible, although some generalize well across datasets ($\alpha=0.75, \tau=5$). In generative and diffusion contexts, proxy-based rewards must be validated for faithfulness to semantic and geometric plausibility, and batch-level aggregation does not localize geometric failures.

Emerging domains—such as temporal stereo (video), event-based, or layered/multiplanar scenes—require further adaptation or extension of existing metrics to fully capture non-rigid, multi-layer, or time-varying stereo consistency (Behrens et al., 11 Dec 2025, Jiang et al., 2024). Integration of stereo consistency with perceptual comfort (e.g., iSQoE) is an ongoing area, especially for VR applications.

7. Conclusion

Stereo consistency metrics have evolved from simple pixelwise or local errors to sophisticated measures encompassing edge, feature, geometric, and semantic agreement. The field now recognizes the necessity of correlating with human perception, enabling robustness to real-world degradations, and supporting emergent multimodal and generative tasks. Contemporary research demonstrates that metrics such as SIoU, MEt3R, and feature-metric errors address core failings of traditional approaches and establish more thorough, interpretable, and perceptually valid benchmarks for evaluating stereo correspondence, depth, and synthesis (Yu et al., 28 Mar 2025, Behrens et al., 11 Dec 2025, Chen et al., 2022, Quenzel et al., 2020).