Depth-aware Dynamic Cost Aggregation (DDCA)

• The paper introduces DDCA, a dynamic cost aggregation strategy that fuses depth-aware priors with per-pixel adaptive filtering to improve stereo matching accuracy.
• It details a pipeline where depth cues are integrated via group-convolution and self-attention, leading to sharper cost volumes and reduced matching errors.
• Empirical evaluations demonstrate that DDCA enhances generalization and robustness, reducing errors by up to 26% on benchmark datasets in both stereo and multi-view contexts.

Depth-aware Dynamic Cost Aggregation (DDCA) is a class of aggregation techniques in stereo and multi-view geometry that utilizes depth priors or depth-aware features to adaptively modulate the cost aggregation process, thereby enhancing matching reliability across challenging image regions and promoting generalization to novel scenes. The central principle of DDCA is the incorporation of global or monocular structural cues—typically encoded as domain-invariant depth features—into the cost aggregation pipeline, leading to per-pixel, per-hypothesis, and, in some architectures, per-group dynamic filter selection. Representative realizations span explicit affinity-based filtering, depth-aware self-attention, and static/dynamic cost volume fusion.

1. Pipeline Integration and Architectural Placement

In modern real-time stereo pipelines, DDCA modules are positioned immediately after initial feature correlation or cost volume construction, before disparity regression. For instance, in the Generalized Geometry Encoding Volume (GGEV) framework, a group-correlation cost volume $C \in \mathbb{R}^{G \times D/4 \times H \times W}$ is first obtained. Depth-aware prior features $f_{da}$ are concurrently extracted using selective channel fusion (SCF) that merges texture-focused (MobileNetV2) and domain-invariant depth cues (frozen Depth Anything V2). The DDCA module operates on each disparity slice $C_d$, using $f_{da}$ as explicit guidance for cost volume sharpening (Liu et al., 7 Dec 2025). In multi-view stereo extensions, similar strategies are realized via windowed Transformer attention or through cost volume fusion that leverages dynamic (optical flow-informed) and static (camera motion-informed) evidence (Chen et al., 2023, Miao et al., 2023).

2. Mathematical Formulation and Mechanisms

The mathematical core of DDCA in (Liu et al., 7 Dec 2025) is affinity-driven, per-pixel, per-group dynamic convolution. Explicitly:

• Query and key embeddings are constructed:

$$Q = \mathrm{Re}(W_q(C_d)) \in \mathbb{R}^{C \times (HW)}, \quad K = \mathrm{Re}(W_k(\mathrm{Pool}(f_{da}))) \in \mathbb{R}^{C \times (S^2)}$$

with $C$ channels, $S$ spatial pooling regions, and 1x1 convolutions $W_q, W_k$.

• Both Q and K are partitioned into $G$ channel groups. For each group $g \in \{1, \dots, G\}$:

$$A^g = (Q^g)^\top K^g \in \mathbb{R}^{HW \times S^2}$$

• Each pixel $i$ in group $g$ maps its affinity vector $A^g[i,:]$ through a learned $W_m$ and softmax, yielding a $K \times K$ dynamic convolutional kernel $M_i^g$:

$$m_i^g = \mathrm{softmax}(A^g[i,:] \cdot W_m), \quad M_i^g = \mathrm{reshape}(m_i^g)$$

• A sliding window convolution is performed over the concatenated feature map $[C_d; f_{da}]$ with $M_i^g$, reassembling the aggregated cost slice $C'_d$.
• The completed aggregated cost volume $C'$ is then fed to a soft-argmin layer for initial disparity estimation:

$$d_0(x, y) = \sum_{d=0}^{D/4-1} d \cdot \mathrm{softmax}(C'(d, x, y))$$

Related instantiations in CostFormer (Chen et al., 2023) use multi-head, local 3D self-attention blocks (DATL/DASTL) with depth-aware position encodings, while DS-Depth (Miao et al., 2023) fuses static and dynamic cost volumes via branching and convolutional fusion modules.

3. Algorithmic Pipeline and Implementation Details

The DDCA core algorithm for stereo cost aggregation can be summarized as:

1. Extract raw group-correlation cost slices $C_d$ and corresponding depth guidance $f_{da}$.
2. Project $C_d$ and pooled $f_{da}$ through 1x1 convolutions to obtain queries and keys, group them.
3. For each group, compute an affinity matrix and generate dynamic kernels via a learned linear transformation and softmax normalization.
4. Apply per-pixel, per-group sliding window convolution over channel-grouped $[C_d; f_{da}]$ using these dynamic kernels.
5. Optionally, aggregate results from multiple kernel sizes (e.g., $3\times3$, $5\times5$).
6. Concatenate or sum to form the final aggregated cost volume, which is then processed by standard disparity regression layers (soft-argmin, GRU refinement).

Implementation specifics include:

• $G=8$ groups for cost/feature partitioning.
• Adaptive average pooling to $S=8$ region centers for global structure encoding.
• Lightweight parameterization: $+$0.03M additional parameters and $\approx$9 ms runtime overhead at KITTI resolution (1248×384) (Liu et al., 7 Dec 2025).
• Efficient group convolution realizations and support for hybrid kernel sizes.
• Total pipeline maintains real-time inference (e.g., 47 ms/frame in GGEV).

4. Comparison with Conventional Aggregation Approaches

Traditional hourglass or 3D convolutional cost aggregation applies fixed spatial filters across all disparities, relying primarily on local cost smoothness. Such static filters lack explicit mechanism for adapting to cues from domain-invariant structure, making them vulnerable to local minima in the presence of occlusions, textureless regions, or repeating patterns.

DDCA modules, by contrast:

• Incorporate global or monocular structure via depth-aware priors, enabling cost propagation that is sensitive to semantics and object boundaries.
• Employ dynamic, per-pixel filters shaped by affinity to the scene context, allowing localized adaptation of receptive field orientation, size, and frequency response.
• Substantially suppress spurious matching and recover sharp edges and thin structures, as evidenced by qualitative cost slice comparisons (Liu et al., 7 Dec 2025, Chen et al., 2023).
• Extend to non-stereo regimes by fusing camera-derived and flow-derived cues for dynamic objects (Miao et al., 2023).

5. Empirical Evaluation and Effectiveness

Comprehensive ablation and benchmark studies corroborate the quantitative benefits of DDCA.

• In GGEV (Liu et al., 7 Dec 2025):
  • Scene Flow $\rightarrow$ KITTI/ETH3D zero-shot generalization: 3-pixel error on KITTI decreases from 5.8 % (RT-IGEV) to 4.1 % (GGEV), and ETH3D error halves from 5.8 % to 2.8 %.
  • On ETH3D, Bad1.0 % = 1.19 for GGEV vs. 2.79 % for HITNet (prior best real-time).
  • Ablation: DDCA achieves a relative improvement up to 26 % on ETH3D Bad2.0 error with only minimal computational overhead.
• In CostFormer (Chen et al., 2023):
  • RDACT block (depth-aware attention-based DDCA) yields a 10 % completeness gain on DTU compared to CNN-only aggregation.
  • Additional Residual Regression Transformer restores accuracy, with final overall DTU metric improving from 0.352 to 0.343 mm.
• In DS-Depth (Miao et al., 2023):
  • Two-branch fusion of static and dynamic cost volumes reduces AbsRel on KITTI from 0.101 to 0.096, and further to 0.095 with advanced loss functions.
  • Dynamic/static fusion is especially effective in dynamic scene regions, reducing AbsRel from 0.169 to 0.127.

6. Robustness and Generalization in Unseen Conditions

DDCA’s incorporation of global depth-aware guidance enhances robustness to domain shift, occlusion, and texture ambiguities. The module explicitly:

• Computes affinity between each disparity (or depth) hypothesis and global geometric priors.
• Generates dynamically shaped filters that align with scene geometry and object contours, preserving fine structures and minimizing erroneous matches.
• Demonstrates pronounced resilience in zero-shot and cross-domain benchmarks where local-only aggregation methods degrade.

In summary, DDCA methods, through explicit coupling of depth-aware priors and cost volume refinement, constitute a high-impact advance in robust and generalizable stereo and multi-view depth estimation. Their hybridity—balancing structural guidance, semantic priors, and dynamic convolution or attention—substantially extends the operative regime of real-time correspondence pipelines, as systematically evaluated across standard datasets and settings (Liu et al., 7 Dec 2025, Chen et al., 2023, Miao et al., 2023).