Progressive Distance Estimator in Deep Learning

• Progressive Distance Estimator is a self-supervised learning paradigm that gradually extends the estimation range to improve point cloud registration and depth completion.
• It employs techniques like exponential moving average, spatial filtering, and multi-scale refinement to enhance performance and generalization even on challenging long-range tasks.
• The method leverages robust feature extraction and correspondence propagation, validated by significant empirical gains on benchmarks such as KITTI and nuScenes.

A Progressive Distance Estimator is a learning paradigm integral to recent advances in both point cloud registration and dense depth estimation, characterized by gradually increasing the spatial or temporal range over which estimations or correspondences are established during training. This staged learning explicitly leverages easier, proximal cases to bootstrap reliable supervision for more distant, challenging scenarios. Notable implementations include the progressive distance extension in unsupervised point cloud registration (as in EYOC) and the progressive multi-scale refinement in inverse Laplacian pyramid-based depth completion (as in LP-Net). These approaches have demonstrated substantial gains in generalization, efficiency, and performance metrics without reliance on dense external supervision (Liu et al., 2024, Wang et al., 11 Feb 2025).

1. Progressive-Distance Extension in Point Cloud Registration

The progressive distance estimator framework for point cloud registration, exemplified by the EYOC method (“Extend Your Own Correspondences”), organizes training as a series of mini-tasks indexed by an increasing frame-interval $I$ in raw LiDAR sequences. Initially, pairs of consecutive LiDAR sweeps ($I=1$) are used, effectively registering nearly identical scenes with identity transforms. As training proceeds, the range of $I$ (i.e., $I \in [1,B]$) grows incrementally according to $B(t) = \max(1,\lfloor 30 t / T \rfloor)$ with $T$ the total number of epochs, ultimately reaching inter-sweep separations equivalent to 50 m.

Each round $k\rightarrow k+1$ involves:

• Exponential moving average (EMA) transfer of student weights into a “labeler.”
• Sampling LiDAR pairs with enlarged $I$.
• Labeler-driven production of noisy, feature-based initial matches.
• Application of spatial filtering for high-fidelity geometric registration.
• Rediscovery of dense correspondences under the new pose for student supervision.

This iterative bootstrapping approach allows a feature extractor, initially trained on trivial (short-range) cases, to generalize and self-supervise on progressively distant (harder) cases without manually annotated pose labels (Liu et al., 2024).

2. Self-Supervised Losses and Correspondence Label Mechanisms

After speculative registration and correspondence regeneration, inlier sets $C_{S\to T}$ (and symmetrically $C_{T\to S}$) are generated using tight nearest-neighbor matches and a geometrical threshold ($\beta_{\text{inlier}}=2$ m). The hardest-contrastive loss enforces attraction between inlier feature pairs $(f_S^i, f_T^j \in \mathbb{R}^K)$: $$L = \frac{1}{|C_{S\to T}|} \sum_{(i,j)\in C_{S\to T}} \left[ m + \|f_S^i-f_T^j\|^2 - \min_{k\neq j, k\in N(i)} \|f_S^i-f_T^k\|^2 \right]_+ + (S \leftrightarrow T)$$ where $m$ is a positive margin and $N(i)$ is a pool of candidate negatives.

The student-labeler update utilizes EMA: $$W_{lab} \leftarrow \lambda W_{lab} + (1-\lambda) W_{stu}$$ with $\lambda \approx 0.2$ empirically optimal, mediating stability and adaptability (Liu et al., 2024).

3. Spatial Filtering of Labeler Matches and Robust Estimation

Unfiltered, feature-space nearest-neighbor matches degrade in inlier ratio at large distances (≈20% at 30 m). To address this, spatial filters, based on the minimum Euclidean distance to the LiDAR origins ($\min(d_1, d_2)$), eliminate geometrically unstable correspondences. Two strategies are employed:

• Hard cut: discard matches with $\min(d_1, d_2) < 40$ m.
• Adaptive cut: prune bins in $(d_1, d_2)$ space with median cosine similarity below $0.6$ (thresholds selected pre-collapse of inlier ratio).

Filtered correspondences (about 200 per pair) are passed to an SC$^2$-PCR solver for pose estimation. This process prunes approximately 70% of false positives while incurring less than 10% loss in true positives (Liu et al., 2024).

4. Progressive Multi-scale Estimation in Depth Completion

The progressive estimator paradigm also underpins multi-scale depth completion via Laplacian Pyramid inversion, as introduced in LP-Net (Wang et al., 11 Feb 2025). Here, a low-frequency global estimate $\hat{D}^{(4)}$ is computed first, followed by four hierarchically finer refinement stages. At each stage $i$ ($3 \rightarrow 0$):

• Upsample and fuse coarse prediction $\hat{D}^{(i+1)}$ with downsampled sparse input $\hat{S}^{(i)}$ using a learned confidence map $c_i$.
• Invoke a Selective Depth Filtering (SDF) module to separately learn smooth (noise suppression) and sharp (edge preservation) bandpass detail, and per-pixel attention-based blending.

This multi-scale, progressive approach eschews inefficient pixel-wise propagation, dramatically improving computational efficiency and accuracy, especially for long-range or rarefied scene structures.

5. Feature Extractor Architectures and Correspondence Propagation

In both progressive point cloud registration and progressive depth completion, strong performance relies on robust feature extractor designs:

• EYOC uses a 3D sparse-convolutional U-Net backbone (MinkowskiEngine), yielding $\mathbb{R}^{N\times K}$ pointwise descriptors for mutual nearest neighbor matching.
• LP-Net deploys a multi-path feature pyramid at the deepest encoder stage, exploiting channel-wise splitting and multi-stride convolutions for context aggregation before successive detail recovery (Wang et al., 11 Feb 2025).

Both frameworks include mechanisms to propagate reliable correspondence or estimation from coarse-to-fine scales or small-to-large spatial intervals, supported by self-supervised supervisory signals derived through their respective progressive schemes.

6. Performance, Ablation Analysis, and Generalization Capacity

Empirical evaluations present the following highlights:

• EYOC achieves mRR = 83.2% on KITTI (vs. Predator 87.9% supervised, FCGF 84.6% finetuned) for 5–50 m registration, with long-range RR [40,50]m = 52.3%, RRE ≈ 1.3°, RTE ≈ 31.8 cm. On WOD and nuScenes, EYOC outperforms or matches state-of-the-art, particularly excelling in domain generalization (e.g., WOD→KITTI adaptation yields mRR improvement from ≈69.9% to 80.6% without pose labels) (Liu et al., 2024).
• LP-Net sets state-of-the-art on KITTI depth completion (RMSE=684.71 mm, MAE=186.63 mm), surpassing prior approaches in both accuracy and efficiency, with ablations confirming monotonic gain at each progressive scale (Wang et al., 11 Feb 2025).

Ablation analysis for progressive training evidences collapse (inlier ratio IR=0%) without distance-extension (i.e., if $B=1$ throughout), and failure if spatial filtering is omitted, even in the presence of correspondence rediscovery. Only full progression with optimized spatial filtering attains strong inlier ratios and maximal mRR.

7. Implications and Extensions

The progressive distance estimator paradigm elucidates a general mechanism for scaling self-supervised learning to increasingly challenging spatial or temporal regimes. Its core components—staged range extension, spatial filtering, speculative self-labeling, and robust, scale-aware feature extraction—may be applicable to broader unsupervised geometric or scene perception problems. The demonstrated gains in generalization and autonomy in both geometric registration and multi-scale regression tasks suggest wide relevance beyond the specific tasks of EYOC and LP-Net (Liu et al., 2024, Wang et al., 11 Feb 2025).