Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow

Abstract: Optical flow estimation can be formulated as an end-to-end supervised learning problem, which yields estimates with a superior accuracy-runtime tradeoff compared to alternative methodology. In this paper, we make such networks estimate their local uncertainty about the correctness of their prediction, which is vital information when building decisions on top of the estimations. For the first time we compare several strategies and techniques to estimate uncertainty in a large-scale computer vision task like optical flow estimation. Moreover, we introduce a new network architecture utilizing the Winner-Takes-All loss and show that this can provide complementary hypotheses and uncertainty estimates efficiently with a single forward pass and without the need for sampling or ensembles. Finally, we demonstrate the quality of the different uncertainty estimates, which is clearly above previous confidence measures on optical flow and allows for interactive frame rates.

• The paper introduces a novel network that jointly predicts optical flow and quantifies uncertainty using empirical, predictive, and Bayesian strategies.
• It incorporates a multi-hypotheses architecture inspired by Winner-Takes-All to generate diverse predictions when the ground-truth is ambiguous.
• Experimental results on datasets like Sintel and KITTI demonstrate superior uncertainty estimation, enabling more reliable deployment in autonomous systems.

Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow

This paper addresses the challenges associated with uncertainty estimation in deep neural networks for optical flow tasks, an area of significant interest given the potential applications in autonomous systems where decision-making relies on the reliability of predictions. The authors propose methods to quantify the uncertainty in optical flow predictions, which is critical in scenarios where incorrect estimations could lead to undesirable consequences.

Overview and Contributions

The authors introduce a network architecture designed to provide both optical flow predictions and associated uncertainty estimates. They highlight the limitations of previous approaches, such as the lack of inherent uncertainty quantification in traditional deep learning models. Three major strategies for uncertainty estimation are explored: empirical methods, predictive methods, and Bayesian approaches.

1. Empirical Uncertainty Estimation: Here, ensembles of networks are utilized to quantify uncertainty empirically. Techniques such as Monte Carlo (MC) dropout, bootstrapped ensembles, and SGDR ensembles are explored. It was observed that these empirical approaches offer satisfactory results, albeit with increased computational demands.
2. Predictive Uncertainty Estimation: The authors propose the use of parametric models within the network to directly estimate the distribution of flow predictions. This method provides a more efficient uncertainty quantification by avoiding the need for multiple network evaluations.
3. Bayesian Uncertainty Estimation: By integrating Bayesian inference through dropout as a variational approximation technique, the paper attempts to provide a more theoretically sound foundation for uncertainty estimation. This includes estimating the model's epistemic uncertainty, which is particularly relevant in assessing the risk of deploying models in critical applications.

The paper also presents a novel multi-headed network inspired by the Winner-Takes-All (WTA) strategy, which provides diverse hypotheses for the predicted flow field. This approach allows the network to make various predictions where the ground-truth is unclear, thus capturing more information on potential outcomes.

Experimental Results and Analysis

A comprehensive evaluation is conducted across multiple datasets, including the Sintel clean and KITTI datasets. One significant finding is the substantial performance improvement in uncertainty estimation achieved when implementing predictive methods compared to empirical and standalone models. Specifically:

• The predictive models demonstrate superior uncertainty quantification, reflected by the area under the sparsification error curve (AUSE).
• Multi-hypotheses networks (FlowNetH) combined with merging networks show competitive performance with an efficient runtime, enabling real-time applications.

Implications and Future Directions

The implications of this work are twofold: First, it provides a robust framework for achieving reliable uncertainty estimates, which is crucial in risk-sensitive applications like autonomous driving. Second, the development of networks capable of generating multiple hypotheses offers a pathway towards systems that can adapt to different scenarios by understanding when the prediction is uncertain and requires human intervention or further validation.

Future research may explore further optimization of these methods, potentially integrating additional sensor data to improve the robustness of flow estimation networks. Additionally, extending these concepts to other domains requiring uncertainty estimates could provide significant advances in areas like medical imaging or robotics.

In conclusion, this paper contributes to advancing the field of optical flow estimation by not only improving uncertainty quantification techniques but also reinforcing confidence in deploying these models in dynamic, real-world environments where the consequences of errors are potentially severe.