Koopman Ensembles for Probabilistic Time Series Forecasting

Abstract: In the context of an increasing popularity of data-driven models to represent dynamical systems, many machine learning-based implementations of the Koopman operator have recently been proposed. However, the vast majority of those works are limited to deterministic predictions, while the knowledge of uncertainty is critical in fields like meteorology and climatology. In this work, we investigate the training of ensembles of models to produce stochastic outputs. We show through experiments on real remote sensing image time series that ensembles of independently trained models are highly overconfident and that using a training criterion that explicitly encourages the members to produce predictions with high inter-model variances greatly improves the uncertainty quantification of the ensembles.

• The paper introduces a novel variance-promoting loss term for deep ensembles of Koopman autoencoders, addressing overconfidence in predictions.
• It uses neural autoencoders to define a latent space that approximates nonlinear dynamical systems via linear evolution.
• Experiments on multispectral satellite time series validate improved uncertainty estimates using CRPS scores and spread-skill plots.

Koopman Ensembles for Probabilistic Time Series Forecasting

Introduction to Koopman Autoencoders and Uncertainty Quantification

Recent advancements in data-driven models, particularly machine learning-based implementations of the Koopman operator, have shown promising results in forecasting the dynamics of physical systems with high accuracy. However, most of these models are deterministic and do not account for uncertainty in their predictions. Given the critical importance of uncertainty quantification in fields like meteorology and climatology, this paper investigates the training of ensembles of models to produce stochastic outputs. Through experiments on real remote sensing image time series, it is demonstrated that ensembles of independently trained models exhibit a tendency towards overconfidence. To address this, a training criterion that explicitly encourages members of the ensemble to produce predictions with high inter-model variances is introduced, significantly improving the uncertainty quantification capabilities of these ensembles.

Data-Driven Koopman Operator Implementations

The Koopman operator theory posits that any nonlinear dynamical system can be represented by a linear operator acting on the set of its measurement functions. This paper focuses on finite-dimensional representations based on neural auto-encoders, which have shown promise in various applications. By defining a latent space via the encoder and using a matrix to govern the evolution of the latent state through time, these models can approximate the dynamical system in question. However, a notable limitation has been the deterministic nature of these models, which this study seeks to overcome.

Uncertainty Quantification for Neural Networks

Uncertainty in machine learning models can be categorized into aleatoric and epistemic uncertainties. Traditional methods for uncertainty quantification in neural networks, although numerous, are not without their challenges. This paper primarily explores the use of deep ensembles as a straightforward yet effective approach for introducing stochasticity into the otherwise deterministic predictions of Koopman autoencoders.

Proposed Methodology

The authors propose a novel training methodology for ensemble models that incorporates a variance-promoting loss term alongside traditional loss components. This term is designed to encourage diversity among the predictions of ensemble members, thus allowing for better representation of model uncertainty. A crucial aspect of this approach is the careful selection of the weight assigned to the variance-promoting term, which is shown through theoretical analysis and empirical validation to play a significant role in the ensemble's performance and uncertainty quantification capability.

Experimental Setup and Results

Experiments conducted on time series data of multispectral satellite images demonstrate the effectiveness of the proposed training methodology. The authors evaluate the performance of ensembles trained with varying weights assigned to the variance-promoting loss term, using metrics such as the Continuous Ranked Probability Score (CRPS) and spread-skill plots. The results indicate that ensembles with a higher weight on the variance-promoting term tend to produce less overconfident and more reliable uncertainty estimates, as evidenced by their improved CRPS scores and closer alignment to the ideal spread-skill relationship.

Concluding Remarks and Future Directions

This study makes a significant contribution by addressing the overconfidence issue prevalent in ensembles of Koopman autoencoders and providing a method to improve their uncertainty quantification capabilities. The proposed variance-promoting training criterion offers a promising direction for future research into ensemble-based forecasting models and their application in uncertainty-sensitive domains. Further exploration into combining this approach with other uncertainty quantification techniques and extending its application to other types of dynamical systems could yield additional insights and advancements in the field of probabilistic forecasting with machine learning models.