Approximate dynamic programming for profit estimation of connected hydro reservoirs

Abstract: In this paper, we study the operational problem of connected hydro power reservoirs which involves sequential decision-making in an uncertain and dynamic environment. The problem is traditionally formulated as a stochastic dynamic program accounting for the uncertainty of electricity prices and reservoir inflows. This formulation suffers from the curse of dimensionality, as the state space explodes with the number of reservoirs and the history of prices and inflows. To avoid computing the expectation of future value functions, the proposed model takes advantage of the so-called post-decision state. To further tackle the dimensionality issue, we propose an approximate dynamic programming approach that estimates the future value of water using a linear approximation architecture. When the time series of prices and inflows follow autoregressive processes, our approximation provides an upper bound on the future value function. We use an offline training algorithm based on the historical data of prices and inflows and run both in-sample and out-of-sample simulations. Two realistic test systems of cascade and network connected reservoirs serve to demonstrate the computational tractability of our approach. In particular, we provide numerical evidence of convergence and quality of solutions. For our test systems, our results show that profit estimation is improved by 20% when including inflows in the linear approximation.