Real-time Building Energy Storage Scheduling under Electrical Load Uncertainty: A Dynamic Markov Decision Process Approach with Comprehensive Analysis of Different Pricing Policies

Abstract: In response to the increasing deployment of battery storage systems for cost reduction and grid stress mitigation, this study presents the development of a new real-time Markov decision process model to efficiently schedule battery systems in buildings under electrical load uncertainty. The proposed model incorporates quantile Fourier regression for load fitting, leading to a large-scale optimization problem with approximately a million decision variables and constraints. To address this complexity, the problem is formulated as a linear program and solved using a commercial solver, ensuring effective navigation and identification of optimal solutions. The model's performance is evaluated by considering different pricing policies and scenarios, including demand peak shaving. Validation of the Markov model is conducted using one year of historical demand data from a school. Findings indicate that MDP performance in adapting to uncertain loads can range from 30 to 99% depending on the pricing policy.