2DXformer: Dual Transformers for Wind Power Forecasting with Dual Exogenous Variables

Abstract: Accurate wind power forecasting can help formulate scientific dispatch plans, which is of great significance for maintaining the safety, stability, and efficient operation of the power system. In recent years, wind power forecasting methods based on deep learning have focused on extracting the spatiotemporal correlations among data, achieving significant improvements in forecasting accuracy. However, they exhibit two limitations. First, there is a lack of modeling for the inter-variable relationships, which limits the accuracy of the forecasts. Second, by treating endogenous and exogenous variables equally, it leads to unnecessary interactions between the endogenous and exogenous variables, increasing the complexity of the model. In this paper, we propose the 2DXformer, which, building upon the previous work's focus on spatiotemporal correlations, addresses the aforementioned two limitations. Specifically, we classify the inputs of the model into three types: exogenous static variables, exogenous dynamic variables, and endogenous variables. First, we embed these variables as variable tokens in a channel-independent manner. Then, we use the attention mechanism to capture the correlations among exogenous variables. Finally, we employ a multi-layer perceptron with residual connections to model the impact of exogenous variables on endogenous variables. Experimental results on two real-world large-scale datasets indicate that our proposed 2DXformer can further improve the performance of wind power forecasting. The code is available in this repository: \href{https://github.com/jseaj/2DXformer}{https://github.com/jseaj/2DXformer}.

Dual Transformers for Wind Power Forecasting: An Overview of 2DXformer

The study presented in "2DXformer: Dual Transformers for Wind Power Forecasting with Dual Exogenous Variables" introduces a novel transformer-based model tailored for the accurate prediction of wind power outputs. This development is underpinned by the pressing need for reliable wind power forecasts to ensure the smooth operation, safety, and efficiency of power systems, given the challenges presented by the intermittent nature of wind energy.

Model and Methodology

The research identifies two main limitations in existing deep learning approaches used for spatiotemporal wind power forecasting. Firstly, the inadequate modeling of inter-variable relationships, and secondly, the uniform treatment of endogenous and exogenous variables, which can complicate models and degrade forecast accuracy. To address these, the authors propose the 2DXformer model, which successfully differentiates between these variable types and focuses distinctly on their interactions.

The 2DXformer is predicated on an Encoder-Only Transformer architecture, equipped with separate embeddings for exogenous static variables (such as wind speed and temperature), exogenous dynamic variables (related to temporal elements like time of day), and endogenous variables (namely, the wind power outputs themselves). These embeddings facilitate sophisticated variable differentiation by employing distinct processing blocks: EnTBlock for endogenous and ExTBlock for exogenous variables.

Incorporating multi-head attention mechanisms, ExTBlock focuses on inter-variable and spatial correlations, while EnTBlock handles the spatial ones and models the impact of exogenous variables on endogenous ones using a Multi-Layer Perceptron (MLP) with residual connections.

Empirical Evaluation

The empirical evaluation, conducted using real-world datasets from two wind farms, demonstrates the effectiveness of the proposed model. 2DXformer consistently outperforms traditional deep learning models (such as MLP, GRU, and Transformer), spatio-temporal models (like AGCRN and MegaCRN), and even models designed with variable differentiation in mind (such as TiDE). With enhancements in metrics like MAE and RMSE, 2DXformer effectively captures the spatiotemporal dynamics of forecasting tasks and reduces prediction errors.

Implications and Future Directions

The implications of this research are significant both in practice and theory. The ability to accurately forecast wind power has substantial operational benefits, aiding in better grid management and planning. From a theoretical standpoint, this research underscores the importance of customized, variable-sensitive models that can adapt to the unique dynamics and complexities of renewable energy sources.

Future research should focus on additional improvements to model efficiency, particularly in reducing computational overhead associated with deeper network architectures. Additionally, integrating more complex real-world constraints and external factors in the modeling process may provide richer insights into improving renewable energy predictions. As the field progresses, strategies to incorporate massive datasets efficiently, perhaps through parallel processing or model compression techniques, may also present significant opportunities for exploration.

Overall, the 2DXformer model represents a notable advancement in specialized transformer architectures applied to renewable energy forecasts, showcasing the potential for cross-disciplinary innovations in artificial intelligence and sustainable energy management.