Plug-in Hybrid Electric Vehicle Energy Management with Clutch Engagement Control via Continuous-Discrete Reinforcement Learning

Abstract: Energy management strategy (EMS) is a key technology for plug-in hybrid electric vehicles (PHEVs). The energy management of certain series-parallel PHEVs involves the control of continuous variables, such as engine torque, and discrete variables, such as clutch engagement/disengagement. We establish a control-oriented model for a series-parallel plug-in hybrid system with clutch engagement control from the perspective of mixed-integer programming. Subsequently, we design an EMS based on continuous-discrete reinforcement learning (CDRL), which enables simultaneous output of continuous and discrete variables. During training, we introduce state-of-charge (SOC) randomization to ensure that the hybrid system exhibits optimal energy-saving performance in both high and low SOC. Finally, the effectiveness of the proposed CDRL strategy is verified by comparing EMS based on charge-depleting charge-sustaining (CD-CS) with rule-based clutch engagement control, and Dynamic Programming (DP). The simulation results show that, under a high SOC, the CDRL strategy proposed in this paper can improve energy efficiency by 8.3% compared to CD-CS, and the energy consumption is just 6.6% higher than the global optimum based on DP, while under a low SOC, the numbers are 4.1% and 3.9%, respectively.