Papers
Topics
Authors
Recent
Search
2000 character limit reached

Static Deep Q-learning for Green Downlink C-RAN

Published 22 May 2024 in cs.IT and math.IT | (2405.13368v1)

Abstract: Power saving is a main pillar in the operation of wireless communication systems. In this paper, we investigate cloud radio access network (C-RAN) capability to reduce power consumption based on the user equipment (UE) requirement. Aiming to save the long-term C-RAN energy consumption, an optimization problem is formulated to manage the downlink power without degrading the UE requirement by designing the power offset parameter. Considering stochastic traffic arrivals at UEs, we first formulate the problem as a Markov decision process (MDP) and then set up a dual objective optimization problem in terms of the downlink throughput and power. To solve this optimization problem, we develop a novel static deep Q-learning (SDQL) algorithm to maximize the downlink throughput and minimize the downlink power. In our proposed algorithm, we design multi-Q-tables to simultaneously optimize power reductions of activated RRHs by assigning one Q-table for each UE. To maximize the accumulative reward in terms of the downlink throughput loss and power reduction, our proposed algorithm performs power reductions of activated RRHs through continuous environmental interactions. Simulation results1 show that our proposed algorithm enjoys a superior average power reduction compared to the activation and sleep schemes, and enjoys a low computational complexity.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. S. Gvozdenovic, J. K. Becker, J. Mikulskis, and D. Starobinski, “IoT-Scan: Network Reconnaissance for the Internet of Things,” IEEE Internet of Things Journal, pp. 1–1, 2023.
  2. W. Chen, X. Lin, J. Lee, A. Toskala, S. Sun, C. F. Chiasserini, and L. Liu, “5G-Advanced Toward 6G: Past, Present, and Future,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 6, pp. 1592–1619, 2023.
  3. Y. Chang, H. Tang, Y. Cheng, Q. Zhao, X. Yuan, and B. Li, “Dynamic Hierarchical Energy-Efficient Method Based on Combinatorial Optimization for Wireless Sensor Networks,” Sensors, vol. 17, no. 7, 2017.
  4. N. Saxena, A. Roy, and H. Kim, “Traffic-Aware Cloud RAN: A Key for Green 5G Networks,” IEEE J. Sel. Areas Commun., vol. 34, no. 4, pp. 1010–1021, 2016.
  5. D. Pompili, A. Hajisami, and T. X. Tran, “Elastic resource utilization framework for high capacity and energy efficiency in cloud RAN,” IEEE Commun. Mag., vol. 54, no. 1, pp. 26–32, 2016.
  6. M. Peng, Y. Sun, X. Li, Z. Mao, and C. Wang, “Recent advances in cloud radio access networks: System architectures, key techniques, and open issues,” IEEE Commun. Surveys Tuts., vol. 18, no. 3, pp. 2282–2308, 2016.
  7. J. Wu, “Green wireless communications: from concept to reality,” IEEE Wireless Commun., vol. 19, no. 4, pp. 4–5, 2012.
  8. M. A. Marotta, H. Ahmadi, J. Rochol, L. DaSilva, and C. B. Both, “Characterizing the relation between processing power and distance between BBU and RRH in a cloud RAN,” ” IEEE Wireless Commun. Lett., vol. 7, no. 3, pp. 472–475, 2017.
  9. Y. Chang, X. Yuan, B. Li, D. Niyato, and N. Al-Dhahir, “A Joint Unsupervised Learning and Genetic Algorithm Approach for Topology Control in Energy-Efficient Ultra-Dense Wireless Sensor Networks,” IEEE Commun. Lett., vol. 22, no. 11, pp. 2370–2373, 2018.
  10. X. Yue, K. Sun, W. Huang, X. Liu, and H. Zhang, “Beamforming Design and BBU Computation Resource Allocation for Power Minimization in Green C-RAN,” in Proc. IEEE Int. Conf. Commun. (ICC), 2021, pp. 1–6.
  11. C. Freitag, M. Berners-Lee, K. Widdicks, B. Knowles, G. Blair, and A. Friday, “The climate impact of ICT: A review of estimates, trends and regulations,” 2021.
  12. B. Mao, F. Tang, Y. Kawamoto, and N. Kato, “AI models for green communications towards 6G,” IEEE Commun. Surveys Tuts., vol. 24, no. 1, pp. 210–247, 2021.
  13. R. Li, Z. Zhao, X. Chen, J. Palicot, and H. Zhang, “TACT: A Transfer Actor-Critic Learning Framework for Energy Saving in Cellular Radio Access Networks,” IEEE Trans. Wireless Commun., vol. 13, no. 4, pp. 2000–2011, 2014.
  14. R. Mahapatra, Y. Nijsure, G. Kaddoum, N. U. Hassan, and C. Yuen, “Energy efficiency tradeoff mechanism towards wireless green communication: A survey,” IEEE Commun. Surveys Tuts., vol. 18, no. 1, pp. 686–705, 2015.
  15. S. Zhang, Q. Wu, S. Xu, and G. Y. Li, “Fundamental Green Tradeoffs: Progresses, Challenges, and Impacts on 5G Networks,” IEEE Commun. Surveys Tuts., vol. 19, no. 1, pp. 33–56, 2017.
  16. B. Dai and W. Yu, “Energy Efficiency of Downlink Transmission Strategies for Cloud Radio Access Networks,” IEEE J. Sel. Areas Commun., vol. 34, no. 4, pp. 1037–1050, 2016.
  17. T. Sigwele, Y. F. Hu, and M. Susanto, “Energy-efficient 5G cloud RAN with virtual BBU server consolidation and base station sleeping,” J. Netw. Comput., vol. 177, p. 107302, 2020.
  18. Z. Yang, M. Chen, W. Saad, and M. Shikh-Bahaei, “Optimization of Rate Allocation and Power Control for Rate Splitting Multiple Access (RSMA),” IEEE Trans. Commun., vol. 69, no. 9, pp. 5988–6002, 2021.
  19. Y. Mao, O. Dizdar, B. Clerckx, R. Schober, P. Popovski, and H. V. Poor, “Rate-Splitting Multiple Access: Fundamentals, Survey, and Future Research Trends,” IEEE Commun. Surveys Tuts., vol. 24, no. 4, pp. 2073–2126, 2022.
  20. W. Teng, M. Sheng, X. Chu, K. Guo, J. Wen, and Z. Qiu, “Joint Optimization of Base Station Activation and User Association in Ultra Dense Networks Under Traffic Uncertainty,” IEEE Trans. Commun., vol. 69, no. 9, pp. 6079–6092, 2021.
  21. R. Tao, W. Liu, X. Chu, and J. Zhang, “An Energy Saving Small Cell Sleeping Mechanism With Cell Range Expansion in Heterogeneous Networks,” IEEE Trans. Wireless Commun., vol. 18, no. 5, pp. 2451–2463, 2019.
  22. M. Qin, W. Wu, Q. Yang, R. Zhang, N. Cheng, H. Zhou, R. R. Rao, and X. Shen, “Green-oriented dynamic resource-on-demand strategy for multi-RAT wireless networks powered by heterogeneous energy sources,” IEEE Trans. Wireless Commun., vol. 19, no. 8, pp. 5547–5560, 2020.
  23. M. Oikonomakou, A. Antonopoulos, L. Alonso, and C. Verikoukis, “Evaluating Cost Allocation Imposed by Cooperative Switching Off in Multioperator Shared HetNets,” IEEE Trans. Veh. Technol., vol. 66, no. 12, pp. 11 352–11 365, 2017.
  24. A. Bousia, E. Kartsakli, A. Antonopoulos, L. Alonso, and C. Verikoukis, “Multiobjective Auction-Based Switching-Off Scheme in Heterogeneous Networks: To Bid or Not to Bid?” IEEE Trans. Veh. Technol., vol. 65, no. 11, pp. 9168–9180, 2016.
  25. Y. Chang, H. Tang, B. Li, and X. Yuan, “Distributed joint optimization routing algorithm based on the analytic hierarchy process for wireless sensor networks,” IEEE Communications Letters, vol. 21, no. 12, pp. 2718–2721, 2017.
  26. K. B. Letaief, W. Chen, Y. Shi, J. Zhang, and Y.-J. A. Zhang, “The roadmap to 6G: AI empowered wireless networks,” IEEE Commun. Mag., vol. 57, no. 8, pp. 84–90, 2019.
  27. Y. Chang, X. Yuan, B. Li, D. Niyato, and N. Al-Dhahir, “Machine-Learning-Based Parallel Genetic Algorithms for Multi-Objective Optimization in Ultra-Reliable Low-Latency WSNs,” IEEE Access, vol. 7, pp. 4913–4926, 2019.
  28. Z. Gu, C. She, W. Hardjawana, S. Lumb, D. McKechnie, T. Essery, and B. Vucetic, “Knowledge-assisted deep reinforcement learning in 5G scheduler design: From theoretical framework to implementation,” IEEE J. Sel. Areas Commun., vol. 39, no. 7, pp. 2014–2028, 2021.
  29. C. Xu, J. Wang, T. Yu, C. Kong, Y. Huangfu, R. Li, Y. Ge, and J. Wang, “Buffer-aware wireless scheduling based on deep reinforcement learning,” in Proc. IEEE Wireless Commun. Netw. Conf. (WCNC).   IEEE, 2020, pp. 1–6.
  30. J. Wang, C. Xu, Y. Huangfu, R. Li, Y. Ge, and J. Wang, “Deep reinforcement learning for scheduling in cellular networks,” in Proc. 11th Int. Conf. Wireless Commun. Signal Process. (WCSP).   IEEE, 2019, pp. 1–6.
  31. L. Zhang, X. Ma, Z. Zhuang, H. Xu, V. Sharma, and Z. Han, “Q-Learning Aided Intelligent Routing With Maximum Utility in Cognitive UAV Swarm for Emergency Communications,” IEEE Trans. Veh. Technol., vol. 72, no. 3, pp. 3707–3723, 2022.
  32. Y. Ding, Y. Zhao, Y. Gao, and R. Zhang, “Q-Learning Quantum Ant Colony Routing Algorithm for Micro-Nano Satellite Network,” in in Proc. IEEE 6th Int. Conf. Comput. Commun. Syst. (ICCCS), 2021, pp. 949–954.
  33. Y. Chen, K. Zheng, X. Fang, L. Wan, and X. Xu, “QMCR: A Q-learning-based multi-hop cooperative routing protocol for underwater acoustic sensor networks,” China Commun., vol. 18, no. 8, pp. 224–236, 2021.
  34. S. Zou, W. Wang, W. Ni, L. Wang, and Y. Tang, “Efficient Orchestration of Virtualization Resource in RAN Based on Chemical Reaction Optimization and Q-Learning,” IEEE Internet Things J., vol. 9, no. 5, pp. 3383–3396, 2022.
  35. A. Ndikumana, K. K. Nguyen, and M. Cheriet, “Federated Learning Assisted Deep Q-Learning for Joint Task Offloading and Fronthaul Segment Routing in Open RAN,” IEEE Trans. Netw. Service Manage., pp. 1–1, 2023.
  36. A. A. hammadi, L. Bariah, S. Muhaidat, M. Al-Qutayri, P. C. Sofotasios, and M. Debbah, “Deep Q-Learning-Based Resource Allocation in NOMA Visible Light Communications,” IEEE Open J. Commun. Soc., vol. 3, pp. 2284–2297, 2022.
  37. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, 2015.
  38. Y. Chang, W. Chen, J. Li, J. Liu, H. Wei, Z. Wang, and N. Al-Dhahir, “Collaborative Multi-BS Power Management for Dense Radio Access Network Using Deep Reinforcement Learning,” IEEE Trans. Green Commun. Netw., vol. 7, no. 4, pp. 2104–2116, 2023.
  39. A. V. Lopez, A. Chervyakov, G. Chance, S. Verma, and Y. Tang, “Opportunities and Challenges of mmWave NR,” IEEE Wireless Commun., vol. 26, no. 2, pp. 4–6, 2019.
  40. F. Marzouk, T. Akhtar, I. Politis, J. P. Barraca, and A. Radwan, “Power Minimizing BBU-RRH Group Based Mapping in C-RAN with Constrained Devices,” in Proc. IEEE Int. Conf. Commun. (ICC), 2020, pp. 1–7.
  41. Y. Yuan and L. Zhu, “Application scenarios and enabling technologies of 5G,” China Commun., vol. 11, no. 11, pp. 69–79, 2014.
  42. O. T. H. Alzubaidi, M. N. Hindia, K. Dimyati, K. A. Noordin, A. N. A. Wahab, F. Qamar, and R. Hassan, “Interference challenges and management in B5G network design: A comprehensive review,” Electronics, vol. 11, no. 18, p. 2842, 2022.
  43. A. A. Amer, I. E. Talkhan, and T. Ismail, “Optimal Power Consumption on Distributed Edge Services Under Non-Uniform Traffic with Dual Threshold Sleep/Active Control,” in 2021 3rd Novel Intelligent and Leading Emerging Sciences Conference (NILES), 2021, pp. 63–66.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up