Design Optimization of NOMA Aided Multi-STAR-RIS for Indoor Environments: A Convex Approximation Imitated Reinforcement Learning Approach

Abstract: Non-orthogonal multiple access (NOMA) enables multiple users to share the same frequency band, and simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) provides 360-degree full-space coverage, optimizing both transmission and reflection for improved network performance and dynamic control of the indoor environment. However, deploying STAR-RIS indoors presents challenges in interference mitigation, power consumption, and real-time configuration. In this work, a novel network architecture utilizing multiple access points (APs), STAR-RISs, and NOMA is proposed for indoor communication. To address these, we formulate an optimization problem involving user assignment, access point (AP) beamforming, and STAR-RIS phase control. A decomposition approach is used to solve the complex problem efficiently, employing a many-to-one matching algorithm for user-AP assignment and K-means clustering for resource management. Additionally, multi-agent deep reinforcement learning (MADRL) is leveraged to optimize the control of the STAR-RIS. Within the proposed MADRL framework, a novel approach is introduced in which each decision variable acts as an independent agent, enabling collaborative learning and decision making. The MADRL framework is enhanced by incorporating convex approximation (CA), which accelerates policy learning through suboptimal solutions from successive convex approximation (SCA), leading to faster adaptation and convergence. Simulations demonstrate significant improvements in network utility compared to baseline approaches.