A Framework on Hybrid MIMO Transceiver Design based on Matrix-Monotonic Optimization

Abstract: Hybrid transceiver can strike a balance between complexity and performance of multiple-input multiple-output (MIMO) systems. In this paper, we develop a unified framework on hybrid MIMO transceiver design using matrix-monotonic optimization. The proposed framework addresses general hybrid transceiver design, rather than just limiting to certain high frequency bands, such as millimeter wave (mmWave) or terahertz bands or relying on the sparsity of some specific wireless channels. In the proposed framework, analog and digital parts of a transceiver, either linear or nonlinear, are jointly optimized. Based on matrix-monotonic optimization, we demonstrate that the combination of the optimal analog precoders and processors are equivalent to eigenchannel selection for various optimal hybrid MIMO transceivers. From the optimal structure, several effective algorithms are derived to compute the analog transceivers under unit modulus constraints. Furthermore, in order to reduce computation complexity, a simple random algorithm is introduced for analog transceiver optimization. Once the analog part of a transceiver is determined, the closed-form digital part can be obtained. Numerical results verify the advantages of the proposed design.