On the Success Probability of the Box-Constrained Rounding and Babai Detectors

Abstract: In communications, one frequently needs to detect a parameter vector $\hbx$ in a box from a linear model. The box-constrained rounding detector $\x^\sBR$ and Babai detector $\x^\sBB$ are often used to detect $\hbx$ due to their high probability of correct detection, which is referred to as success probability, and their high efficiency of implimentation. It is generally believed that the success probability $P^\sBR$ of $\x^\sBR$ is not larger than the success probability $P^\sBB$ of $\x^\sBB$. In this paper, we first present formulas for $P^\sBR$ and $P^\sBB$ for two different situations: $\hbx$ is deterministic and $\hbx$ is uniformly distributed over the constraint box. Then, we give a simple example to show that $P^\sBR$ may be strictly larger than $P^\sBB$ if $\hbx$ is deterministic, while we rigorously show that $P^\sBR\leq P^\sBB$ always holds if $\hbx$ is uniformly distributed over the constraint box.