Maximum Entropy of Sums of Independent Ternary Random Variables

Abstract: The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if (X_1,\ldots,X_n) are independent random variables taking values in ({0,1,2}), then the entropy of (S_n=X_1+\cdots+X_n) is maximized when (X_1,\ldots,X_{n-1}) are uniform on ({0,2}) and the probability mass function of (X_n) is given by (\Prob(X_n=0) = \Prob(X_n=2) = w/2), (\Prob(X_n=1) = 1-w), where (w = \big(1 + 2^{-H(B_n)+H(B_{n-1})}\big)^{-1}) and (B_m\sim \Bin(m,1/2)). The statement can be seen as an extension to ternary alphabets of the Shepp--Olkin--Mateev theorem. The proof uses the Hermite--Biehler theorem, Newton's inequalities, and Yu's maximum-entropy theorem for ultra-log-concave distributions.