Lattice attack on group ring NTRU: The case of the dihedral group

Abstract: Group ring NTRU (GR-NTRU) provides a general structure to design different variants of NTRU-like schemes by employing different groups. Although, most of the schemes in literature are built over cyclic groups, nonabelian groups can also be used. Coppersmith and Shamir in 1997 have suggested that noncommutativity may result in better security against some lattice attacks for some groups. Lattice attacks on the public key of NTRU-like cryptosystems try to retrieve the private key by solving the shortest vector problem (SVP) or its approximation in a lattice of a certain dimension, assuming the knowledge of the public key only. This paper shows that dihedral groups do not guarantee better security against this class of attacks. We prove that retrieving the private key is possible by solving the SVP in two lattices with half the dimension of the original lattice generated for GR-NTRU based on dihedral groups. The possibility of such an attack was mentioned by Yasuda et al.(IACR/2015/1170). In contrast to their proposed approach, we explicitly provide the lattice reduction without any structure theorem from the representation theory for finite groups. Furthermore, we demonstrate the effectiveness of our technique with experimental results.