Knapsack and subset sum problems in nilpotent, polycyclic, and co-context-free groups

Abstract: It is shown that the knapsack problem (introduced by Myasnikov, Nikolaev, and Ushakov) is undecidable in a direct product of sufficiently many copies of the discrete Heisenberg group (which is nilpotent of class 2). Moreover, for the discrete Heisenberg group itself, the knapsack problem is decidable. Hence, decidability of the knapsack problem is not preserved under direct products. It is also shown that for every co-context-free group, the knapsack problem is decidable. For the subset sum problem (also introduced by Myasnikov, Nikolaev, and Ushakov) we show that it belongs to the class NL (nondeterministic logspace) for every finitely generated virtually nilpotent group and that there exists a polycyclic group with an NP-complete subset sum problem.