On linguistic subsets of groups and monoids

Abstract: We study subsets of groups and monoids defined by language-theoretic means, generalizing the classical approach to the word problem. We expand on results by Herbst from 1991 to a more general setting, and for a class of languages $\mathbf{C}$ we define the classes of $\mathbf{C}^\forall$-flat and $\mathbf{C}^\exists$-flat groups. We prove several closure results for these classes of groups, prove a connection with the word problem, and characterize $\mathbf{C}^\forall$-flat groups for several classes of languages. In general, we prove that the class of $\mathbf{C}^\forall$-flat groups is a strict subclass of the class of groups with word problem in $\mathbf{C}$, including for the class $\mathbf{REC}$ of recursive languages, for which $\mathbf{C}^\forall$-flatness for a group resp. monoid is proved to be equivalent to the decidability of the subgroup membership problem resp. the submonoid membership problem. We provide a number of examples, including the Tarski monsters of Ol'shanskii, showing the difficulty of characterizing $\mathbf{C}^\exists$-flat groups. As an application of our general methods, we also prove in passing that if $\mathbf{C}$ is a full semi-$\mathrm{AFL}$, then the class of epi-$\mathbf{C}$ groups is closed under taking finite index subgroups. This answers a question recently posed by Al Kohli, Bleak & Elliott.