Magnetic order and paramagnetic phases in the quantum J1-J2-J3 honeycomb model

Abstract: Recent work shows that a quantum spin liquid can arise in realistic fermionic models on a honeycomb lattice. We study the quantum spin-1/2 Heisenberg honeycomb model, considering couplings J1, J2, and J3 up to third nearest neighbors. We use an unbiased pseudofermion functional renormalization group method to compute the magnetic susceptibility and determine the ordered and disordered states of the model. Aside from antiferromagnetic, collinear, and spiral order domains, we find a large paramagnetic region at intermediate J2 coupling. For larger J2 within this domain, we find a strong tendency to staggered dimer ordering, while the remaining paramagnetic regime for low J2 shows only weak plaquet and staggered dimer response. We suggest this regime to be a promising region to look for quantum spin liquid states when charge fluctuations would be included.