Spin-liquid versus spiral-order phases in the anisotropic triangular lattice

Abstract: We study the competition between magnetic and spin-liquid phases in the Hubbard model on the anisotropic triangular lattice, which is described by two hopping parameters t and t' in different spatial directions and is relevant for layered organic charge-transfer salts. By using a variational approach that includes spiral magnetic order, we provide solid evidence that a spin-liquid phase is stabilized in the strongly-correlated regime and close to the isotropic limit t'/t=1. Otherwise, a magnetically ordered spiral state is found, connecting the (collinear) N\'eel and the (coplanar) 120{\deg} phases. The pitch vector of the spiral phase obtained from the unrestricted Hartree-Fock approximation is substantially renormalized in presence of electronic correlations, and the N\'eel phase is stabilized in a wide regime of the phase diagram, i.e., for t'/t < 0.75. We discuss these results in the context of organic charge-transfer salts.