Spin liquid phase in a spatially anisotropic frustrated antiferromagnet

Abstract: We explore the effect of the third nearest-neighbors on the magnetic properties of the Heisenberg model on an anisotropic triangular lattice. We obtain the phase diagram of the model using Schwinger-boson mean-field theory. Competition between N\'eel, spiral and collinear magnetically ordered phases is found as we vary the on the ratios of the nearest, J1, next-nearest, J2, and third-nearest, J_3, neighbor exchange couplings. A spin liquid phase is stabilized between the spiral and collinear ordered states when J2/J1 < 1.8 for rather small J3/J1 < 0.1. The lowest energy two-spinon dispersions relevant to neutron scattering experiments are analyzed and compared to semiclassical magnon dispersions finding significant differences in the spiral and collinear phases between the two approaches. The results are discussed in the context of the anisotropic triangular materials: Cs2CuCl4 and Cs2CuBr4 and layered organic materials, kappa-(BEDT-TTF)2X and Y[Pd(dmit)2]2.