Competing Chiral Orders in the Topological Haldane-Hubbard Model of Spin-1/2 Fermions and Bosons

Abstract: Motivated by experiments on ultracold atoms which have realized the Haldane model for a Chern insulator, we consider its strongly correlated Mott limit with spin-$1/2$ fermions. We find that slave rotor mean field theory yields gapped or gapless chiral spin liquid Mott insulators. To study competing magnetic orders, we consider the strong coupling effective spin Hamiltonian which includes chiral three-spin exchange. We obtain its classical phase diagram, uncovering various chiral magnetic orders including tetrahedral, cone, and noncoplanar spiral states which can compete with putative chiral quantum spin liquids. We study the effect of thermal fluctuations on these states, identifying crossovers in the spin chirality, and phase transitions associated with lattice symmetry breaking. We also discuss analogous effective spin Hamiltonians for correlated spin-$1/2$ bosons. Finally, we point out possible experimental implications of our results for cold atom experiments.