Spin interaction and magnetism in cobaltate Kitaev candidate materials: an $ab$ $initio$ and model Hamiltonian approach

Abstract: In the quest for materials hosting Kitaev spin liquids, much of the efforts have been focused on the fourth- and fifth-row transition metal compounds, which are spin-orbit coupling assisted Mott insulators. Here, we study the structural and magnetic properties of 3$d$ transition metal oxides, Na$_2$Co$_2$TeO$_6$ and Na$_3$Co$_2$SbO$_6$. The partial occupancy of sodium in former compound is addressed using a cluster expansion, and a honeycomb lattice of sodiums is found to be energetically favored. Starting from the \textit{ab initio} band structures, a many-body second order perturbation theory leads to a pseudospin-$\frac{1}{2}$ Hamiltonian with estimated magnetic interactions. We show that the experimentally observed zigzag magnetic state is stabilized only when the first neighbor Kitaev coupling dominates over the Heisenberg term, both of which are highly suppressed due to presence of $e_g$ orbitals. A third neighbor Heisenberg interaction is found dominant in both these compounds. We also present a phase diagram for Na$_2$Co$_2$TeO$_6$ by varying the electron-electron and spin-orbit interactions. The computed spin excitation spectra are found to capture essential features of recent experimental magnon spectrum, lending support to our results.