Quantum liquids of the S=3/2 Kitaev honeycomb and related Kugel-Khomskii models

Abstract: The $S=3/2$ Kitaev honeycomb model (KHM) is unique among the spin-$S$ Kitaev models due to a massive ground state quasi-degeneracy that hampered previous numerical and analytical studies. In a recent work~\cite{jin2022unveiling}, we showed how an SO(6) Majorana parton mean-field theory of the $S=3/2$ isotropic KHM explains the anomalous features of this Kitaev spin liquid (KSL) in terms of an emergent low-energy Majorana flat band. Away from the isotropic limit, the $S=3/2$ KSL generally displays a quadrupolar order with gapped or gapless Majorana excitations, features that were quantitatively confirmed by DMRG simulations. In this paper, we explore the connection between the $S = 3/2$ KHM with Kugel-Khomskii models and discover new exactly soluble examples for the latter. We perform a symmetry analysis for the variational parton mean-field \emph{Ans{\"a}tze} in the spin and orbital basis for different quantum liquid phases of the $S=3/2$ KHM. Finally, we investigate a proposed time-reversal symmetry breaking spin liquid induced by a {[}111{]} single ion anisotropy and elucidate its topological properties as well as experimental signatures, e.g. an unquantized thermal Hall response.