Higher order exchange driven noncoplanar magnetic state and large anomalous Hall effects in electron doped kagome magnet Mn$_3$Sn

Abstract: Owing to the geometrical frustration, Mn$_3 $Sn exhibits a 120$^{\circ}$ in-plane triangular antiferromagnetic (AFM) order with a large anomalous Hall effect (AHE). Here, we present a combined theoretical and experimental study to demonstrate that the in-plane AFM structure in Mn$_3 $Sn can be significantly modified to a tunable noncoplanar magnetic state by suitable electron doping. With the help of Density Functional Theory calculations we show that the presence of higher-order exchange interactions in the system leads to the stabilization of the noncoplanar magnetic ground state, which is further established by neutron diffraction study in the Fe-doped Mn$_3 $Sn sample. Interestingly, we find a large scalar spin chirality (SSC) induced topological AHE that can be significantly tuned with the degree of non-coplanarity. We carry out 60~T magnetic and Hall resistivity measurements to demonstrate the contribution of SSC to the observed AHE. We also illustrate a simultaneous manipulation of dual order in the system, where the AHE arising from the in-plane 120$^{\circ}$ triangular AFM order can switch its sign without affecting the AHE generated by the SSC. The present study opens up a new direction to explore novel quantum phenomena associated with the coexistence of multiple orders.