Canted antiferromagnetism in a spin-orbit coupled $S_{\text{eff}} = 3/2$ triangular-lattice magnet DyAuGe

Abstract: Exploration of nontrivial magnetic states induced by strong spin-orbit interaction is a central topic of frustrated magnetism. Extensive studies are concentrated on rare-earth-based magnets and 4d/5d transition metal compounds, which are mostly described by an effective spin $S_{\text{eff}} = 1/2$ for the Kramers doublet of the lowest crystal-electric-field levels. Variety of magnetic orderings may be greatly enhanced when magnetic dipolar moments intertwined with multipolar degrees of freedom which are described by higher-rank tensors and often require the magnetic ions with $S_{\text{eff}} > 1/2$. Here, our synchrotron x-ray diffraction near the Dy $L_3$ edge has unveiled a canted antiferromagnetic ground state arising from a quasi-quartet ($S_{\text{eff}} = 3/2$) of 4f electrons in a triangular-lattice (TL) rare-earth intermetallics DyAuGe. Magnetic moment and electric-quadrupole moment are closely interlocked and noncollinear magnetic-dipole alignment is induced by antiferroic electric-quadrupole (AFQ) ordering in the TL layers. The correlation between the AFQ and canted magnetic structures is further confirmed by phase transitions in an in-plane magnetic field. These findings offer insights into the emergence of nontrivial magnetic states in frustrated TL systems described beyond the $S_{\text{eff}} = 1/2$.