Defect Engineering for Stabilizing Magnetic and Topological Properties in Mn(Bi1-xSbx)2Te4

Abstract: MnBi2Te4 is a versatile platform for exploring diverse topological quantum states, yet its potential is hampered by intrinsic antisite defects. While Sb substitution has been employed to tune the Fermi level towards the charge neutral point, it exacerbates the formation of Mn-Sb antisite defects. Here, we address this challenge by combining first-principles calculations with strategic synthesis to systematically investigate and control antisite defects in Mn(Bi1-xSbx)2Te4. Our calculations reveal that increasing antisite defect density progressively destroys the field-forced magnetic Weyl state, eventually driving the system into a trivial magnetic insulator. Motivated by these findings, we develop an optimized chemical vapor transport method, yielding high-quality Mn(Bi1-xSbx)2Te4 crystals with significantly reduced antisite defect density. The emergence of strong Shubnikov-de Haas oscillations in the forced ferromagnetic state and a pronounced anomalous Hall effect near charge neutrality, with opposite signs for n- and p-type samples, confirms the type-II Weyl semimetal nature. These findings underscore the critical role of antisite defects in determining the magnetic and topological properties of Mn(Bi1-xSbx)2Te4 and establish defect engineering via optimized synthesis as a crucial strategy for realizing its exotic magnetic topological states.