Mn-Sb Site Mixing in Layered Magnetic Topological Compounds

• Mn-Sb site mixing is the antisite disorder where Mn and Sb atoms swap positions in layered compounds, crucially impacting magnetic ordering and topological properties.
• This disorder modulates exchange interactions and electronic band topology, leading to tunable quantum phases such as antiferromagnetic, ferromagnetic, and spin-glass states.
• Synthesis conditions and ionic size matching govern Mn-Sb defect levels, with diffraction and STEM-EDS techniques precisely quantifying antisite mixing to optimize device performance.

Mn-Sb site mixing refers to the phenomenon whereby manganese (Mn) and antimony (Sb) atoms exchange places, or occupy each other's crystallographic sites, in layered compounds such as MnSb$_2$Te$_4$, MnBi$_2$Te$_4$, and their solid solutions Mn(Bi$_{1-x}$Sb$_x$)$_2$Te$_4$ and Mn(Bi$_{1-x}$Sb$_x$)$_4$Te$_7$. This antisite disorder alters the local electronic structure, magnetic exchange interactions, and topological properties, thereby affecting quantum phases of matter, transport signatures, and potential device applications. These materials host alternating magnetic and topological layers, and the fine control of cation site mixing is central to realizing magnetic topological insulators, Weyl semimetals, and axion insulators. Mn-Sb mixing is typically quantified as the fraction of Mn occupying nominal Sb sites (or vice versa), and its density correlates with synthesis conditions, thermodynamic stability, and the ionic radii of participating atoms.

1. Crystallographic Context and Definitions

In the archetypical MnSb$_2$Te$_4$ (space group: $R\bar{3}m$), septuple layers consist of Te$^1$–Sb$^1$–Te$^2$–Mn–Te$^2$–Sb$^1$–Te$^1$, with the central cation plane ideally populated by Mn, flanked by Sb planes. Mn-Sb antisite disorder arises when Mn occupies Sb $6c$ sites and/or Sb occupies Mn $3a$ sites, resulting in mixed cation occupancy. Occupancy variables are defined for each cation plane:

• $x_\mathrm{Mn@Sb}^{(t)}$: Fraction of Mn on the "top" Sb plane.
• $x_\mathrm{Mn@Sb}^{(b)}$: Fraction on the "bottom" Sb plane.
• $x_\mathrm{Sb@Mn}$: Fraction of Sb on the central Mn plane.

The net site mixing per septuple layer is given by averaging these fractions. For instance, in polycrystalline MnSb$_2$Te$_4$, high-resolution STEM-EDS maps and X-ray diffraction refine these values, revealing substantial antisite mixing: $x_\mathrm{Mn@Sb}^{(t)} \simeq 0.08$, $x_\mathrm{Mn@Sb}^{(b)} \simeq 0.05$, $x_\mathrm{Sb@Mn} \lesssim 0.02$, with an intra-layer asymmetry ($\Delta_\mathrm{site}$) up to $0.03$ (Leon et al., 16 Jan 2026, Li et al., 2021).

This disorder increases with Sb content in Mn(Bi$_{1-x}$Sb$_x$)$_2$Te$_4$: site mixing on Sb planes rises monotonically from $\sim$3\% (x=0) to $\sim$16\% (x=1), as quantified by neutron and X-ray diffraction data (Riberolles et al., 2021, Chen et al., 25 Dec 2025, Liu et al., 2020).

2. Synthesis-Dependent Formation and Thermodynamics

The equilibrium concentration of Mn-Sb antisite defects is set by defect formation energies and growth kinetics. First-principles DFT calculations give defect formation energies for Mn-on-Sb swaps on the order of $E_f$(Mn$_\mathrm{Sb}$) $\sim$ 0.8--1.2 eV in NiMnSb and 0.23--0.28 eV for Mn-on-(Bi,Sb) sites in Mn(Bi$_{1-x}$Sb$_x$)$_2$Te$_4$ under Mn-rich, Te-rich conditions (Belashchenko et al., 2015, Chen et al., 25 Dec 2025). The Boltzmann law, $$c_\mathrm{defect} \approx \exp[-E_f/(k_BT_\mathrm{growth})]$$, yields defect levels up to 20--30\% under specific anneal regimes (e.g., $T_\mathrm{anneal} \simeq 893$ K for 14 days in MnSb$_2$Te$_4$) (Li et al., 2021).

Ionic size-matching amplifies this: Mn$^{2+}$ (0.83 Å) closely matches Sb$^{3+}$ (0.76 Å), reducing the energetic penalty and boosting site mixing compared to Mn-Bi ($R_\mathrm{Bi^{3+}}=1.03$ Å) (Li et al., 2021).

Growth methods critically impact the defect density. Optimized chemical vapor transport (OCVT) suppresses antisite levels to $\lesssim$5\% Mn-on-Sb (for $x=0.20$), and Bridgman/self-flux techniques typically yield higher values (Chen et al., 25 Dec 2025).

3. Experimental Quantification: Structural and Analytical Methods

Site mixing is robustly established by:

• Single-crystal and neutron diffraction: Explicit refinement of occupancy factors $O_j$ for each Wyckoff site ($3a$, $6c$), with charge-neutrality constraints (Li et al., 2021, Liu et al., 2020).
• HAADF-STEM and EDS: Electron column intensity mapping in cross-sectional views, converting peak integral ratios ($I_\mathrm{Mn}/(I_\mathrm{Mn}+I_\mathrm{Sb})$) to site occupancy for precise $\pm0.01$ measurement (Leon et al., 16 Jan 2026).
• DC-SQUID magnetometry: Saturation moment analysis assigning parallel/antiparallel alignment to Mn central/antisite layers yields Mn$_\mathrm{Mn}$ and Mn$_\mathrm{Sb}$ fractions. Example: for $x=0.20$, $a_\mathrm{MnMn}=88.1\%$, $b_\mathrm{MnSb}=4.95\%$ (Chen et al., 25 Dec 2025).
• STM topographies: Triangular depressions on Te layers correspond to Mn$_\mathrm{Sb}$ antisites, giving surface densities of 8--10\% in MnSb$_2$Te$_4$ (Liu et al., 2020).

The following table collates representative Mn@Sb fractions in MnSb$_2$Te$$_4 crystals produced under distinct conditions:</p> <div class='overflow-x-auto max-w-full my-4'><table class='table border-collapse w-full' style='table-layout: fixed'><thead><tr> <th>Method</th> <th>$$x_\mathrm{Mn@Sb}$$</th> <th>Description</th> </tr> </thead><tbody><tr> <td>Single crystal SCXRD (<a href="/papers/2104.00898" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Li et al., 2021</a>)</td> <td>0.193</td> <td>Pronounced site mixing, annealed bulk</td> </tr> <tr> <td>Neutron diffraction (<a href="/papers/2007.12217" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Liu et al., 2020</a>)</td> <td>0.13–0.16</td> <td>Growth-dependent, random antisite distribution</td> </tr> <tr> <td>STEM-EDS, <a href="https://www.emergentmind.com/topics/policy-label-divergence-pld" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">PLD</a> films (<a href="/papers/2601.11353" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Leon et al., 16 Jan 2026</a>)</td> <td>0.08 (t), 0.05 (b)</td> <td>Polycrystalline bulk, thin films are symmetric ($$\Delta_\mathrm{site}\approx 0$$)</td> </tr> </tbody></table></div><h2 class='paper-heading' id='impact-on-magnetic-exchange-and-ground-state-evolution'>4. Impact on Magnetic Exchange and Ground State Evolution</h2> <p>Mn-Sb antisite disorder fundamentally alters magnetic coupling:</p> <ul> <li><strong>Interlayer magnetic transitions</strong>: In Mn(Bi$$_{1-x}$Sb$_x$)$_2$Te$_4$$, antisite content tunes ground state from A-type antiferromagnetic (<a href="https://www.emergentmind.com/topics/adversarial-flow-models-afm" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">AFM</a>,$$x<0.13$) to ferromagnetic (FM,$x>0.13$), the threshold marked by$\Delta E = E_\mathrm{FM} - E_\mathrm{AFM}$$changing sign (<a href="/papers/2007.12217" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Liu et al., 2020</a>, <a href="/papers/2512.21680" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Chen et al., 25 Dec 2025</a>).</li> <li><strong>Ferrimagnetic mode generation</strong>: Antisite Mn forms weak layers (spin$$s < S/2$$) coupled antiferromagnetically to the central Mn layer ($$S$$), producing ferrimagnetic septuples (<a href="/papers/2103.09335" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Riberolles et al., 2021</a>). Linear spin-wave theory yields three magnon branches; the optical mode$$\omega_2(0)\approx0.5\,\mathrm{meV}$$directly matches INS resonance data.</li> <li><strong>Spin gap and damping</strong>: With increasing Sb substitution, the spin gap collapses (from$$\Delta\sim0.6$meV to$\lesssim0.1$meV), magnon bandwidth and damping increase ($\Gamma$from 0.7 meV to$\gtrsim$3 meV). The disorder-driven broadening destroys well-defined magnons (Riberolles et al., 2021).

Spin glass phase: Extreme site mixing ($\sim30\%$$) frustrates AFM interlayer coupling, yielding spin glass freezing ($$T_\mathrm{SG}\sim24$K) in MnSb$_2$Te$_4$$(<a href="/papers/2104.00898" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Li et al., 2021</a>). Canonical SG canonical features (bifurcating FC/ZFC susceptibility, slow relaxation, finite coercivity) directly result from the high defect density.</li> </ul> <h2 class='paper-heading' id='effects-on-electronic-structure-and-band-topology'>5. Effects on Electronic Structure and Band Topology</h2> <p>Mn-Sb site mixing modulates band structure, Weyl topology, and Fermi level:</p> <ul> <li><strong>Band topology</strong>: Defect-free FM MnSb$$_2$Te$_4$hosts Weyl points along$\Gamma$–$Z$; antisite disorder ($x_\mathrm{Mn@Sb}>5\%$$) gaps out the Weyl cone, yielding a trivial insulator ($$E_g\sim0.2$–$0.3$$eV), as shown by DFT+U+SOC and scanning tunneling spectroscopy (<a href="/papers/2007.12217" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Liu et al., 2020</a>, <a href="/papers/2512.21680" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Chen et al., 25 Dec 2025</a>). For Mn(Bi$$_{1-x}$Sb$_x$)$_4$Te$_7$$, increasing Sb drives the system through AFM TI, ferrimagnetic Weyl, and FM axion-insulator phases, contingent on three Mn sublattice occupancies (<a href="/papers/2008.09097" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Hu et al., 2020</a>).</li> <li><strong>Topological invariants</strong>: In the$$k\cdot p$model for MnSb$_2$Te$_4$, the mass term$M(k)=M_0+Bk^2+\lambda\,x_\mathrm{Sb@Mn}-\lambda'\,x_\mathrm{Mn@Sb}$modulates$\mathbb{Z}_2$$index, enabling phase transitions via site mixing-induced band inversion shifts (<a href="/papers/2601.11353" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Leon et al., 16 Jan 2026</a>).</li> <li><strong>Transport properties</strong>: Site mixing correlates with carrier density and mobility. Mn antisites on Sb sites and Te vacancies drive heavy$$p$-type conduction (hole density$p\sim1.8\times10^{20}$cm$^{-3}$, mobility$\mu\sim45$cm$^2$/V$\cdot$$s at 1.5 K) and anomalous Hall effects linked to frustrated magnetism (<a href="/papers/2104.00898" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Li et al., 2021</a>, <a href="/papers/2512.21680" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Chen et al., 25 Dec 2025</a>). In optimized OCVT samples, Shubnikov-de Haas oscillations and large, sign-tunable anomalous Hall responses confirm a Weyl semimetal regime at low antisite density (<a href="/papers/2512.21680" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Chen et al., 25 Dec 2025</a>).</li> </ul> <h2 class='paper-heading' id='site-mixing-in-thin-films-and-symmetry-control'>6. Site Mixing in Thin Films and Symmetry Control</h2> <p>The spatial distribution of antisite disorder can break inversion symmetry. Recent HR-STEM and EDS work reveals an anisotropic distribution$$(\Delta_\mathrm{site}\sim3\%)$between top/bottom Sb planes in bulk MnSb$_2$Te$_4$$, leading to Janus-like reduced-symmetry structures (<a href="/papers/2601.11353" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Leon et al., 16 Jan 2026</a>). This inversion-breaking opens second-order nonlinear susceptibility and piezoelectric tensor components, with implications for magneto-piezoelectric coupling and electric-field–tunable topology.</p> <p>Thin-film growth via <a href="https://www.emergentmind.com/topics/pulsed-laser-deposition-pld" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">pulsed laser deposition</a> (PLD) on Sb$$_2$Te$_3$seed layers can suppress this anisotropy ($\Delta_\mathrm{site}\sim0$$), stabilizing symmetric site mixing. Growth parameters—seed anneal temperature, fluence, quench rate—strongly modulate the final cation distribution (<a href="/papers/2601.11353" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Leon et al., 16 Jan 2026</a>).</p> <h2 class='paper-heading' id='broader-implications-and-material-families'>7. Broader Implications and Material Families</h2> <p>Mn-Sb site mixing is widespread in the broader MnTe(Bi$$_2$Te$_3$)$_n$and Mn(Bi$_{1-x}$Sb$_x$)$_{2,4}$Te$_{4,7}$ families (Liu et al., 2020, Hu et al., 2020). The role of antisite disorder as both a tuning knob (for magnetic exchange, interlayer coupling, and topological phase manipulation) and a source of electronic and magnetic degradation (band gap closure, trivialization of topology, spin-glass formation) is central to materials engineering.

Defect engineering—through synthesis temperature control, precursor ratios, and seed-layer selection—is necessary to maintain low antisite densities in magnetic topological insulators, preserve band inversion, and realize QAHE and field-forced Weyl regimes (Chen et al., 25 Dec 2025, Liu et al., 2020, Hu et al., 2020).

Summary Table: Mn–Sb Site Mixing and Consequences

Property	Low Antisite Density	High Antisite Density
Magnetic Ground State	AFM, QAHE, Weyl semimetal	FM/Ferri, Spin glass, damped magnons
Topological Phase	TI, axion insulator, Weyl	Trivial insulator, gap closure
Transport	High mobility, SdH, QAHE	p-type, heavy holes, suppressed mobility
Synthesis	OCVT, low T, PLD/seed control	Slow anneal, thermal equilibration

A plausible implication is that rational control of site mixing will determine the ultimate feasibility of quantum technologies based on Mn(Sb,Bi)$_2$Te$_4$ derivatives.