Evidence of competing ground states between fractional Chern insulator and antiferromagnetism in moiré MoTe2

Abstract: Two-dimensional moire materials present unprecedented opportunities to explore quantum phases of matter arising from the interplay of band topology and strong correlations.One of the most striking examples is the recent observation of fractional quantum anomalous Hall (FQAH) effect in twisted bilayer MoTe$_2$ (tMoTe2) with relatively large twist angles(~3.7deg-3.9deg). The electronic ground states are usually expected to be sensitive to the twist angle, as the twist angle determines the electron bandwidth and correlation strength in the moire system. Here, we report the observation of unexpected competing magnetic ground states in tMoTe2 moire superlattice, on which balance can be tipped by both twist angle and electric field (E). Specifically, we observed anomalous antiferromagnetic (AFM) ground states with zero Hall resistance at both v_h=1 and 2/3, at intermediate twist angles ~3deg. The AFM orders are suppressed by applying vertical E, and emergent ferromagnetism accompanied by integer Chern insulator (ICI) or fractional Chern insulator (FCI) states are observed near the critical E (E_c) of moire superlattice symmetry transition. Our results demonstrate tMoTe2 as a fascinating platform for exploring unexpected correlated phases with nontrivial topology and fractional excitations and point to electric-field-controlled ultralow-power spin-valleytronic devices.