Edge stability and edge quantum criticality in 2D interacting topological insulators

Abstract: Robustness of helical edge states in 2D topological insulators (TI) against strong interactions remains an intriguing issue. Here, by performing the first sign-free quantum Monte Carlo (QMC) simulation of the Kane-Mele-Hubbard-Rashba model which describes an interacting 2D TI with two-particle backscattering on edges, we verify that the gapless helical edge states are robust against a finite range of two-particle backscattering when the Coulomb repulsion is not strong. However, when the Coulomb repulsion is strong enough, the helical edge states can be gapped by infinitesimal two-particle backscattering, resulting in edge magnetic order. We further reveal universal properties of the magnetic edge quantum critical point (EQCP). At magnetic domain walls on edges, we find that a fractionalized charge of e/2 emerges. Implications of our results to recent transport experiments in the InAs/GaSb quantum well, which is a 2D TI with strong interactions, will also be discussed.