Rigidity of Hawking mass for surfaces in three manifolds

Abstract: It is well-know that Hawking mass is nonnegative for a stable constant mean curvature ($CMC$) sphere in three manifold of nonnegative scalar curvature. R. Bartnik proposed the rigidity problem of Hawking mass of stable $CMC$ spheres. In this paper, we show partial rigidity results of Hawking mass for stable $CMC$ spheres in asymptotic flat ($AF$) manifolds with nonnegative scalar curvature. If the Hawking mass of a nearly round stable $CMC$ surface vanishes, then the surface must be standard sphere in $\mathbb{R}^3 $ and the interior of the surface is flat. The similar results also hold for asymptotic hyperbolic manifolds. A complete AF manifold has small or large isoperimetric surface with zero Hawking mass must be flat. We will use the mean-field equation and monotonicity of Hawking mass as well as rigidity results of Y. Shi in our proof.