Inferring the pair-instability mass gap from gravitational wave data using flexible models

Abstract: We use hierarchical Bayesian inference with non-parametric Gaussian process models to investigate the effective inspiral spin parameter, $\chi_{\rm eff}$, as a function of primary black hole mass in the third gravitational-wave transient catalog (GWTC-3). Our analysis reveals a transition in the population at a primary mass of $46^{+7}{-5}\,M\odot$. Beyond this mass, the $\chi_{\rm eff}$ distribution broadens, becomes consistent with being symmetric around zero, and has a median of $-0.03^{+0.36}_{-0.59}$ (90\% credibility). These results are consistent with the presence of a pair-instability mass gap that is repopulated by black holes that are the remnant of a previous merger, formed in dense star clusters. However, asymmetric distributions skewed toward positive $\chi_{\rm eff}$ are not excluded by current data. Below the inferred transition mass, we constrain the fraction of second-generation black holes to be $\lesssim 10\%$. These results provide model-independent support for a high-mass and high-spin population of black holes in the data, consistent with earlier work using parametric models. Imminent gravitational-wave data releases will be essential to sharpen constraints on spin symmetry and clarify the origin of the black holes.