Search for nonresonant Higgs boson pair production in the four leptons plus two b jets final state in proton-proton collisions at $\sqrt{s}$ = 13 TeV

Abstract: The first search for nonresonant production of Higgs boson pairs (HH) with one H decaying into four leptons and the other into a pair of b quarks is presented, using proton-proton collisions recorded at a center-of-mass energy of $\sqrt{s}$ = 13 TeV by the CMS experiment. The analyzed data correspond to an integrated luminosity of 138 fb$^{-1}$. A 95% confidence level upper limit of 32.4 is set on the signal strength modifier $\mu$, defined as the ratio of the observed HH production rate in the HH $\to$ ZZ*b$\mathrm{\bar{b}}$ $\to$ 4$\ell$b$\mathrm{\bar{b}}$ decay channel to the standard model expectation. Possible modifications of the H trilinear coupling $\lambda_\text{HHH}$ with respect to the standard model (SM) value are investigated. The coupling modifier $\kappa_{\lambda}$, defined as $\lambda_\text{HHH}$ divided by its SM prediction, is constrained to be within the observed (expected) range -8.8 (-9.8) $<$ $\kappa_{\lambda}$ $<$ 13.4 (15.0) at 95% confidence level.