Search for a W' boson decaying to a vector-like quark and a top or bottom quark in the all-jets final state at $\sqrt{s} =$ 13 TeV

Abstract: A search is presented for a heavy W' boson resonance decaying to a B or T vector-like quark and a t or a b quark, respectively. The analysis is performed using proton-proton collisions collected with the CMS detector at the LHC. The data correspond to an integrated luminosity of 138 fb$^{-1}$ at a center-of-mass energy of 13 TeV. Both decay channels result in a signature with a t quark, a Higgs or Z boson, and a b quark, each produced with a significant Lorentz boost. The all-hadronic decays of the Higgs or Z boson and of the t quark are selected using jet substructure techniques to reduce standard model backgrounds, resulting in a distinct three-jet W' boson decay signature. No significant deviation in data with respect to the standard model background prediction is observed. Upper limits are set at 95% confidence level on the product of the W' boson cross section and the final state branching fraction. A W' boson with a mass below 3.1 TeV is excluded, given the benchmark model assumption of democratic branching fractions. In addition, limits are set based on generalizations of these assumptions. These are the most sensitive limits to date for this final state.