COSMOS2020: Exploring the dawn of quenching for massive galaxies at 3 < z < 5 with a new colour selection method

Abstract: We select and characterise a sample of massive (log(M${*}/$M${\odot})>10.6$) quiescent galaxies (QGs) at $3<z\<5$ in the latest COSMOS2020 catalogue. QGs are selected using a new rest-frame colour selection method, based on their probability of belonging to the quiescent group defined by a Gaussian Mixture Model (GMM) trained on rest-frame colours ($NUV-U, U-V, V-J$) of similarly massive galaxies at $2<z\<3$. We calculate the quiescent probability threshold above which a galaxy is classified as quiescent using simulated galaxies from the SHARK semi-analytical model. We find that at $z\geq3$ in SHARK, the GMM/$NUVU-VJ$ method out-performs classical rest-frame $UVJ$ selection and is a viable alternative. We select galaxies as quiescent based on their probability in COSMOS2020 at $3<z\<5$, and compare the selected sample to both $UVJ$ and $NUVrJ$ selected samples. We find that although the new selection matches $UVJ$ and $NUVrJ$ in number, the overlap between colour selections is only $\sim50-80\%$, implying that rest-frame colour commonly used at lower redshifts selections cannot be equivalently used at $z\>3$. We compute median rest-frame SEDs for our sample and find the median quiescent galaxy at $3<z<5$ has a strong Balmer/4000 Angstrom break, and residual $NUV$ flux indicating recent quenching. We find the number densities of the entire quiescent population (including post-starbursts) more than doubles from $3.5\pm2.2\times10^{-6}$ Mpc$^{-3}$ at $4<z<5$ to $1.4\pm0.4\times10^{-5}$ Mpc$^{-3}$ at $3<z<4$, confirming that the onset of massive galaxy quenching occurs as early as $3<z<5$.