A radiative seesaw model in a non-invertible selection rule with the assistance of a non-holomorphic modular $A_4$ symmetry

Abstract: We propose a two-loop neutrino mass model where fermionic and bosonic dark matter (DM) candidates are simultaneously connected to the neutrinos. But the fermionic DM candidate is favored compared to the bosonic one due to generating the fermionic DM mass at one-loop level. In order to obtain our desired Lagrangian and Higgs potential, we introduce a $Z_3$ gauging TY non-invertible fusion rule with the assistance of a non-holomorphic modular $A_4$ symmetry. The fusion rule forbids the mass of DM candidate at tree level but its mass is generated at one-loop level where the DM mass term dynamically violates the fusion rule. After that, the neutrino mass matrix is induced at one-loop level where a remnant $Z_2$ symmetry is still remained. The symmetry assures the stability of our DM candidate. The non-holomorphic modular $A_4$ symmetry plays a role in forbidding the interactions between the SM particles and heavier fermions $X_R$ and an isospin singlet inert scalar boson $S_0$ that run in the DM mass loop, in addition to reduction our free parameters that leads to our predictions for lepton sector. We perform $\chi^2$ numerical analysis for the lepton masses, mixing angles, and phases, and we show several predictions for NH and IH. Then, we demonstrate our lepton flavor violations, muon anomalous magnetic dipole moment, and the relic density of the DM candidate fixing the best fit point of the lepton sector.