Branching rules and commuting probabilities for Triangular and Unitriangular matrices

Abstract: This paper concerns the enumeration of simultaneous conjugacy classes of $k$-tuples of commuting matrices in the upper triangular group $GT_n(\mathbf F_q)$ and unitriangular group $UT_m(\mathbf F_q)$ over the finite field $\mathbf F_q$ of odd characteristic. This is done for $n=2,3,4$ and $m=3,4,5$, by computing the branching rules. Further, using the branching matrix thus computed, we explicitly get the commuting probabilities $cp_k$ for $k\leq 5$ in each case.