Hypergeometric orthogonal polynomials of Jacobi type

Abstract: Motivated by the theory of hypergeometric orthogonal polynomials, we consider quasi-orthogonal polynomial families - those that are orthogonal with respect to a non-degenerate bilinear form defined by a linear functional - in which the ratio of successive coefficients is given by a rational function $f(u,s)$ which is polynomial in $u$. We call this a family of Jacobi type. Our main result is that, up to rescaling and renormalization, there are only five families of Jacobi type. These are the classical families of Jacobi, Laguerre and Bessel polynomials, and two more one parameter families $E_n^{(c)},F_n^{(c)}$. Each family arises as a specialization of some hypergeometric series. The last two families can also be expressed through Lommel polynomials, and they are orthogonal with respect to a positive measure on $\mathbb{R}$ for $c>0$ and $c>-1$ respectively. We also consider the more general rational families, i.e. quasi-orthogonal families in which the ratio $f(u,s)$ of successive coefficients is allowed to be rational in $u$ as well.