General P-Splines for Non-Uniform B-Splines

Abstract: We proposed a new penalized B-splines estimator, the general P-spline, to accommodate non-uniform B-splines on unevenly spaced knots. It is a complement to Eilers and Marx's standard P-spline tailored for uniform B-splines on equidistant knots. At its core, we derived a novel general difference penalty that accounts for irregular knot spacing, while still being easy to compute and interpret. Both P-spline variants are useful for practical smoothing, because either one can produce a more satisfactory fit than the other, depending on the knot sequence being used and the data being analyzed. Therefore, practitioners should try out both before betting on either one, for which we have implemented general P-spline in R packages gps and gps.mgcv. The new general P-spline is closely related to O'Sullivan spline (or O-spline) through a sandwich formula that links general difference penalty to derivative penalty. Though both penalties seem equally powerful in wiggliness control for their mathematical association and statistical similarity, simulation studies show that general P-spline either outperforms O-spline in terms of mean squared error, or performs equally well, making it a superior replacement of O-spline.