On Bayesian Generalized Additive Models

Abstract: Generalized additive models (GAMs) provide a way to blend parametric and non-parametric (function approximation) techniques together, making them flexible tools suitable for many modeling problems. For instance, GAMs can be used to introduce flexibility to standard linear regression models, to express "almost linear" behavior for a phenomenon. A need for GAMs often arises also in physical models, where the model given by theory is an approximation of reality, and one wishes to express the coefficients as functions instead of constants. In this paper, we discuss GAMs from the Bayesian perspective, focusing on linear additive models, where the final model can be formulated as a linear-Gaussian system. We discuss Gaussian Processes (GPs) and local basis function approaches for describing the unknown functions in GAMs, and techniques for specifying prior distributions for them, including spatially varying smoothness. GAMs with both univariate and multivariate functions are discussed. Hyperparameter estimation techniques are presented in order to alleviate the tuning problems related to GAM models. Implementations of all the examples discussed in the paper are made available.