Modelling and smoothing parameter estimation with multiple quadratic penalties

Journal of the Royal Statistical Society: Series B (Statistical Methodology) · 2000 · Volume 62 · Issue 2

Abstract

Penalized likelihood methods provide a range of practical modelling tools, including spline smoothing, generalized additive models and variants of ridge regression. Selecting the correct weights for penalties is a critical part of using these methods and in the single-penalty case the analyst has several well-founded techniques to choose from. However, many modelling problems suggest a formulation employing multiple penalties, and here general methodology is lacking. A wide family of models with multiple penalties can be fitted to data by iterative solution of the generalized ridge regression problem minimize ||W1/2(Xp−y) ||2ρ+Σi=1mθip′Sip(pis a parameter vector,Xa design matrix,Sia non-negative definite coefficient matrix defining theith penalty with associated smoothing parameter θi,Wa diagonal weight matrix,ya vector of data or pseudodata and ρ an ‘overall’ smoothing parameter included for computational efficiency). This paper shows how smoothing parameter selection can be performedefficientlyby applying generalized cross-validation to this problem and how this allows non-linear, generalized linear and linear models to be fitted using multiple penalties, substantially increasing the scope of penalized modelling methods. Examples of non-linear modelling, generalized additive modelling and anisotropic smoothing are given.

Keywords

Generalized additive models, Generalized cross-validation, Generalized ridge regression, Model selection, Multiple smoothing parameters, Non-linear modelling, Penalized likelihood, Penalized regresion splines

Enable JavaScript for citation tools and additional paper interactions.