Penalized maximum likelihood estimation in logistic regression and discrimination

Abstract

Maximum likelihood estimation of the parameters of the binary logistic regression model for pr( H|x ) is discussed with separate discussion of sampling from (i) the conditional distribution of H given x , (ii) the joint distribution of H and x , and (iii) the conditional distribution of x given H . Difficulties associated with continuous x in the latter sampling scheme are discussed. To avoid these, penalized maximum likelihood estimates are introduced, which give estimates of the logistic parameters and a nonparametric spline estimate of the marginal distribution of x . Extensions to multinomial logistic regression are outlined.