Nonparametric roughness penalties for probability densities

Abstract

SUMMARY Given a number of observations x 1 ..., x N , a nonparametric method is suggested for estimating the entire probability density curve. The method is to subtract a roughness penalty from the log likelihood, where the roughness penalty is a certain functional of the assumed density function f . Those used are linear combinations of ∫ y' 2 dx and ∫ y' 2 dx , where y = √f . The method appears to be consistent under wide conditions, although consistent methods can be rough. Numerical examples are given and show that for certain values of the coefficients in this linear expression the density function turns out to be very smooth even when N is small. Multivariate extensions are proposed, including one to distributions having some continuous and some discrete components, but numerical examples of these have not been tried. Some of the techniques are borrowed from quantum mechanics and tensor calculus.