A Bayesian approach to density estimation

Abstract

Assume a priori that the log density is a sample function from a Gaussian process subject to the condition that the density integrates to one. The posterior distribution given a number of observations is then still a Gaussian process with the same condition and the same covariance function. The mean value function is changed according to a simple formula. This prior may thus be regarded as a conjugate prior for an unknown density. The mode of the posterior distribution is given implicitly by a simple formula, which can be solved numerically. The mode is a close approximation to the optimal estimate with squared error loss in the discrete case. Some examples with data are given.