Multivariate Discriminant Analysis and Maximum Penalized Likelihood Density Estimation

Journal of the Royal Statistical Society: Series B (Statistical Methodology) · 1995 · Volume 57 · Issue 3

DOI: 10.1111/j.2517-6161.1995.tb02044.x

Abstract

A new theoretical point of view is discussed in the framework of density estimation. The multivariate true density, viewed as a prior or penalizing factor in a Bayesian framework, is modelled by a Gibbs potential. Estimating the density consists in maximizing the posterior. For efficiency of time, we are interested in an approximate estimatorf̂ = Bπof the true densityf, whereBis a stochastic operator andπis the raw histogram. Then, we investigate the discrimination problem, introducing an adaptive bandwidth depending on theknearest neighbours and chosen to optimize the cross-validation criterion. Our final classification algorithm referred to as APML forapproximate penalized maximum likelihoodcompares favourably in terms of error rate and time efficiency with other algorithms tested, including multinormal, nearest neighbour and convex hull classifiers.

Keywords

adaptive bandwidth, cross-validation, em algorithm, gibbs potential, kernel density estimates, nearest neighbours, penalized likelihood, sparse consistency

Enable JavaScript for citation tools and additional paper interactions.