Dimension Reduction and Adaptation in Conditional Density Estimation

Abstract

An orthogonal series estimator of the conditional density of a response given a vector of continuous and ordinal/nominal categorical predictors is suggested. The estimator is based on writing a conditional density as a sum of orthogonal projections on all possible subspaces of reduced dimensionality and then estimating each projection via a shrinkage procedure. The shrinkage procedure uses a universal thresholding and a dyadic-blockwise shrinkage for low and high frequencies, respectively. The estimator is data-driven, is adaptive to underlying smoothness of a conditional density, and attains a minimax rate of the mean integrated squared error convergence. Furthermore, if a conditional density depends only on a subgroup of predictors, then the estimator seizes the opportunity and attains a corresponding minimax rate of convergence. The latter property relaxes the notorious “curse of dimensionality.” Moreover, the estimator is fast, because neither projections nor shrinkages are computation-intensive. A numerical study for finite samples and a real example are presented. Our results indicate that the proposed estimation procedure is practical and has a rigorous theoretical justification.