The Adjoint Projection Pursuit Regression

Abstract

Consider a projection pursuit regression model y = g ( α + β x ) + ε, with an arbitrary monotonic link function g . We assume the link function is unknown; thus we can only estimate the direction of β , that is, the ratios among the components of β , say, β 1 / β 2 . The direction of β is a useful estimand that measures, say, how many units of x 2 is equivalent to one unit of x 1 in terms of potency in affecting the outcome y . Projection pursuit regression usually solves the equation cov( x , y − E ( y | β x )) = 0. As an alternative, we propose to solve the adjoint equation cov( x − E ( x | β x ), y ) = 0. The adjoint equation has the advantage that E ( x | β x ) might be easier to estimate than E ( y | β x ). We establish two main results for the population case. First, β is the unique solution (up to a multiplicative scalar) to the adjoint equation. Second, β satisfies a fixed-point property based on a modified least squares regression derived from the adjoint equation. We apply the population results to two important empirical problems: diagnosis and estimation. For diagnosis, we check whether the linear least squares estimate for β is valid for the direction of β . In particular, we test whether the estimate agrees with either the adjoint equation or the fixed-point property. The tests are analogous to Tukey's 1 df test. For estimation, we propose the adjoint projection pursuit regression estimate, which solves the empirical adjoint equation. The estimate is √ n -consistent for β up to a multiplicative scalar. The rate of convergence is insensitive to the choice of the smoothing parameter in estimating E ( x | β x ); any window size of order O (1/√ n ) or smaller achieves the optimal convergence rate. The estimate can be implemented numerically by iterating the modified least squares regression; a simulation study is given to illustrate this procedure.