Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information

Abstract

SUMMARY This paper concerns normal approximations to the distribution of the maximum likelihood estimator in one-parameter families. The traditional variance approximation is 1/§, where θ is the maximum likelihood estimator and § is the expected total Fisher information. Many writers, including R. A. Fisher, have argued in favour of the variance estimate 1/ I(x ), where I(x ) is the observed information, i.e. minus the second derivative of the log likelihood function at θ given data x . We give a frequentist justification for preferring 1/ I(x ) to 1/§. The former is shown to approximate the conditional variance of 8 given an appropriate ancillary statistic which to a first approximation is I(x ). The theory may be seen to flow naturally from Fisher's pioneering papers on likelihood estimation. A large number of examples are used to supplement a small amount of theory. Our evidence indicates preference for the likelihood ratio method of obtaining confidence limits.