Evaluating probability forecasts
Tze Leung Lai, Shulamith T. Gross, David Bo Shen
The Annals of Statistics · 2011 · Volume 39 · Issue 5
Abstract
Probability forecasts of events are routinely used in climate predictions, in forecasting default probabilities on bank loans or in estimating the probability of a patient’s positive response to treatment. Scoring rules have long been used to assess the efficacy of the forecast probabilities after observing the occurrence, or nonoccurrence, of the predicted events. We develop herein a statistical theory for scoring rules and propose an alternative approach to the evaluation of probability forecasts. This approach uses loss functions relating the predicted to the actual probabilities of the events and applies martingale theory to exploit the temporal structure between the forecast and the subsequent occurrence or nonoccurrence of the event.
Keywords
60G42, 62P05, 62P99, forecasting, loss functions, Martingales, scoring rules
Enable JavaScript for citation tools and additional paper interactions.