Eliciting Informative Feedback: The Peer-Prediction Method
Nolan Miller, Paul Resnick, Richard Zeckhauser
Management Science · 2005 · Volume 51 · Issue 9
Abstract
Many recommendation and decision processes depend on eliciting evaluations of opportunities, products, and vendors. A scoring system is devised that induces honest reporting of feedback. Each rater merely reports a signal, and the system applies proper scoring rules to the implied posterior beliefs about another rater’s report. Honest reporting proves to be a Nash equilibrium. The scoring schemes can be scaled to induce appropriate effort by raters and can be extended to handle sequential interaction and continuous signals. We also address a number of practical implementation issues that arise in settings such as academic reviewing and online recommender and reputation systems.
Keywords
proper scoring rules, electronic markets, honest feedback
Enable JavaScript for citation tools and additional paper interactions.