“Keep on Turkin’”?

Abstract

Over the past decade, there has been tremendous growth in academic research use of online samples, coupled with a shift away from traditional sample groups (especially undergraduate college students). In particular, researchers are turning to crowdsourcing platforms like Mechanical Turk (MTurk) to obtain convenient and inexpensive samples. Launched by Amazon in 2005, MTurk is “an online crowdsourcing labor market” (Levay et al.2016). In their review of work published over four years in theJournal of Consumer Research(volumes 39–42), Goodman and Paolacci (2017) observed that 27% of all surveys and experiments in the journal were conducted using MTurk. Even more strikingly, this proportion of MTurk-based empirical work grew from 9% in the first of the four years they reviewed to 43% in the final year. Similar growth has been observed in other social science fields, including (Chandler and Paolacci2017, p. 500) “more than 40% of papers published in theJournal of Personality and Social Psychologyand more than 20%” inPsychological Science.