Tutorial 6: Amazon Mechanical Turk

How can Amazon Mechanical Turk (MTurk) enable you to conduct large-scale experiments on how people perform simple tasks involving perception, learning, or decision making? Or enable you to create a large database of labeled images or videos? In this introduction to MTurk, you will learn about the kinds of tasks for which MTurk is well suited, how to get started on setting up an experiment, the advantages and concerns of conducting experiments on MTurk, some tips for using this tool effectively, and additional resources for learning more about MTurk.

image

Amazon Mechanical Turk (MTurk) is an online crowdsourcing platform that enables researchers to conduct large-scale experiments on the Internet. Many experiments described in this course were conducted on MTurk. (Image ® Amazon.com. All rights reserved. This content is excluded from our Creative Commons license. For more information, see http://ocw.mit.edu/help/faq-fair-use/)

Unit Activities

Useful Background

  • No background is needed for this tutorial.

Tutorial 6.1: Amazon Mechanical Turk


Description: Learn how Amazon Mechnical Turk can be used in large-scale experiments, on simple tasks involving perception or decision-making, or to crowdsource data on large image/video libraries. Discuss the pros and cons of using MTurk, and tips for effective use.

Instructor: Tomer Ullman


Click here for the tutorial transcript

Click here for the tutorial slides

Further Study

Gureckis, T. M., J. Martin, et al. “psiTurk: An Open-Source Framework for Conducting Replicable Behavioral Experiments Online.” Behavioral Research Methods 48, no. 3 (2015): 829–42.

Litman, L., J. Robinson, and T. Abberbock. “TurkPrime.com: A Versatile Crowdsourcing Data Acquisition Platform for the Behavioral Sciences.” Behavioral Research Methods (2016): 1–10.

Also see other online forums for the use of Amazon Mechanical Turk, including Turker Nation, Mturkgrind, and additional tools such as Turkopticon.