Panos Ipeirotis (2015) Demographics of Mechanical Turk: Now Live!

Abstract

One of the most common question that I receive is whether I have new data about the demographics of Mechanical Turk workers. The latest data that I had collected were back in 2010, and it was not clear how things have changed since then. The key problem was not that I could not run additional surveys; that would have been trivial. However, the results of the surveys were always changing over time: the aggregate data varied too much across surveys, so I refrained from publishing data that seemed to be unreliable.

So, I thought of how I tackle two problems at once:
- Make it easy for people to see current data about the demographics of Mechanical Turk workers
- Make it easy to understand the inherent variability of the collected data, and potentially understand the source of the variability

For that reason, we built a new site: http://demographics.mturk-tracker.com/ (please also check the API)

The site displays live data about the demographics of the workers, based on a small 5-question survey that users are asked to answer (paying 5 cents for each). To be able to capture the time variability, we post one survey every 15 mins, allowing us to observe changes in the answers over time. We also restrict each worker to be able to answer the survey only once per month.

File

File:Demographics of Mechanical Turk Now Live! (April 2015 edition) A Computer Scientist in a Business School.pdf

Source

A Computer Scientist in a Business School

Enlaces

URL: http://www.behind-the-enemy-lines.com/2015/04/demographics-of-mechanical-turk-now.html

Wayback Machine: http://web.archive.org/web/20170302011416/http://www.behind-the-enemy-lines.com/2015/04/demographics-of-mechanical-turk-now.html