Comments on: Experimenting on Mechanical Turk: 5 How Tos

By: together, in a sense : Using Mechanical Turk for experiments

together, in a sense : Using Mechanical Turk for experiments — Thu, 28 Jan 2010 18:14:52 +0000

[...] Experimenting on Mechanical Turk: 5 How Tos: Another set of tips on Mechanical Turk experiments, from Markus Jakobsson (PARC). [...]

By: Experiments on Mechanical Turk « Algorithmic Game Theory

Experiments on Mechanical Turk « Algorithmic Game Theory — Mon, 07 Dec 2009 14:26:39 +0000

[...] on it. I’ve seen others report on such experiments too. Markus Jacobsson from PARC gives general tips for conducting such human experiments using Mechanical Turk [...]

By: Ryan M

Ryan M — Sun, 01 Nov 2009 01:43:34 +0000

Good answer. Correct, the way to do “negative” qualifications is to define a new qualification in terms of another, with scripting. Cookies <- awesome tip.

Not that it matters, but Amazon AWS almost never refers to the workers as "turkers"; they are "workers"

By: Twitter Trackbacks for Experimenting on Mechanical Turk: 5 How Tos - PARC blog [parc.com] on Topsy.com

Fri, 28 Aug 2009 16:28:54 +0000

[...] link is being shared on Twitter right now. @parcinc said Experimenting on Mechanical Turk — 5 how to’s: [...]

By: Markus Jakobsson

Markus Jakobsson — Thu, 23 Jul 2009 22:17:39 +0000

Greg, thank you for your question. And yes, I am referring to qualifications. To allow people *without* classifications, you need to use a macro, as the MTurk interface does not allow for “negative” demands.

Another way of doing something like this is to use cookies, and disallow people who have the cookie. This will automatically be done if you use a service like SurveyMonkey, where people cannot fill out a survey twice without clearing their cookies in between.

By: Greg Little

Greg Little — Thu, 23 Jul 2009 19:26:23 +0000

This is a technical question:

You say: “It does allow you to assign predicates with each user who performs a task, and later offer HITs only to users who have (or who don’t have) predicates of your choice.”

Are you referring to “qualifications”? If so, how do you prevent workers with a qualification from doing your HIT?