Mechanical Turk Qualification Questions
We use qualification questions to present the IRB consent form so
workers only have to see it once, and also to verify that they speak
the language that we are interested in. This is not perfect because
people can always cheat, but it's better than nothing.
We use the Mechanical Turk Command Line Tools (CLT) to set up and
submit custom qualifications to our account. We currently do this
from Erica's laptop since it is done very infrequently.
It is installed under Installs/aws-mturk-clt-1.3.1 and the
qualifications are under qualifications/.
- Look under bin/mturk.properties to make sure that we are
linked with the right account, or add new account information there.
- In qualifications,
run ./makeTemplate.sh qualificationName
- Look at englishTest or turkishTest as examples -- we allow up to 3
or 4 languages spoken since birth but no more, and also our target
language must be selected. This is done by assigning 1 point to each
language, 10 points to our target language, and accepting scores between
10 and 13.
- Run ./createQualification.sh to upload the qualification
to your account. It should then be there in the list of
qualifications when you are creating a HIT.