We need your help to make sure Turker Nation stays online. Please click the thermometer to learn more.
Note: clicking on the above banners and making ANY purchase returns a commission to Turker Nation.
If you can't see the ad, please click on Shop on Amazon instead. | Want to advertise here? PM Spamgirl to learn more!



Results 1 to 1 of 1

Thread: How to block past workers from doing new batches of HITs (API & GUI instructions)

  1. #1
    Community Manager spamgirl's Avatar
    Join Date
    Aug 2011
    Gender
    Female
    Location
    Canada
    Posts
    11,076
    Thanks
    2,448
    Thanked 6,326 Times in 2,055 Posts

    How to block past workers from doing new batches of HITs (API & GUI instructions)

    If you are looking to allow people from multiple countries to do your HITs by using the "In" comparator, please click here to get to the right post.

    -*-

    These instructions allow you to block people who have worked on your HITs in the past from doing new batches of HITs. The reason for this could be that you have a survey that you plan to repost and don't want past participants to take it again, or you want fresh eyes on your work so you're looking for a new group of people to complete the latest iteration of your HITs. Whatever your reasoning, here are the instructions:

    Through the API: https://github.com/leeper/MTurkR/wik...ions-as-Blocks

    Through the RUI/GUI:

    1. Create a Qualification

    Go here.

    Click on .

    Friendly Name: I'd recommend using your requester name, date and HIT type. For example, Spamgirl 12.8.13 Block Past Workers
    Description: Describe the work this qualification entitles the Worker to do. For example, Workers with this qualification have already done my HITs before, so I'm using this qualification instead of blocking.

    2. Assign Qualifications

    Go here.

    Click on the first Worker ID you'd like to give the qual to. Click .

    Check the box next to your new Qualification, then click on .

    Repeat for every Worker on your list.

    3. Create your HITs

    Go here.

    Choose a template of your choice. Note that you cannot use your qualification with the Categorization or Sentiment templates as they don't allow you to change your qualifications, they force you to use the expensive Masters qualification. For categorization or sentiment HITs, use the Other custom template instead. Click Create Project.

    The only settings which matter towards using your qualification are found when you click the Advanced >> link at the bottom right corner of the big gray area. Next, click the Worker Requirements >> link.

    Worker Requirements: Choose Customize worker requirements.

    Click Remove twice. On the last row, click Clear.

    In the first drop down box, choose your qualification under Qualification Types you have created.
    In the second drop down box, choose has not been granted.

    Require qualification for preview: Yes.

    You will get a warning when you try to advance to the next page telling you that not using Masters is bad. Ignore this - it's only bad for Amazon's wallet.

    Everything else is up to you.

    Repeat step 2 every time your HITs are completed as you will need to give the qualification to all new people who do your HITs. Be sure to do this before reposting your HITs to ensure they can't retake it.

    -*-

    Why this is your best bet:

    1) If a worker has to accept a HIT and then submit their ID, or try to remember if they've done it before (trust me, they won't), they'll get upset and your Turk Opticon rating will suffer.

    2) Asking a worker to email you to find out if they've done it before is time consuming and more than any Turker will want to bother with for one HIT.

    3) Putting a PUBLIC list of worker IDs online and asking Turkers to see if their ID is there IS A BIG NO-NO! This exposes their ID and all associated data, such as their wish list and any demographic information listed there, to the world. If you take this route, reputation damage will be swift and severe. On top of that, websites aren't 100% reliable, so they may not even be able to access the list.

    4) Blocks put workers' accounts in jeopardy of suspension, so using them will totally ruin your reputation. Here is the hard block email text they receive when you block them:

    Greetings from Mechanical Turk. We regret to inform you that you were blocked from working on HITs by the following requester(s): (That's YOU)

    Requesters typically block Workers who submit poor quality work.

    Requesters rely on Mechanical Turk for high quality work results. In order to maintain quality, we continuously monitor service activity. Should additional Requesters block you in the future, we may suspend your account. Please ensure your work quality is at a high standard. We encourage you to read the HIT instructions and follow them carefully.

    We realize that this block may be a onetime occurrence for you. Should you maintain high work quality with no further complaints for the next few months we will dismiss this event.

    Regards, The Mechanical Turk Team

    -*-

    If your survey will only be posted once, the best way to ensure Turkers only complete it once is to post a single HIT and specify how many workers you want to do it. For example, 1 HIT, 100 Turkers. This can be done through the GUI, Command Line Tools and API.

    You can also use Qualtrics and other third-party services to do the work for you, but that will cost you money.

    The best, easiest and most effective way to ensure no retakes is a qualification, though, so check out the link at the beginning of this post!
    Last edited by spamgirl; 08-20-2014 at 05:10 PM.
    It's all good.

  2. The Following 4 Users Say Thank You to spamgirl For This Useful Post:


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •