Our server bill is due soon and we need your help in paying it. Click the thermometer to learn more.
Note: clicking on the above banners and making ANY purchase returns a commission to Turker Nation.
If you can't see the ad, please click on Shop on Amazon instead. | Want to advertise here? PM Spamgirl to learn more!




Page 2 of 2 FirstFirst 1 2
Results 11 to 13 of 13

Thread: New requester seeking feedback on a series of HITs

  1. #11
    Hi again everyone, and thanks again to everyone who has previously provided feedback,

    I redid the instructions and did some UI tweaks in response to feedback here. It now looks something like this:

    https://s3.us-east-2.amazonaws.com/p...t_threads.html
    https://s3.us-east-2.amazonaws.com/p...l_threads.html

    Unfortunately I'm finding that Turkers are still almost always getting confused about what they're supposed to do. Either they submit the HIT without selecting anything, or they don't submit anything and put it back. Ideally the Turker would select items like what are shown in the examples:

    https://pilazza-turk-resource.s3.us-...threads_6.html
    https://s3.us-east-2.amazonaws.com/p...threads_0.html

    But instead they're selecting nothing. I ran an experiment that asked a bunch of Turkers about the terminology they'd use for the highlighted items, and found that their answers were all over the map. Perhaps the problem is that there's no common vocabulary for labeling the parts of a forum site, and so when I ask them to select a certain header/topic/thread/comment/etc. they don't know what I mean? I was hoping that changing the examples from tiny screenshots to links to pages where everything is filled out correctly would help, but it seems like a significant portion of Turkers don't even look at the examples (in one case the HIT I was testing with was an exact duplicate of an example with everything filled out, and I still got no useful results from it).

    Anyone have any ideas for what might be going wrong, how to clarify the instructions so that they make more sense, or other strategies I could use that would improve accuracy?

  2. #12
    Admin and Forum Support Rosie The Robot's Avatar
    Join Date
    May 2015
    Gender
    Female
    Location
    Serving God in the U.S.A.
    Posts
    5,734
    Thanks
    11,926
    Thanked 11,934 Times in 4,300 Posts

    Tell us about what qualifications you are using for your HITs. We recommend 5 or 10k approved HITs and an approval rating greater than or equal to 99%. We discourage the use of the Masters qual.
    If Turker Nation has helped you make more money than you would have without it, please consider chipping in to help pay the monthly server bill. When you see the big thermometer above the board, please click on it for more info! Thank you!

  3. #13
    Admin and Forum Support Rosie The Robot's Avatar
    Join Date
    May 2015
    Gender
    Female
    Location
    Serving God in the U.S.A.
    Posts
    5,734
    Thanks
    11,926
    Thanked 11,934 Times in 4,300 Posts

    Also, I don't feel that I understand this one. Is this something that they see after the instructions?

    They should be reading the instructions at least once. I recommend you give them a qual test which involves reading the instructions and demonstrating understanding of them. Once they pass it, give them the qual. If they do not do good work, remove the qual. If you want to be really nice, you can let them re-earn the qual by retaking the qual test. If you want to make sure that you get a lot of test takers, you can offer a paid qual test in the form of a HIT and use the mTurk system for retakes.
    If Turker Nation has helped you make more money than you would have without it, please consider chipping in to help pay the monthly server bill. When you see the big thermometer above the board, please click on it for more info! Thank you!

Page 2 of 2 FirstFirst 1 2

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •