We need help getting enough donations to pay the monthly server bill. Click the thermometer to learn more.
Note: clicking on the above banners and making ANY purchase returns a commission to Turker Nation.
If you can't see the ad, please click on Shop on Amazon instead. | Want to advertise here? PM Spamgirl to learn more!




Page 1 of 3 123 LastLast
Results 1 to 10 of 21

Thread: requester questions

  1. #1

    requester questions

    Hello,

    I'm a new requester, would it be annoying if I send out emails to notify workers who have pass the qualification test about new HITs?

    Is there anyway I can advertise my HITs? I personally feel the pay is quite generous, we try to set it above US minimum wage.

  2. #2
    Here is the qualification test for the HIT:https://www.mturk.com/mturk/preview?...OLS1YV5BMBQCCJ

    I don't know how to make more people take the qualification test, my goal is to hope as many people pass the test as possible. But still only 1/10 people get the 2/2 multiple choice questions correct. Can anyone help me take a look and point out what's confusing?

  3. #3
    Moderator RippedWarrior's Avatar
    Join Date
    Sep 2011
    Gender
    Male
    Location
    Canada
    Posts
    2,032
    Thanks
    1,539
    Thanked 2,439 Times in 1,018 Posts

    Hi again nopotato,

    The link you posted is for a HIT, and there is no more HITs available. Is your qualification test posted as a HIT? OR is is a test attached to a HIT?

    As for sending a message, that's fine. This is a common practice.

    What is your requester name?
    Eschew obfuscation, espouse elucidation. Batch HITs for newbs. 1000 req'd. 5000 req'd.

  4. The Following 2 Users Say Thank You to RippedWarrior For This Useful Post:


  5. #4
    Quote Originally Posted by RippedWarrior View Post
    Hi again nopotato,

    The link you posted is for a HIT, and there is no more HITs available. Is your qualification test posted as a HIT? OR is is a test attached to a HIT?

    As for sending a message, that's fine. This is a common practice.

    What is your requester name?
    Hi, Here is the new version of the qualification test(posted as HIT) https://www.mturk.com/mturk/preview?...J4OV7P7F3GX7BE.

    The pass rate is around 15-20% now, but still below expectation. Do you have any advice on how to go about improving the test? You're also welcomed to take the test.

  6. #5
    Moderator RippedWarrior's Avatar
    Join Date
    Sep 2011
    Gender
    Male
    Location
    Canada
    Posts
    2,032
    Thanks
    1,539
    Thanked 2,439 Times in 1,018 Posts

    Hi nopotato, I've submitted the test and sent you my worker ID by pm if you wouldn't mind taking a look.

    It took me a long time to read, re-read and comprehend the instructions. I'm not sure how valid my feedback will be until I know if I did it correctly

    Here's my thoughts:

    - I'm sure some workers are put-off by the complexity of the instructions vs. the reward, and give up

    - As a requester myself, I can tell you that for every intelligent, astute and conscientious worker out there, there are 2 that are idiots. I'm not sure how to put that politely. I find there are many workers who don't read or follow instructions very well.

    - Your qualification HIT gives workers 1 shot to learn something new and pass a test with 100% compliance. When a qualification test is used (available using the API only), a retake delay can be set so the worker has a chance to try again. This makes the test into a training opportunity where workers can develop the appropriate skill level.

    The way your task is set up, I would recommend creating a "second chance" HIT for workers that were close but not quite there. You could identify what the error was and offer them a second HIT to try again. This will give them the opportunity to learn from their mistakes and hopefully develop a better understanding of the task.
    Eschew obfuscation, espouse elucidation. Batch HITs for newbs. 1000 req'd. 5000 req'd.

  7. The Following User Says Thank You to RippedWarrior For This Useful Post:


  8. #6
    Moderator RippedWarrior's Avatar
    Join Date
    Sep 2011
    Gender
    Male
    Location
    Canada
    Posts
    2,032
    Thanks
    1,539
    Thanked 2,439 Times in 1,018 Posts

    Title: Qualification Test for Image Captioning Task (tutorial+3 Questions ) | PANDA
    Requester: Stephanie Zhou [A1S6R6WR6HULK8] (Contact)
    (TO): No TO (Submit a new TO rating for this requester)
    Description: Actual Task pays $0.25/sentence. Plan to release 10000 HITs in few weeks. The qualification test contains tutorial section + 2 multiple choice and 1 sentence writing. Please READ instructions and do ALL tutorial questions. Will notify result via email.
    Time: 2 hours
    Hits Available: 1
    Reward: $0.02
    Qualifications: Total approved HITs is not less than 5000, Location is one of: AT, CA, GB, NZ, US, HIT approval rate (%) is not less than 99
    Eschew obfuscation, espouse elucidation. Batch HITs for newbs. 1000 req'd. 5000 req'd.

  9. #7
    Thank you so much for taking the qualification test and giving me feedback.

    It's the fourth version of the qualification test and we do offer a lot of opportunity to retake for everyone(we release a new test every 2-4 days). In the first version of the test, I made 9 multiple choice question and even I got one of the question wrong when taking the test because I fell into my own trap.

    So in the earlier version of the test, I'm quite generous and even allow Turkers who got 5/9 question to pass (the best score is 6/9), and some of them are not doing a satisfactory job in the actual task now. Do you think it's acceptable to ask everyone who took the first two version of the test to re-take the test in order to maintain their qualification?

    Also if you have time, could you tell me which part of the instruction that you find most confusing?

    Again, thanks a lot for your help.

  10. #8
    Moderator RippedWarrior's Avatar
    Join Date
    Sep 2011
    Gender
    Male
    Location
    Canada
    Posts
    2,032
    Thanks
    1,539
    Thanked 2,439 Times in 1,018 Posts

    Quote Originally Posted by nopotato View Post
    and some of them are not doing a satisfactory job in the actual task now. Do you think it's acceptable to ask everyone who took the first two version of the test to re-take the test in order to maintain their qualification?.
    My instinct would be to say to revoke their qualification if they are not doing a satisfactory job. I get the feeling you don't want to go this route because you want the largest pool of workers as possible.

    By all means, you could reduce their qualification score so they can't access the live task, and make them do a remedial HIT.

    Quote Originally Posted by nopotato View Post
    Also if you have time, could you tell me which part of the instruction that you find most confusing?
    I can do this when I get home this evening.
    Eschew obfuscation, espouse elucidation. Batch HITs for newbs. 1000 req'd. 5000 req'd.

  11. The Following User Says Thank You to RippedWarrior For This Useful Post:


  12. #9
    Thank you for the tip.

    One thing that I find frustrating is that people stop doing my HITs after 24hrs. I posted a HIT yesterday and this morning I have 300+/600 HIT left, yet zero people did it since this morning(it has been 8 hours). I've never rejected any HIT from any workers, I reply to all emails and give detailed feedback, I post notification for every HIT, and I think the pay ($0.25/sentence) is reasonable. I couldn't really think of what am I doing wrong to have so many left-over HITs. Is there anything I can do at this point?

  13. #10
    Moderator RippedWarrior's Avatar
    Join Date
    Sep 2011
    Gender
    Male
    Location
    Canada
    Posts
    2,032
    Thanks
    1,539
    Thanked 2,439 Times in 1,018 Posts

    I have a question nopotato. In one HIT, I see 8 images - 7 flashlights and a microphone. Is the microphone significant enough to flag the HIT?
    Eschew obfuscation, espouse elucidation. Batch HITs for newbs. 1000 req'd. 5000 req'd.

Page 1 of 3 123 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •