We typically are NOT getting enough donations to pay the monthly server bill. We need your help! Click the thermometer to learn more.
Note: clicking on the above banners and making ANY purchase returns a commission to Turker Nation.
If you can't see the ad, please click on Shop on Amazon instead. | Want to advertise here? PM Spamgirl to learn more!




Results 1 to 3 of 3

Thread: Feedback on HIT wording and pricing

  1. #1
    Requester
    Join Date
    Jun 2017
    Gender
    Male
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Feedback on HIT wording and pricing

    Hi, I'm an artist working with artificial intelligence, and I'm hoping to use Mechanical Turk to generate content for an upcoming piece. I posted my first HIT a couple days ago, and I was disappointed with the results. I'm hoping to get feedback from the community in order improve my HIT.

    My goal is to have turkers describe the movements of performers in a video of a dance performance. I'm looking for basic, simple descriptions like "dancer 1 opens her arms and steps to the left. She runs in a circle and stops." Here is the description of the task I used:

    Watch a three minute video of a dance performance, and describe the movements you see.

    Use simple, clear words.
    As a rule of thumb, you should use at least 300 words, but not more than 500.
    Do not worry about getting every detail, or describing everything that is hapenning. It is enough to focus on one or two dancers at a time.
    Indicate different dancers with "dancer 1" or "dancer 2".
    I wanted to keep it simple, but I think this was too vague. The responses I got were... abstract, and described everything but the actual, simple, physical movements. I accepted the answers anyway, because I figure it was my fault in the wording of the task.

    Do you think it would be sufficient to add something like "Describe the physical movements of the performers in simple, clear terms. For example: 'dancer 1 opens her arms and take three steps to her left.'"? Or do I need to rework the task's wording more heavily?

    More importantly, I only got 2 responses within 1 day. When I did the task myself, it took me roughly 6 minutes to work through a 3 minute video, so I figured $1 for the task would equal $10/hr. (I'd love to pay more, but I'm on a tight budget), and if people could do it faster, it would be more per hour. I set the task time to 30 minutes, to keep people from feeling rushed, but I think if someone is taking that long, they're probably doing more work than is necessary. Is this pricing unfair? Is my estimation of the amount of time totally off base?

    Ultimately, I have 15 3-minute videos that I would like to get maybe 20 different descriptions of (for a total of 300 completed tasks). It seems like it could be kind of fun and creative for people, but I want to be sure that I'm compensating them fairly, and that they have a clear understanding of what I need. Thank you for any advice and guidance you can offer.

  2. #2
    Super Sized Member
    Join Date
    Aug 2011
    Gender
    Other
    Posts
    3,379
    Thanks
    2,102
    Thanked 4,396 Times in 1,503 Posts

    You are doing it yourself so you are going to be faster but the biggest problem that I see is that you are posting a writing task and asking for a minimum of 300 words. Most writers on mturk will not work for less than .01 per word, but it is not just a writing task, it is watching a video while writing. If I am asked to write with no research I can type at around 50 wpm, but if I have to pay attention to something while writing and forced into 3 minutes of video, going back and forth describing actions, it is probably a 10 minute hit.

    Are you using the master qualification? If you are, get rid of it and you will open up the task to more workers.

    Good luck

  3. #3
    Moderator RippedWarrior's Avatar
    Join Date
    Sep 2011
    Gender
    Male
    Location
    Canada
    Posts
    2,030
    Thanks
    1,538
    Thanked 2,427 Times in 1,015 Posts

    Hi ianh, I would enjoy your HIT. This is my kind of writing - no creativity involved

    Your timer is fine. I would even extend that to an hour or more. Workers can accept up to 25 HITs at once, and the timer starts as soon as they accept it. A worker may accept your HIT but have several others to complete before they start yours, so a longer timer is always better.

    Based on what I read above, I would suggest expanding the instructions and definitely provide some examples. I think that would really help.

    To avoid scammers, use qualifications of "Total Approved HITs is not less than 5000" and "HIT Approval Rating is not less than 98%". If you want only English-speaking countries, you can set a locale requirement of "Location is one of: AU, CA, GB, US".
    Eschew obfuscation, espouse elucidation. Batch HITs for newbs. 1000 req'd. 5000 req'd.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •