I found this article today while researching Mturk. The article was posted by Samuel Bennett on Crowdsourcing.org. The article frequently referred to Houdini guidelines. Please read the entire article as it holds much information about how "we" turkers are. (sarcasm)
http://www.crowdsourcing.org/documen...startups/13258
4. Don’t set HIT prices too low.
Or too high.
One of the ways that new requesters frequently get into trouble is by settingunreasonably low prices for tasks. If you view Mechanical Turk as simply anopportunity to take advantage of the most desperately low-cost labor available,then you are doing it wrong. Resist the urge to try to price every task at one or twocents. A good rule of thumb is to aim for a rate of $4-6/hour for simple tasks (e.g.image moderation, image to text transcription, sentiment analysis). If it takes 30seconds to complete one HIT properly, then a price of $0.05 per HIT works out to a rate $6/hour. Be realistic about the amount of time it takes to complete a singleHIT
–
try completing a few while timing yourself to see how long it takes. Alsoexpect to pay a higher per hour rate for writing tasks or anything that requires ahigher degree of skill.You can also try searching Mechanical Turk for tasks that are similar to what youare looking to do. If you need to have a bunch of images tagged, look for otherimage tagging tasks; if you need to collect contact names from a list of websites,look for other data collection tasks.
The article does hold useful information. I disagree that we are "jealousy" guarding our reputation. Our HIT approval is all we will really have on Mturk and to just disregard it as something so petty as jealousy is ridiculous!answers
–
image moderation, sentiment analysis and most basic classification tasksfit into this category. However writing tasks, transcription, research, translation or other open-ended tasks are not good candidates for this approach.b.
Take advantage of worker qualifications
–
Mechanical Turk also allows you set certain criteria that workers must meet in order to work on your HITs. You can require workers to have previously completed a minimum number of HITs and have achieved a minimum approval rate before being allow to work on your HITs.Although this can be useful on the margins, keep in mind that this is a fairly blunt instrument
–
a worker who has completed several hundred HITs doing tasks like image moderation or tagging wont necessarily be any good at writing or online research or transcription. However, if your tasks are at all dependent on english language skills (e.g. writing, transcription, sentiment analysis), you can frequently capture a noticeable improvement in average quality by limiting the worker location to the United States.Mechanical Turk Qualification Settings.
Be very careful about rejecting work
–
Although Mechanical Turk allows requester to reject any individual work assignment, this power should be used very sparingly. The best workers tend to guard their approval reputation jealously and will avoid requesters known to frequently reject assignments. Try to reserve rejections only for true spammers.
Reply With Quote
Originally Posted by chrism
