Must read for turkers! Guideline for requester pay.

[TinQC]

I found this article today while researching Mturk. The article was posted by Samuel Bennett on Crowdsourcing.org. The article frequently referred to Houdini guidelines. Please read the entire article as it holds much information about how "we" turkers are. (sarcasm)

http://www.crowdsourcing.org/document/5-mechanical-turk-tips-for-startups/13258


4. Don’t set HIT prices too low.
Or too high.
One of the ways that new requesters frequently get into trouble is by settingunreasonably low prices for tasks. If you view Mechanical Turk as simply anopportunity to take advantage of the most desperately low-cost labor available,then you are doing it wrong. Resist the urge to try to price every task at one or twocents. A good rule of thumb is to aim for a rate of $4-6/hour for simple tasks (e.g.image moderation, image to text transcription, sentiment analysis). If it takes 30seconds to complete one HIT properly, then a price of $0.05 per HIT works out to a rate $6/hour. Be realistic about the amount of time it takes to complete a singleHIT
–
try completing a few while timing yourself to see how long it takes. Alsoexpect to pay a higher per hour rate for writing tasks or anything that requires ahigher degree of skill.You can also try searching Mechanical Turk for tasks that are similar to what youare looking to do. If you need to have a bunch of images tagged, look for otherimage tagging tasks; if you need to collect contact names from a list of websites,look for other data collection tasks.




answers
–
image moderation, sentiment analysis and most basic classification tasksfit into this category. However writing tasks, transcription, research, translation or other open-ended tasks are not good candidates for this approach.b.
Take advantage of worker qualifications

–
Mechanical Turk also allows you set certain criteria that workers must meet in order to work on your HITs. You can require workers to have previously completed a minimum number of HITs and have achieved a minimum approval rate before being allow to work on your HITs.Although this can be useful on the margins, keep in mind that this is a fairly blunt instrument
–
a worker who has completed several hundred HITs doing tasks like image moderation or tagging wont necessarily be any good at writing or online research or transcription. However, if your tasks are at all dependent on english language skills (e.g. writing, transcription, sentiment analysis), you can frequently capture a noticeable improvement in average quality by limiting the worker location to the United States.Mechanical Turk Qualification Settings.
Be very careful about rejecting work

–
Although Mechanical Turk allows requester to reject any individual work assignment, this power should be used very sparingly. The best workers tend to guard their approval reputation jealously and will avoid requesters known to frequently reject assignments. Try to reserve rejections only for true spammers.

The article does hold useful information. I disagree that we are "jealousy" guarding our reputation. Our HIT approval is all we will really have on Mturk and to just disregard it as something so petty as jealousy is ridiculous!

[spamgirl]

Dumbasses. They can take their $6/hour and enjoy the douchebag scammers who do their HITs because no one good will touch them. They make millions off the backs of us. Fuck them.

Here's a tip: If YOU wouldn't do the HIT for the pay you're offering, then why the fuck do you expect me to?

[Jessann]

$6 an hour is actually better than the majority of requester's pay. I laughed a little when I read that though because that is how much I made at my first job when I was 15 years old working in an amusement park.:facepalm:

[chrism]

Tin, I'll admit it! I do guard my approval reputation jealously and will avoid requesters known to frequently reject assignments. :D

He could have chosen a better word, but he got his point across. Rejection rates matter to those of us that care about our work.

As for the suggestion to price $4-$6 an hour? Just perpetuates the 'digital sweatshop' instead of a global mutually beneficial workforce. Pathetic.

[Hollie - Closed]

$4-6, how very generous of them....

[katmar]

While $6/hr is better than the majority of requester's pay, it doesn't make it right and it's absolute BS. And this jerk suggests $6 as the top amount to pay. Samuel Bennett can suck it.

[sleepstar - Closed]

are they referring to Houdini the requester? their pay is absolute dogshit and i've never completed even one of their tasks

[TinQC]

I'm only assuming it it. The name was mentioned a few times during the article.. as in here at Houdini.

[chrism]

This article was contributed to the site by SB - it came from Houdini's site.

Here is the original location. Please remove if not appropriate to post the link.

http://houdiniapi.com/2012/04/five-mechanical-turk-tips-for-startups/?utm_source=bit.ly&utm_medium=blog&utm_campaign=general_campaign

[TinQC]

Tin, I'll admit it! I do guard my approval reputation jealously and will avoid requesters known to frequently reject assignments. :D

He could have chosen a better word, but he got his point across. Rejection rates matter to those of us that care about our work.

As for the suggestion to price $4-$6 an hour? Just perpetuates the 'digital sweatshop' instead of a global mutually beneficial workforce. Pathetic.

Yes we all covet our % rate but for them to say it's jealousy is absurd. It makes us sound like children pissed off cause someone got a pink Power Ranger instead of a blue ( bad analogy but you know what I'm saying). I do care extremely about my work, but I'm not jealous of others with a rating higher than I :)

[chrism]

TinQC, I understand what you mean.

Guess it did not rub me the wrong way as much because I was too distracted by the advice to set rates below (US) minimum hourly rates.:(

I looked at Houdini's mark-up and well, the rest of the article just seemed so benign in comparison. :(


SEO Text Generator (150 words)
Generate unique, seo-optimized text about a keyword (150 words). All articles are proofread to ensure correct spelling and grammar.
$2.500 per article

PG-13 Image Moderation
Flags images inappropriate to be shown to children under the age of 13. This Blueprint provides a more sensitive level of moderation than our standard moderation criteria.
2.5¢ per image

Business Information Extractor
Capture and collect company information (name, address, homepage, email) from a list of websites.
50.0¢ per business listing

Handwriting Converter
Convert handwriting notes and documents into text
40.0¢ per task

Human Sentiment Analysis (Coming Soon)
Perform human-moderated sentiment analysis of user-provided input text.
6.5¢ per task

[TinQC]

Those numbers seem to be great but it's not what the article said :) I respect what you're saying but... saying that those of us who do the other types of hits are deserving less than min. wage (here in the US) is pretty silly. I mean an hour is an hour and in the hour I work I click my ass off and I just find it offensive that they do not think I'm worth min. wage.

[snickers495]

I was just reading a few old articles about mechanical turk and all the comments criticize how pathetic and terrible the system and the turkers are to work for pennies. I was so peeved at the articles, then something like this comes along and reenforces the criticism. This makes Mechanical turk look like an exploitation system, with the requesters just trying to work out the bare minimum to pay us. F them.

[Jazz]

Yes we all covet our % rate but for them to say it's jealousy is absurd. It makes us sound like children pissed off cause someone got a pink Power Ranger instead of a blue ( bad analogy but you know what I'm saying). I do care extremely about my work, but I'm not jealous of others with a rating higher than I :)

I wonder if they were going for "zealously" and landed on the wrong word. Jealously is absurd.

[TinQC]

I wonder if they were going for "zealously" and landed on the wrong word. Jealously is absurd. Must be because the word they used is just not right. Yhe word jealousy is what really rubbed me the wrong way

[sparky32]

Must be because the word they used is just not right. Yhe word jealousy is what really rubbed me the wrong way

It's actually a proper use of the word jealously (adverb). Different from the word jealousy (noun).

jeal‧ous‧ly
1 if you jealously guard or protect something, you try very hard to keep or protect it:
a jealously guarded secret

[TinQC]

I was thinking of the word in the sense of a a noun. Thank you for making that clarification!

[Jebediah]

Makes me wish there was a more advanced turkopticon that helped people measure the average pay rate per time. For example take pay rate of the hit and the time it takes to complete average to complete such classification hit, then automatically use that math to calculate average per hour for type of hit from requester. It would take honest people or the submission end though, some fake numbers could ruin the system. But it would be a way to have an easy referral list for potential income from requester on a classification of hit. Of course some would go into advanced categories such as writing would have subcategories such as transcription and proofreading but you folk's likely get the gist... wonder how hard it would be to make something like that.

[rosewater]

Don't forget taxes are going up 50% next year- ss going up from 4.2 to 6.4 and federal going up for everyone that makes less than $8700.

[qtkitty]

Since the workers are not the requesters formal employees ...mTurk is sort of acting like an PEO and the workers in this case are 1099 employees .. aka we are not employees of either but rather independent contractors. Basically neither are being held liable.

Although, I think it would be best practice to supply minimum wage. My thinking yes they have to pay 10% commission to mTurk, but they are not paying the brick and mortar location for the minimum wage employee to report to, computer, supplies, administration costs, utilities, Unemployment taxes ect. At this point changing that would be difficult, because people will continue to do the hits.

[abfabuk]

However, if your tasks are at all dependent on english language skills (e.g. writing, transcription, sentiment analysis), you can frequently capture a noticeable improvement in average quality by limiting the worker location to the United States.


This is kind of insulting to those of us from England! >:(

I would like to think our English language skills are of a reasonable quality as well! ;)

[avidlcfan]

This is kind of insulting to those of us from England! >:(

I would like to think our English language skills are of a reasonable quality as well! ;)

I agree with you, even though you are fascinated with the letter "u".

[taintturk]

If you noticed one of the two reference sources at the bottom of the article, on the houdini site, one was this - http://blog.crowdflower.com/2010/05/gambit-worker-survey/

So this writer took some if his ideas from an article written by CROWDFLOWER on a survey of gambit users, NOT mturk users.

There is a huge difference between the two. NOBODY IN THE WORLD uses gambit for a full time job. It is impossible. The pay on sites like inboxdollars, or swagbucks is ridiculously low. So when asking users why they use gambit, of course the answers are because they are bored, or better than playing video games, etc.
They do not know any better. They swallowed the blue pill, and are stuck in the matrix, while people of turkernation have taken the red pill and see how the world is. They say that there are tasks that are not suited for Mturk because they have no idea that there is a quality workforce but you have to pay fairly to access it.

Now the second reference article - http://people.cs.umass.edu/~wallach/workshops/nips2010css/papers/wais.pdf
There is no mention at all what the pay rate was. All it talked about was finding the 79 qualified workers out of 4800 roughly.

Then you look into all of the sources for that article. http://www.ics.uci.edu/~jwross/pubs/RossEtAl-WhoAreTheCrowdworkers-altCHI2010.pdf

It just goes on and on and on... flawed research written by people who are avoiding the main point.
You will never get a majority of the good workers for any survey or experiment if you do not pay fairly.

Their results are based on the casual and inexpirenced turkers. Not the writers and professional turkers on TN.
They are getting the TO, not the TN.

---------- Post added at 09:26 AM ---------- Previous post was at 09:23 AM ----------




Here's a tip: If YOU wouldn't do the HIT for the pay you're offering, then why the fuck do you expect me to?


+1000, spot on.

[valdp]

This article was contributed to the site by SB - it came from Houdini's site.

Here is the original location. Please remove if not appropriate to post the link.

http://houdiniapi.com/2012/04/five-mechanical-turk-tips-for-startups/?utm_source=bit.ly&utm_medium=blog&utm_campaign=general_campaign

I just looked on this, and there is a link to this forum in the comments to see what the turkers feel about this. Whoever put there on there, good job!

[taintturk]

I just looked on this, and there is a link to this forum in the comments to see what the turkers feel about this. Whoever put there on there, good job!

Yep, kudos to them.

[74Turker]

Whoever the Kevin Peter fellow is, pretty much summed up my feelings about the article. It would be nice, but I doubt the kind of people who would suggest $4-6 an hour would give a crap enough to take a look at the turkernation forum to hear about our opinions. Though, I'm hoping they'd take at least a quick look and see spamgirl's message!

[taintturk]

You never know. Quite often requesters come here and ask for advice before they publish hits.

[A2-D2]

They're both WordPress sites. You can comment anonymously. I just wrote a rebuttal and posted it to both sites.

CORRECTION: You can comment anonymously on Crowdsourcing.org, but not Houdini's site. The latter makes you enter a valid email address. I should have used my Gmail one, not my Yahoo one, which is tied to my WordPress and Gravatar accounts. So much for anonymity...

[74Turker]

You never know. Quite often requesters come here and ask for advice before they publish hits.

You're right, I just tend to err on the side of pessimism, lol. I can see new requesters taking a look, but the ones that wrote the article and would agree with it... I have doubts.

[Noliah]

"Jealously" guard it? This is obviously written by someone who has to pay out for writing. I like to think that I'm "wisely" guarding my rejection rating. Protecting it. Who else will if I don't? I have one rejection, and the helpful hint the requester posted helped soften the blow. They're not all out to take advantage of us, but the ones who are will hopefully have to eat a boatload of Karma one day. :shitstorm:

[taintturk]

What is funny is that Houdini is a crowdsourcing noob. It did not ring a bell till I saw some hits posted today for 0.00
Probably some guys who think they can get people to pay for them designing mechanical turk hits.
They post its under houdini inc. and every HIT I have ever seen has SHIT pay. They have a HOF/S thread with a positive, but it seems people only rated thumbs up for quick pay, not the actual pay. http://turkernation.com/showthread.php?6127-Houdini-Inc.&highlight=houdini

---------- Post added at 03:33 PM ---------- Previous post was at 03:21 PM ----------

geez, reading their site is just making me fucking cringe.

In this link they tell requesters that using majority rules is the best practice. taken from their blog http://houdiniapi.com/blog/ mar 22 http://ontolo.com/blog/LBHTD-How-to-Use-Mechanical-Turk-for-Link-Building

Pricing sentiment analysis at .065 per to their customers using majority rules, they would have to be paying a penny per hit to pay amazon four times, pay the worker shit pay, and then make a penny of profit per sentiment.
Just ridiculous. New company trying to underpay the underpaid for providing a service that with a little knowledge can be done for free.

I have to stop reading this trash and quit following links, it is just depressing what people think they can get away with.

[genoa]

This morning I went to the link provided for this article intending to leave some kind of rebuttal. The more I thought about it, though, I felt I could not improve on any of the raw and colorful posts already left on TN. :)

I wasn't sure if outsiders could get to this part of the forum, but went ahead and left the link. It was, also, an unspoken invitation for requesters to join TN.

In the past, I have advocated that responding to articles like this is a good way of getting our message across about fair pay.

The comments left on Houdini were very good and I'm glad several took the time to post them. I was a little surprised that Houdini let them remain, but the day's not over yet.

In addition to the pay issue I was also taken aback by the article's recommendation to restrict HIT's to the US if good English was needed. Even though our Canadian neighbors have a problem with proper spelling :) I thought the author was very very disrespectful to them (and those in England, Australia, etc.)

[taintturk]

I just saw a classify sentiment post by Houdini inc.

The pay was .05 for classifying 10 freaking tweets!!!

PATHETIC. They can put their hits where the sun don't shine.

[houdini]

Hello,

Since I am the one responsible for the article that spawned this post, I wanted to come here and "face the music" in person. Thanks for those who stopped by our website to share their thoughts on the topic. I think it is valuable for anyone reading our article to get some perspective from workers about what they consider to be fair pay, so we appreciate hearing from those who took the time to speak out constructively. We'll be leaving those posts up on our blog, along with the link back to this post for others to see.

Before I explain our perspective behind the original article, I wanted to note that we have since revised our post based on the feedback from this thread. We have eliminated the section that referred to an hourly wage of $4-6. Its clear that I bungled this pretty badly. I mentioned $4-6 as point of reference for unskilled or low skilled task like image moderation while suggesting a "higher" rate skilled tasks like writing or editing. In reality we often end up paying over $15/hour for writing tasks, but because did not mention this I made it seems like most tasks should be paid out near the low end of the range. I have taken this part out entirely and replaced it instead with a link to a thread here on turker nation where workers discuss the hourly rates that they have personally experienced. The new paragraph reads as follows (changes in bold):

"One of the ways that new requesters frequently get into trouble is by setting unreasonably low prices for tasks. If you view Mechanical Turk as simply an opportunity to take advantage of the most desperately low-cost labor available, then you are doing it wrong. Resist the urge to try to price every task at one or two cents. Take a look at http://turkernation.com/showthread.php?3247-How-much-money-is-everybody-currently-averaging-an-hour to get a sense of the range of typical hourly rates. You can compute a rough hourly wage by estimating the time it takes to complete a given task. For example, if it takes 30 seconds to complete one HIT properly, then a price of $0.10 per HIT works out to a rate $12/hour. Be realistic about the amount of time it takes to complete a single HIT – try completing a few while timing yourself to see how long it takes. Also expect to pay a higher per hour rate for writing tasks or anything that requires a higher degree of skill."

I have also changed the sentence later in the article that states: "The best workers tend to guard their approval reputation jealously and will avoid requesters known to frequently reject assignments." to "The best workers tend to guard their approval reputation zealously..." Although I only intended to point out that good workers will tend to steer clear of requesters known to reject a lot of HITs, some thought that the word "jealously" made turkers sound petty. One commenter suggested that "zealously" would have been a better word, and I tend to agree.

Back to the original article - Houdini is a small company (only 2 people) that helps other companies run jobs on Mechanical Turk. So we manage a range of different jobs under our requester name, from content creation, to online data collection, to simple image moderation. We often get requests from people who approach us to run ridiculously low priced jobs - as low as $1-2/hour. We usually dismiss those jobs out of hand, but we will take some jobs that end up working out to $4-6/hour range for non-skilled tasks are fairly easy to do. Taking a look at this thread: http://turkernation.com/showthread.php?3247-How-much-money-is-everybody-currently-averaging-an-hour/page7 you can see that most workers tend to be making an average of $4 -5/hour for low skilled tasks like image moderation or categorization, or else they are doing more skilled tasks like writing/edits HITs and making around $10-14/hour. In short, the hourly rate tends to lie across a range - usually in direct proportion to the difficulty and skill level required to complete the task.

This is pretty the same range for us - we end up paying about $6/hour for image moderation or simple data collection, to about $8 for more skilled online research, up to $14-16/hour for writing and editing HITs. Of course we set the HIT price before we know how long task will take to complete on average, so we don't always know what the hourly rate will turn out to be until after we have already run a bunch of HITs.

Now, an hourly rate of $4-6/hour is certainly not a kings ransom for an adult living in the U.S. who is earning that amount as their primary wage. However there are also lots of turkers who are either a) individuals from foreign countries with a much lower cost of living, b) students or homemakers only looking to make a little extra cash in their spare time or c) workers who look for very simple tasks, albeit lower paying tasks, that they can complete while watch tv or doing some other work. A recent study showed that over 25% of regular turkers completed HITs while working at another job: http://blog.microtask.com/2012/02/happily-ever-after-how-bored-workers-and-their-bosses-can-benefit-from-crowdsourcing/. $4-6 is not too bad when you are adding it on top of your regular pay!

In contrast, many of regulars here on Turker Nation are practically "professional" turkers - people who rely on the income as part of their livelihood and consequently spend a large amount of time on Mechanical Turk building up their reputation. We fully expect that many of you will not be interested in $4-6/hour HITS no matter how easy they are - it simply wouldn't be worth your time. We would expect anyone like that would naturally stay clear from any of these lower priced HITs. We continue to go back and forth whether we should continue to accept jobs at the low end of the range, but we feel like it is fair to provide some HITs that are at the low end of the scale for those workers who don't have the experience or the skills to do higher paying writing or editing tasks. These workers may not be the "best" worker on mTurk (at least not yet) but this provides a natural progression for new turkers to start with easier/lower paying taking ad moving up to more skilled/higher paying tasks.

Anyway, I just wanted to share our views on the issue. Although we can't promise that we'll never run another $5/hour HIT, we'll certainly keep these comments in mind and try to do better in the future. For anyone else that want to share their thoughts, feel free to leave a comment on our blog or to email us directly at admin@houdiniapi.com. Thanks for taking the time to listen.

[johnsmith]

Having read only your response here Houdini, and not the original post yet, I'd say if the information you stated is accurate, that's pretty fair reasoning.

[spamgirl]

Back to the original article - Houdini is a small company (only 2 people)

From http://houdiniapi.com/

We Are Chris Conley
Muhammad At-Tauhidi
Brian Auton
Jameson Detweiler

You might want to update your website then.


Although we can't promise that we'll never run another $5/hour HIT, we'll certainly keep these comments in mind and try to do better in the future.

Again, the issue wasn't what YOU do, but that the article said "Don't pay more than $4-6 per hour or you're paying too much!" The damage is done, requesters have read that and believe it's the standard. YOU told the world that's all a Turker is worth. That's bullshit.

As to what you do, you just won't get the best people working on your HITs if you pay $5. Did you know in Canada $10.75 is minimum wage? If only mTurk was in Canada, anything that paid less would be illegal. I could get more at fucking McDonald's what what you're paying on "simple" HITs. Glad I'm here. Won't touch your HITs with a ten-foot pole, but you don't care because some "individuals from foreign countries" will do a shitty ass job for you instead.

I dunno why anyone gets so upset about requester pay anymore... we know they're all greedy and selfish, who on Earth isn't anymore? But telling OTHERS to be greedy and selfish, THAT is what went over the line.

[taintturk]

I turned the focus to "them" in my post spamgirl.

Houdini,

Thank you for joining and your response and revisions. I also agree with the comments from spamgirl above.

On turkernation you will find that many people here have approval rates from 99.5%-99.9%, many of us are not casual turkers, many of us rely on this income and take a considerable amount of pride in the work we do.
Reading on your website, the issue of using majority rules or "plurality" really stuck. It is not only a waste of money and resources, you are instantly eliminating the best workers from your tasks, because honestly, we will not work on those hits. http://mechanicalturk.typepad.com/blog/2012/01/tbd.html
They are usually under paid because the questions are asked multiple times and no one wants to have their work rejected because someone else is watching tv and clicking buttons randomly.

If your intention is to be a crowdsourcing company in competition with the big boys, then you have to do your own research with qualifications, workers, and clients and achieve a balance with all three without making one of the three suffer.
None of us think that requesters should not make money, nor the crowdsourcing company they work with. By the time the requester pays, amazon gets paid, the crowdsourcing company gets paid, multiple workers paid to review the same crap over and over, it all equates to .005 per sentiment. There is a better way.

[Project 2501]

What I am reading from Houdini, they do not even read the demographic surveys that have been released about workers on Mturk. The last report that was released stated the 50% of the workforce held bachelor’s degrees. You are undervaluing the Mturk workforce with your blog post.

http://turkernation.com/showthread.php?7951-Crowdsourcing-Industry-Report

Telling other requesters to limit their pay is saying you do not wish to complete for workers. Too bad, competition for the good workers already exists and it looks like you are losing. Competition exists for workers not only on Mturk but at many other work at home places that pay better than you. I cannot list them because it is against the rules. To say it is okay to pay students and work at home moms only 4 to 6 dollars a hour smacks to me that we are WORTHLESS. Well, I have never touched one of your hits and I never will. I can only hope you will always be late with the projects you run on Mturk.

[spamgirl]

Reading on your website, the issue of using majority rules or "plurality" really stuck. It is not only a waste of money and resources, you are instantly eliminating the best workers from your tasks, because honestly, we will not work on those hits. http://mechanicalturk.typepad.com/blog/2012/01/tbd.html
They are usually under paid because the questions are asked multiple times and no one wants to have their work rejected because someone else is watching tv and clicking buttons randomly.

It's sad when a company considers themselves experts when they don't have the first clue as to what they're talking about. If you don't really know what you're saying, don't write an authoritative bullshit article on the topic.

If requesters want honest advice, they should ignore other requesters and ask GOOD Turkers. We're the ones who know what we're willing to do.

We're so lucky to have the choice not to work for requesters who devalue us and what we are capable of!

[chrism]

Thank you for engaging directly in this discussion. And for making changes to the article.

What I would ask is that you do a bit of soul-searching and think about how (as a US Company) you reached the conclusion that $4-$6 an hour was an acceptable wage for ANY level of work. You and your company are in a position to influence others in how micro-working develops as an industry.

What do you see going forward? A global workforce that is the equivalent of a 'digital sweatshop'? I see the potential for a global workforce that is mutually beneficial to workers and requesters. You save money by not hiring employees or using temp agencies. Workers have reduced barriers to entry to find paying work.

This could be a great step forward for all of us. The potential of crowd-sourcing is still developing. Many workers here at TN are working hard to do their part of the equation and develop their skills. Your part is to treat mTurkers as you would want to be treated at your job. With fairness and respect.

ok..stepping off my soapbox...:D

[hypeline]

While I don't agree with everything you said, I appreciate the fact that you took the time to come here and post. Having an open and honest dialogue is the first step in making the industry better for both workers and requesters.

Having said that, there are a few points that I would like to mention. First, like Chrism said, even if requesters pay minimum wage or higher, Mturk still allows them save a significant amount of money over hiring employees or using contract labor through a temp agency. The typical costs associated with having employees such as taxes, workers comp, health benefits, office or workspace costs, equipment costs and recruiting expenses don't apply in the crowdsourcing environment.

While this may be good news for requesters, it is bad news for workers since many of those costs are passed on to them. Self employment tax, health insurance, the cost of buying and maintaining equipment, etc. all can quickly eat away at a workers income. Because of this, it is even more important for requesters to make sure they are paying fair wages.

Another area where requesters benefit is by only paying for the actual work that gets done - not by the hour. That means that workers don't get paid for the time they spend logging in to their computer, reading HIT instructions or taking short breaks to use the restroom. Again, good for the requester, bad for the worker.

Part of what makes articles like the one in question so offensive is that even with all of the substantial savings listed above, they still advocate paying wages that are downright insulting to even the most unskilled workers.

I'd also like to point out that it is not the 1950's. Most stay-at-home moms (and dads) that I know are highly educated and extremely hard working. Same goes for college students and workers in other countries. Implying that they should be paid less is not only a disservice to their intelligence, but it also perpetuates false stereotypes.

In the future when potential clients approach you with low paying jobs, perhaps you could have an open discussion with them about the countless ways that crowdsourcing is saving them money. That may make it easier for them to see that even by paying fair wages they are still getting a great deal.

Finally, as a company that acts as somewhat of an intermediary between requesters and workers, I would think it would behoove you to only work with clients who are willing to pay appropriate wages. It creates a win-win-win situation for everyone. Workers are paid fairly for their time and talent, your commission increases and the requesters get better results by attracting the best workers in the industry.

[Shyaamist - closed]

I have to put in my :2cents:. I didn't do these hits before cause I thought they were asking for way to much info for way to little money. But unless things change, I will be boycotting these for sure now!

[panyi42]

So glad I read this thread, super insightful. will DEFINITELY think twice about accepting low paying HITS in the future out of the respect for other hardworking Turkers.

[taintturk]

So glad I read this thread, super insightful. will DEFINITELY think twice about accepting low paying HITS in the future out of the respect for other hardworking Turkers.

Experience with turking and this forum will help you tremendously in being able to skip hits in that pay range.

[chrism]

Another article continuing the discussion ... :)

http://www.crowdsourcing.org/blog/the-hidden-online-workforce-at-amazon-mechanical-turk-/13432

[Project 2501]

The direct link to the article.

http://caroleteeonline.blogspot.com/2012/04/hidden-online-workforce-at-amazon.html


The Hidden Online Workforce at Amazon Mechanical Turk


In light of a recent article I read on the web (I won't pay it homage by providing a link) regarding how much to pay the on demand workforce that frequents Amazon Mechanical Turk, I find it necessary to offer a different view for wanna-be requesters. In the article I just read, yet another requester suggests that the available online workforce be paid no more than 6.00 per hour. The author also stated that 4.00 an hour is okay depending on the task.

By offering low wages, you're never going to have access to these talented turkers. They are a highly intelligent, full-time online workforce with a lot of experience and stellar work ethics - and they won't work for next to nothing. You'll never get to use them if you don't treat them right.

Offer a fair wage, and watch your HITs disappear faster than Houdini could shake a straightjacket under water. Pay a low wage and your HITs could sit for a while. Even if they don't, you're certainly going to experience low-quality results from inexperienced workers and complete rubbish from those trying to game the system. How's that going to help your bottom line?

At the bottom of Carol Tee's blog is where you can post comments. I see nobody has posted, yet.

[genoa]

The direct link to the article.

http://caroleteeonline.blogspot.com/2012/04/hidden-online-workforce-at-amazon.html



At the bottom of Carol Tee's blog is where you can post comments. I see nobody has posted, yet.

Very cute how she snuck Houdini's name in the article without actually accusing him/her.

[Shyaamist - closed]

Another article continuing the discussion ... :)

http://www.crowdsourcing.org/blog/the-hidden-online-workforce-at-amazon-mechanical-turk-/13432


I TOTALLY agree with this article! Its sad the amount of talent these requesters are missing out on because they are not willing to pay for it!

[Jebediah]

True you get what you pay for, but many still try to go for the cheapest available. For instance the chinese knock-off market lives and lives well, despite the product's rarely last as long as their basis model's. And yet folks buy these product's all the same in the prospect of saving a minor amount. Do they consider the amount they could have saved in product lifetime? Or consider the possible safety consideration's of buying the cheapest product available? Nah probably not just trying to save money. And I could see how the habit would become easy enough with less inscrutable requesters. They saved a buck now, who cares if the results are not the same quality? And new requester's would likely not even realize the difference that rates make in actually getting turkers who are focused and not turking while there is a commercial break, hoping the their hit is done before "C.S.I." comes back on. (For example at least.)

[BoomMike - Banned]

There was a university that posted their guidelines for putting studies up on MTurk, here is the PDF

http://iris.uwaterloo.ca/ethics/human/application/sops/UWHSOP%20606%20-%20Using%20Mechanical%20Turk.pdf

One of the most interesting things I found is that when giving time estimates, they must include the time to read to consent form and debriefing in their time estimate. So the stuff that must Turkers probably skip over... it makes me feel a lot better taking a 15 minute long University study for $1 and seeing a massively long consent form at the beginning.

[Hollie - Closed]

From the article that BoomMike posted:

As of July 2011, typical Amazon Mechanical Turk remuneration is $.50 for a 15 to 20 minute study or $.25 for a study less than 15 minutes. C. automatically sets remuneration at $.50. If you wish to increase the remuneration above $.50 (e.g., you have a 30 to 45 minute study and you wish to provide $2 remuneration), you need to ask C. to manually override the “reward” field and set the remuneration to the amount you request. It is important to keep in mind that an amount too low may attract too few participants while an amount too high (in comparison to similar HITs) may attract participants who are wishing to complete HITs strictly for the remuneration.

That is insane. They have automatic price ranges and only if I requester knows better do they get increased. No wonder C. pays so badly. (Edited out requester name and put C. Anyone that reads the article will know what I am referring to)

[SunnyD]

Now, I know why I'm paid so little. I'd question the $6 an hour goal, but the truth is that that I make that amount maximum. I know others are making more, so I must be doing something wrong. Anyway, I would be happy if most requesters shot for $6 an hour. I could accept it sadly enough.