We need your help to make sure Turker Nation stays online. Please click the thermometer to learn more.
Note: clicking on the above banners and making ANY purchase returns a commission to Turker Nation.
If you can't see the ad, please click on Shop on Amazon instead. | Want to advertise here? PM Spamgirl to learn more!




Results 1 to 5 of 5

Thread: The Myths of Academic Research Pay: Or why your opinion is wrong

  1. #1
    Community Manager spamgirl's Avatar
    Join Date
    Aug 2011
    Gender
    Female
    Location
    Canada
    Posts
    11,619
    Thanks
    2,460
    Thanked 6,390 Times in 2,072 Posts

    The Myths of Academic Research Pay: Or why your opinion is wrong

    Running this poll on whether ERBs should force researchers to pay at least US federal minimum wage has brought in some interesting comments, and some of them have been clear that they believe my opinion is wrong. I'd like to bring up a few (anonymously) and discuss them one by one.

    "College students are broke, we can't make them pay minimum wage." "Grants are small, we can't make them pay minimum wage."

    The only time in history (that I know of) where such a large group of researchers used a captive labour pool to do research in such a variety of fields was when they thought they could use prisoners for any purpose, even prisoners of war. That means there is currently no precedent for how we, workers, should be treated when we become participants in a study. So, what did researchers do before using mTurk? They had two choices: the first was to pay nothing or just reimburse costs and hope that participants would like the topic of the survey or feel altruistic enough to participate. The second choice, which became more frequent as few were willing to participate for free, was to provide compensation, and if you check out offline surveys you will see they can pay as much or more than $100 for participation. I was recently invited to a study that could only be done offline, and I was paid $90 for one hour. When was the last time an academic survey paid you $90 for one hour of your time? So, if they're broke, how did they pay? Grants, the same grants that these students and professors could be applying for in order to pay us more f Yes, the student would have to slave way at grant applications for a while, but that is actually really good practice for if they plan to get a job in academia. Now instead of getting a grant they put $20 into mTurk to do their survey, no grant needed. If we complain, they plead poverty, but the reality is THE GRANTS ARE OUT THERE and we could be paid fairly, but many of the academics paying peanuts on mTurk are just too lazy to try. And we're letting them be lazy. (I say this, again, as a student at a university who sees this first-hand.)

    The point is that academics were paying fairly or more than fairly before they realized they could use mTurk. Have college students suddenly gotten more broke since then? Have grants shriveled up to such a great extent in three years? Nope, but the excuse that researchers can't afford to pay us continues. And, to be blunt, it's bullshit. I'm an academic researcher, I see the discussions going on around mTurk, and they're not talking about being able to afford to do more research or better research, it's about getting what they would be doing anyway done cheaper. Often the left over money is going to employ students to do even more of the academic's work for them in many cases, meaning the money that should be employing YOU is going to someone else who probably doesn't need it to live. Is that fair?

    On top of that, look at the type of studies you're doing. Are they groundbreaking? Are they changing the world? If not, why should you participate by giving up your labour for slave wages? You aren't actually contributing to the betterment of society by telling them how many lily pads fill up the pond in 60 days. Many of these surveys are fluff crap intended only for publishing and not for societal benefit, so why should we let these people who are working their way to six-figure jobs do so off our discounted labour? The benefit to them is a published paper, which leads to a Master's or PhD or post-doc or professor or tenure position, which means the big bucks, while you have no chance of ever making that sort of money by doing these stupid surveys. In fact, at this point you have no hope of ever making more than a few bucks an hour off them! This is a power imbalance, and by accepting it when we don't have to (which I'll get to later), you're only allowing yourself to be walked upon.

    "If we make this happen, it'll be the end of mTurk."

    Will it? Prolific Academic insists on minimum wage, and while some people abuse the system by posting work that isn't at that level (which they compensate with bonuses), the bulk of their clients adhere to the rules and love the system. There's lots of work over there and more being added by the day. They are *growing*, not shrinking and getting ready to close. It would not be the end of mTurk, it would be the end of unethical academic behaviour. On top of that, if all universities insisted on paying all online participants at least minimum wage, the researchers would have no choice since paying offline participants would still be more expensive. Where I live, offline participants are paid $10 per hour or more, which is 50c higher than US federal minimum wage. They'd still be saving by using mTurk, especially when you factor in the time it takes to get physical space to house people doing surveys, pay people to administer those surveys, advertise for participants, etc. mTurk doesn't just win on cost, it wins on simplicity of accessing a huge participant pool.

    That said, this is the same argument that comes up whenever increases in minimum wage are brought up. I live in Toronto where minimum wage is now $11.25/hour. Our unemployment rate is 6.4%. Companies aren't fleeing, and most researchers won't either if pay rates increase. The fact is that mTurk isn't just about cheap, it's about fast and easy, and even paying minimum wage it's WAY cheaper than lab-based experiments.

    In the end, if 100% of surveys currently pay an average of $3 hour, and 50% of those researchers DO leave, but the rest have to pay $7.25/hour, we'll still make more money. Remember when the fee hike happened and fees were doubled? Researchers initially complained, but guess what? Most of them are back now. Why? They just couldn't afford NOT to use mTurk. And those who didn't come back? They're on Prolific Academic.

    "How can they guarantee minimum wage to all workers?"

    They can't, but if they are told to pay minimum wage to anyone who takes near the average amount of time to complete the study, then even those 10-20% below will get paid very close to minimum wage, and that should be fair. Of course, we know the timer on mTurk is broken, so they'd need to use internal timers on sites like Qualtrics to be sure of how long it took people. This would avoid the problem of holding HITs in the queue to make it seem like it took longer, too. Also, anyone holding HITs would become an outlier, and academics know how to factor those people out of the equation. Academics are smarter than we give them credit for.

    Now, how do they know what to pay up front? Well, they could run the survey and test it for a while, or they could just "top up" the payment with a bonus at the end. Either way, they can make sure to pay minimum wage to the majority of workers doing the study.

    So, how would this affect the surveys themselves? They'd be shorter, more efficient, and better tested and run. The researchers will have to make them shorter, and more efficient, in order to be able to pay less. On top of that, they won't just "throw the spaghetti at the wall and see what sticks", such as putting 10 different psych scales into one study to see if any of them actually prove their hypothesis, which is really bad science. They also wouldn't want to have to run them again and again, so they'd do a better job the first time. They'd avoid errors because paying for errors would become even more costly. They'd also likely stop p-hacking, which is when they add more participants until they get the statistical effect they were looking for, which is rife on mTurk and really, really, really, really bad science (so bad that your paper will not get published if they know you did it!) In all, this would make them do more and better work, which would make them a better researcher. This is a great by-product of forcing minimum wage on them, the fact that research will be cleaner and better run. In the end, forcing them to pay minimum wage will actually help them improve as researchers.

    "This would be hard to enforce, we could never make it happen."

    Ethics Review Boards make or break studies. If they say researchers have to pay minimum wage, they have to pay minimum wage. The good news is they're already starting to have conversations about whether paying LABOURERS less than minimum wage is ethical, I just have to take proof to them that we agree that it isn't.

    "But if we make them pay US participants minimum wage, they'll use non-US participants."

    Actually, they often can't as they only get ethics approval for Americans. That's why so many surveys are US only - learning about people in other regions doesn't help American researchers explain American issues.
    It's all good.

  2. The Following 3 Users Say Thank You to spamgirl For This Useful Post:


  3. #2
    Community Manager spamgirl's Avatar
    Join Date
    Aug 2011
    Gender
    Female
    Location
    Canada
    Posts
    11,619
    Thanks
    2,460
    Thanked 6,390 Times in 2,072 Posts

    "What if minimum wage where the researcher is happens to be higher than federal minimum wage?"

    Then they should pay their local minimum wage or federal minimum wage, whichever is higher. But that's step two of convincing Ethics Review Boards in this process.

    "I'm a Requester and here I am doing your survey that says it's only for workers. We shouldn't see participants as employees because they're volunteering!"

    You might want to look up the definition of volunteering, especially in the eyes of ethics rules. Volunteering means there is no undue pressure convincing us to participate. My grocery bill is undue pressure. My rent is undue pressure. My medical bills are undue pressure. While you might not have a gun to my head, the debt collector does. That is undue pressure to participate in your study, therefore I am not volunteering. This is the same issue that comes up around paying participants in offline studies as well, and to compensate the researchers only offer reimbursement of costs. The same should be done with mTurk, and since the costs are none, there should be no pay. If you want volunteers, DON'T PAY ANYTHING. That is the only way to guarantee that in fact your participants ARE volunteers. If you want participants and aren't getting them by paying nothing, you'll have to pay fairly.

    "But if we just stop doing these underpaying surveys, researchers will raise their prices!"

    It's been 11 years and this hasn't happened. Sometimes you have to protect people from themselves. Those who are so desperate for money that they do the HITs that don't pay enough will only stand to benefit if we take this to IRBs and insist the researchers pay minimum wage, and IRBs will actually force the researchers to do so. We can't ask people to starve their children by boycotting HITs, but we can ask the people who control those paying us to make them pay fairly.

    "We're not employees, so we don't deserve minimum wage."

    Are the only people who deserve minimum wage those who are, by law, entitled to it? Why aren't we also owed fair compensation? 40% of the US workforce is currently not covered by minimum wage laws, imagine what would happen if they all suddenly were told they were going to have to work for $2 per hour just because the law doesn't stop their clients or customers from paying that little. Just because there isn't yet a law mandating something doesn't mean it's not ethically right, and it certainly doesn't mean we should stop fighting for that right.

    I personally straddle a strange line in that I am a worker, a Requester, and a researcher. I see benefit to all of us if minimum wage is applied to study payments. As a worker, I'll be paid fairly for my time, and I'll see surveys of higher quality. As a researcher, I'll be forced to work harder and do a better job, which is good for my discipline. As a Requester, maybe the reputation of mTurk will be improved if people stop being paid slave wages (although convincing Requesters to do so is a totally separate issue) which means I can admit I'm using it and not get flak for it. This can be a win-win-win situation, even for Amazon which would get increased fee income. I think it's worth the battle as a war unfought is one which we'll never know the outcome of.
    It's all good.

  4. The Following 5 Users Say Thank You to spamgirl For This Useful Post:


  5. #3
    "College students are broke, we can't make them pay minimum wage." "Grants are small, we can't make them pay minimum wage."
    I could add something. Why not come to mturk and do some HITs? They can fund their mturk account and use that to pay for their HITs/surveys. Also, they can try to place themselves as turkers. They can do other surveys and see how other researchers set them up. They can experience how efficient a survey is, how worthwhile it is to do one, etc.

    Another thing is whether mturk ever consider lowering fees for those turker-requester accounts - the ones that turk and act as requesters.


    "But if we just stop doing these underpaying surveys, researchers will raise their prices!"
    It could be ongoing and it could be recent. Not just surveys, there has been perpetuation of low pay across many HITs. I've seen some requesters experimenting with different pay to see how long or how high they need to pay. It could also base on how certain HITs have paid like following how other requesters pay for similar type of HITs. Again, I don't know.

  6. The Following 3 Users Say Thank You to garson For This Useful Post:


  7. #4
    Member
    Join Date
    Jun 2012
    Gender
    Female
    Posts
    1,247
    Thanks
    1,152
    Thanked 2,492 Times in 707 Posts

    I think my biggest issue with surveys, esp on mturk, where they know most of us are doing it to earn a living - this is my full time job - in that they will expect you to do 45 minutes of work for pennies. That just tells me that you don't really value my opinion. I spend way more time thinking through surveys when the pay is fair and I won't touch a survey if it's way underpaid.

    But someone, somewhere out there will. So this will never change, I feel. It's been that way since I came to mturk in 2011. Granted, there are surveys that do pay fair and I have even gotten bonuses with a thank you for taking your time or something similar and that is really appreciated.

  8. The Following 2 Users Say Thank You to lolabear For This Useful Post:


  9. #5
    Unless Requesters are forced to pay minimum wage, new mTurkers are forced to work their way up by working underpaid HITs. I'm just inching my way out of that hole myself - and I'm doing it slowly, in part because I don't want to encourage slave wage payers. My pension allows me a (limited) amount of wiggle-room to be a bit picky.

    Not everyone has the cushion I enjoy. So yes, there'll always be a plentiful supply of workers who are desperate to grab pennies.

    Pressure on the Requesters is the only way to go.
    Last edited by Shalynne; 10-03-2016 at 11:02 PM.

  10. The Following 5 Users Say Thank You to Shalynne For This Useful Post:


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •