We typically are NOT getting enough donations to pay the monthly server bill. We need your help! Click the thermometer to learn more.
Note: clicking on the above banners and making ANY purchase returns a commission to Turker Nation.
If you can't see the ad, please click on Shop on Amazon instead. | Want to advertise here? PM Spamgirl to learn more!




Results 1 to 5 of 5

Thread: Using Attention Checks in Your Surveys May Harm Data Quality

  1. #1
    Community Manager spamgirl's Avatar
    Join Date
    Aug 2011
    Gender
    Female
    Location
    Canada
    Posts
    12,276
    Thanks
    2,532
    Thanked 6,633 Times in 2,151 Posts

    Using Attention Checks in Your Surveys May Harm Data Quality

    Thinking of checking up on respondent attention mid-survey to make sure that you’re getting good data? Think again.

    In this article, we highlight how new findings from our Qualtrics Methodology Lab are helping us to revisit and refine advice that is commonly given to survey researchers, namely the use of attention check questions to ensure data quality. But first some background.

    https://www.qualtrics.com/blog/using...-data-quality/
    It's all good.

  2. The Following 2 Users Say Thank You to spamgirl For This Useful Post:


  3. #2
    *** Forum Supporter & Moderator *** Rosie The Robot's Avatar
    Join Date
    May 2015
    Gender
    Female
    Location
    Serving God
    Posts
    2,943
    Thanks
    7,301
    Thanked 5,892 Times in 2,164 Posts

    I know that I often second guess my responses and make sure that they are consistent, for fear of rejection. Since I am a psychologically inconsistent person, (randomly super calm or hyperexcitable for example) it impedes my ability to be completely honest.

    Sent from my SM-G900V using Tapatalk
    If Turker Nation has helped you make more money than you would have without it, please consider chipping in to help pay the monthly server bill. When you see the big thermometer above the board, please click on it for more info! Thank you!

  4. The Following User Says Thank You to Rosie The Robot For This Useful Post:


  5. #3
    Super Sized Member
    Join Date
    Aug 2011
    Gender
    Other
    Posts
    3,379
    Thanks
    2,102
    Thanked 4,396 Times in 1,503 Posts

    Thanks spamgirl

  6. The Following User Says Thank You to taintturk For This Useful Post:


  7. #4
    *** Forum Supporter & Moderator *** Rosie The Robot's Avatar
    Join Date
    May 2015
    Gender
    Female
    Location
    Serving God
    Posts
    2,943
    Thanks
    7,301
    Thanked 5,892 Times in 2,164 Posts

    There is this really lame attention check that one of the survey companies uses, I've never seen it on mturk. I only mention it because it could have an effect on anyone who has been burned by it, and mess up the data of other surveys. So, this company asks in the survey of we prefer a over b and then asks if we prefer b over a. If you have no preference and honestly answer no to both questions, it tells you that you were not paying attention and boots you.
    If Turker Nation has helped you make more money than you would have without it, please consider chipping in to help pay the monthly server bill. When you see the big thermometer above the board, please click on it for more info! Thank you!

  8. The Following User Says Thank You to Rosie The Robot For This Useful Post:


  9. #5
    Community Manager spamgirl's Avatar
    Join Date
    Aug 2011
    Gender
    Female
    Location
    Canada
    Posts
    12,276
    Thanks
    2,532
    Thanked 6,633 Times in 2,151 Posts

    Quote Originally Posted by Rosie The Robot View Post
    There is this really lame attention check that one of the survey companies uses, I've never seen it on mturk. I only mention it because it could have an effect on anyone who has been burned by it, and mess up the data of other surveys. So, this company asks in the survey of we prefer a over b and then asks if we prefer b over a. If you have no preference and honestly answer no to both questions, it tells you that you were not paying attention and boots you.
    That's horrible!! What a dumb question.
    It's all good.

  10. The Following User Says Thank You to spamgirl For This Useful Post:


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •