We need your help to make sure Turker Nation stays online. Please click the thermometer to learn more.
Note: clicking on the above banners and making ANY purchase returns a commission to Turker Nation.
If you can't see the ad, please click on Shop on Amazon instead. | Want to advertise here? PM Spamgirl to learn more!




Results 1 to 5 of 5

Thread: Using Attention Checks in Your Surveys May Harm Data Quality

  1. #1
    Community Manager spamgirl's Avatar
    Join Date
    Aug 2011
    Gender
    Female
    Location
    Canada
    Posts
    11,830
    Thanks
    2,472
    Thanked 6,455 Times in 2,088 Posts

    Using Attention Checks in Your Surveys May Harm Data Quality

    Thinking of checking up on respondent attention mid-survey to make sure that you’re getting good data? Think again.

    In this article, we highlight how new findings from our Qualtrics Methodology Lab are helping us to revisit and refine advice that is commonly given to survey researchers, namely the use of attention check questions to ensure data quality. But first some background.

    https://www.qualtrics.com/blog/using...-data-quality/
    It's all good.

  2. The Following 2 Users Say Thank You to spamgirl For This Useful Post:


  3. #2
    Member Rosie The Robot's Avatar
    Join Date
    May 2015
    Gender
    Female
    Location
    Serving God
    Posts
    2,293
    Thanks
    5,902
    Thanked 4,393 Times in 1,647 Posts

    I know that I often second guess my responses and make sure that they are consistent, for fear of rejection. Since I am a psychologically inconsistent person, (randomly super calm or hyperexcitable for example) it impedes my ability to be completely honest.

    Sent from my SM-G900V using Tapatalk
    Holding little Emily up in prayer.

  4. The Following User Says Thank You to Rosie The Robot For This Useful Post:


  5. #3
    Super Sized Member
    Join Date
    Aug 2011
    Gender
    Other
    Posts
    3,378
    Thanks
    2,102
    Thanked 4,394 Times in 1,502 Posts

    Thanks spamgirl

  6. The Following User Says Thank You to taintturk For This Useful Post:


  7. #4
    Member Rosie The Robot's Avatar
    Join Date
    May 2015
    Gender
    Female
    Location
    Serving God
    Posts
    2,293
    Thanks
    5,902
    Thanked 4,393 Times in 1,647 Posts

    There is this really lame attention check that one of the survey companies uses, I've never seen it on mturk. I only mention it because it could have an effect on anyone who has been burned by it, and mess up the data of other surveys. So, this company asks in the survey of we prefer a over b and then asks if we prefer b over a. If you have no preference and honestly answer no to both questions, it tells you that you were not paying attention and boots you.
    Holding little Emily up in prayer.

  8. The Following User Says Thank You to Rosie The Robot For This Useful Post:


  9. #5
    Community Manager spamgirl's Avatar
    Join Date
    Aug 2011
    Gender
    Female
    Location
    Canada
    Posts
    11,830
    Thanks
    2,472
    Thanked 6,455 Times in 2,088 Posts

    Quote Originally Posted by Rosie The Robot View Post
    There is this really lame attention check that one of the survey companies uses, I've never seen it on mturk. I only mention it because it could have an effect on anyone who has been burned by it, and mess up the data of other surveys. So, this company asks in the survey of we prefer a over b and then asks if we prefer b over a. If you have no preference and honestly answer no to both questions, it tells you that you were not paying attention and boots you.
    That's horrible!! What a dumb question.
    It's all good.

  10. The Following User Says Thank You to spamgirl For This Useful Post:


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •