The exponential growth of low-cost, incentivized surveys being regularly pushed onto consumers in the US has been having a significant effect on response rates…and respondent quality. While this push-supply invitation method is continuing to provide an important sample source at a time where respondent supply is a major worry, the surge in popularity of ‘offerwall’ survey delivery has been opening discussion around both the frequency of respondent participation and the quality of survey results.
Sure, quarantine periods exist to protect respondent experience and to help filter out pro survey takers, but upholding these quarantine periods is becoming more challenging as the chain has become more convoluted, resulting in respondents being able to take part in as many surveys as they like on a frequency of their choosing.
All this has led to a new concern that didn’t exist when quarantine periods were conceptualized: it opens doors for pro survey takers where financial gain is the main objective (BTW we still maintain that incentives are important!). This can lead to the final insights being based on the opinions of people who take surveys very regularly. The growing reliance on what are generally accepted to be ‘non-traditional’ sample sources also comes at a time when there is heightened scrutiny on the quality of online samples.
Of course, it’s difficult to state a right or wrong in the case of push supply and potential over use of respondents, as it will be of concern to some in the industry more than others – but we like to think that the researchers who choose Voxco software are generally more interested in getting higher quality results.
Reg Baker (among others) has called for a review of sampling practices, re-stating the importance of an underlying science behind sampling. So in order to help, Mike Misel at Greenbook has offered these five tips to consider around using non-traditional sample sources:
- Know the source. Expect source transparency regarding how and from where the respondents are recruited.
- Understand the incentive model (and its potential impact on sampling and end results).
- Understand respondent flow from the supply source origin to the survey and every other step along the way.
- Enquire about your supply source profiling and targeting capabilities. eg. can you recontact the same respondent in the future?
- Consider and account for potential biases which exist with any sample source or methodology, adjust sample-frame designs as needed.