An article posted by Anouar El Haji on Greenbook this week has ignited a fiery debate in the comments section. The title of the article – “Why surveys cannot be trusted.” To generalize, Anouar positions his argument as ‘people can lie on surveys, and frequently misrepresent themselves, especially on questions that require value judgements or other subjective measurements.’ He claims this risk makes surveys untrustworthy.
Anouar is the founder of Veylinx, a software platform that lets consumers choose between two products to buy (‘vote with their wallets’), which indicates the perceived value of products. Of course his position is a little biased as he sells a product that conceivably does not let respondents lie as they are ‘putting their money where their mouth is’. And of course we’re a little biased when we jump in to defend well-made surveys. But it sure was nice to see the entire comments section vigorously disagreeing with him and defending the unmatched value of the consumer survey.
Within hours of the article being posted, the comments section ignited with researchers around the world. Charles Schillingburg from Insights Solutions stated that “consumer attitudes are relative and not absolutes.” Within a representative sample of known consumers of a brand, “there is no reason to lie in surveys […] as they are trying to help manufacturers of the products they buy.”
“The argument may apply to b-school grads using DIY research tools who think after a course in school that they understand research,” said Victor Crain of Crain & Associates Research. But it doesn’t apply to any of the respected research professionals in the industry.
Nick Tortorello, VP at ICR weighs in with his significant experience: “I can tell you after working in the survey industry for over 40 years, and making it a career, that surveys when done properly are the closest thing we have to a crystal ball and confessional. Yes, some people lie and exaggerate, but most people don’t do these things. […] When surveys have good objective questions, with reliable samples, they produce accurate results and can change the world.”
As we said, it was nice to see the vehement opposition to the idea that surveys cannot be trusted. To cap off Nick’s admission that ‘yes, sometimes people lie and exaggerate’, Brian Lunde states with some finality: “Perhaps it can similarly be said that survey research is the worst method for understanding the reasons for human behavior except for every other method that has been tried.”
So what can we learn from this? Not much that we didn’t already know: well-written and executed surveys get results. In most cases, better results than any other known research method. It’s something market research professionals know and live every day. If you’re a survey geek like us, get in touch!