The New York Times recently shone a light on the polling industry and how two of the most important trend lines in the industry may be about to cross.
As online polling rises, and the use of telephone surveys declines due mostly to rising costs and diminishing response rates, there are concerns among public opinion researchers that web polling will overtake phone polling before online polling is truly ready to handle the responsibility.
Ready or not, online polling is here and political analysts are facing a deluge of data from new firms employing promising (but not always proven) methodologies. The wide margin of difference in poll results for Donald Trump is noticeable – he fares better in online polls than in traditional polls. But it’s still not clear which method is capturing the true public opinion.
The abundance of web-based polling reflects how much easier it has become over the past decade. But big challenges remain. Random sampling is at the heart of scientific polling, and there’s not yet any way to randomly reach people on the Internet as auto-dialers allow for telephone polls. Internet pollsters obtain their samples through other, decidedly non-random, means. To compensate, they sometimes rely on extensive statistical modeling.
The online pollsters may get grouped together because they’re using the same channel, but they’re using a far greater diversity of methodology than phone pollsters. The statistical techniques that the Internet pollsters then use to adjust these data vary nearly as much. And because no one is yet sure of the ‘right’ methods, that variance in methodology is a risk.
There is one big reason why online polling (and other non-personal polling methods like IVR) numbers are so varied: Anonymity. Voters are likelier to acknowledge their support of a given candidate in an anonymous online survey than in an interview with a real person. Research suggests that the social acceptability of an opinion shapes a respondent’s willingness to divulge it.
This social acceptability bias may be making online surveys more accurate. But we don’t know. What is also possibly adding to the problem is that the online polls have biased, non-random and non-representative samples. The fact that online pollsters have sometimes relied on extensive modeling to adjust their results could be a sign of the limitations of online polling.
The current web-based approach to polling are high-profile experiments into the next generation of opinion research, and the industry is watching closely.