What Can Pollsters Learn From Polling Errors: Challenges & Solutions


What Can We Learn From Polling Errors: Challenges & Solutions CATI
Table of Contents

Elections indicate who wins but not why. The 2016- 2020 U.S. presidential elections suffered the worse polling performance in decades and saw the highest number of polling errors. Now, ahead of the 2022 midterm elections, pollsters are “overstating democratic support once again,” says an article by The Wall Street Journal.

It isn’t wrong to say that polling genuinely contributes to democracy. To ask the people, with regularity, for their thoughts strikes us as being both useful and a check on the claims of those in power.

Public opinion polling, done right, remains the best way of obtaining citizens’ opinions. 

However, it does bear some shortcomings, inaccuracy and skepticism being the top ones. Many potential respondents might simply slam down their telephones, and some might quit after two questions. 

Some of their doubts are about pollsters’ methods.

Do they ask the right questions? 

Are they manipulating the wording of questions to get the responses they want? 

And whom did they interview? 

Some doubts are wrapped up in a mistrust of the political parties, marketers, and media giants that pay for the polls. Yet, if you asked whether politicians, business leaders, and journalists should pay attention to the people’s voices, almost everyone would say yes. 

Several challenges to public opinion polling are enfolded in this hypothetical tale. Declining response rates, emerging technologies, and early voting pose more obstacles for even the most responsible pollsters.

In this article, we’ll look at the most common changes to polling and solutions that can help tackle those challenges:

Challenges to Polling

To summarise, we can say the most significant challenges to effective boil down to:

Inaccurate results

Citizens use information shortcuts when making political decisions- with new and personal information driving out the old and impersonal. This can often lead to inaccurate predictions.

For example, most pre-election polls during the 2020 U.S. presidential elections overstated Joe Biden’s lead over Donald Trump in the national vote for president and, in some states, incorrectly indicated that Biden would likely win or that the race would be close when it was not. 

These problems led some commentators to argue that “polling is irrevocably broken,” ” pollsters should be ignored, ” or that “the polling industry is a wreck and should be blown up.” 

With the public lacking fixed preferences on many issues, political actors have ample incentive to supply those shortcuts in ways that might broaden support for themselves and the policies they champion. In a democracy, citizens are typically more concerned with some matters than others, and most citizens are not continuously engaged in public affairs.  

Opinion bias

The public does not have fully formed and unambiguous views on many issues. That does not mean there is anything wrong with the public.

People sometimes answer survey questions just to be polite—because they probably ought to have an opinion. That gives pollsters a lot of running room to “manufacture” opinion, especially on issues of narrow rather than wide concern. Even with strong views, a single polling question rarely captures those views well. 

Opinions that pitch one candidate against another might capture which candidate people prefer but fail to capture the respondent’s willingness to go out and vote, thereby failing to predict whether the opinion will have any real impact.

Take the 2016 U.S. elections, for example; it was a classic case of biased opinion. Pre-election polls and forecasts consistently predicted that Democrat Hillary Clinton would win a resounding victory over Republican Donald Trump. Still, he surprised the world by winning 56.8% of the electoral college vote to capture the presidency. 

Clinton supporters perceived a poll as more credible when it showed her with a comfortable lead than when it showed a neck-in-neck race; the opposite was true for Trump supporters. 

Misleading information

Coupled with inaccuracy and biased opinions, polls often put out misleading information. When interest groups commission pollsters to ask leading questions to gather “scientific” proof that the public agrees with whatever demand they make on the government, they demean polling and mislead the public. 

When analysts, sometimes innocently, use poll numbers as a definitive guide to public opinion, even on issues to which most people have given little thought, they are writing fiction more than citing fact. When political consultants use information gathered through polling and focus groups to camouflage their clients’ controversial policies with soothing, symbol-laden, and misleading rhetoric, they frustrate democratic deliberation.

The Solution

The traditional gold standard of polling is probability sampling, where you contact people selected at random from a list of the population. But probability sampling isn’t so great anymore. 

With response rates in the 10 percent range, there is concern that the select group of people responding to surveys is nothing like a random sample of adult Americans or voters.

A viable solution to these challenges is securing a verified research sample. A group of verified professionals who have agreed to take your survey and complete it with an honest opinion. It is one of the best ways to review your competitive landscape when you don’t want biased opinions, inaccurate results, or low response rates.

With an online panel provider, you can easily: 

  • Manage everything from creating new panels, 
  • Set feedback requirements, 
  • Generate on-demand insights at scale. 

You can get in touch with a research panel provider like Voxco Audience that helps you access high-quality samples with 90+ data & profiling points to accelerate your survey research.

Election polling in closely divided electorates like the U.S. demands a very high degree of precision from polling. Sizable differences in the margin between the candidates can result from small sample composition errors. Changing a small sample share can significantly affect the margin between two candidates.

Another viable approach would be adopting a multimodal way of gathering data

You can contact many people by phone, online, or even face-to-face and then interview those who choose to respond.

Adopting a multimodal way of gathering data offers several advantages, most notably convenience and cost, and the ethical advantage that respondents are not hassled so much to participate. Thereby avoiding biased opinions and inaccurate information. 

Should you still conduct polls?

A simple answer would be “yes, but with the right tools.” Public opinion is an elusive commodity. Attempts to measure it will perforce reveal inconsistency and change. 

Some surveys are more carefully produced than others. Surveys built with online survey software can be useful to marketers and campaign managers who need information fast—and know its limits. 

Switching to a multimodal way of research or securing a verified research sample (as suggested above) can knock the hurdles that stand in the way of polling.

Join the network of 500+ happy survey creators.

Explore all the survey question types
possible on Voxco

Read more

Page header image mobile 07

Modus Research Case Study

MARKET RESEARCH Modus Research Case Study Modus Research is the leading full-service provider of B2B Market Research in Canada. The Modus Research team has seasoned research

Read More »

Primary Research

Unlocking Insights: A Comprehensive Guide to Primary Market Research Try a free Voxco Online sample survey! Unlock your Sample Survey SHARE THE ARTICLE ON In

Read More »
Quantitative Research1

Quantitative Research

Quantitative Research Examples See what question types are possible with a sample survey! Try a Sample Survey In Quantitative research, marketers collect and analyze numerical

Read More »