1. Skip to content
  2. Skip to main menu
  3. Skip to more DW sites

Are polls to be trusted?

Michael KniggeJanuary 27, 2016

Not a day goes by without pundits and the media scrutinizing a new US election poll tracking candidates' performances. But a look at past election surveys suggests it is better to take polls with a grain of salt.

https://p.dw.com/p/1HkhD
Wahlkabine USA Wähler
Image: Getty Images/AFP/F.J. Brown

Political polls haven't been very good at predicting the winners of key elections lately.

Recent polls predicting the outcome of parliamentary elections in Britain, Israel and the Greek referendum have one thing in common. They were wrong.

In the UK, polls suggested a tight race between the Conservatives and Labour that could end up in a hung parliament. Instead, David Cameron's Conservative Party won a clear majority.

In Israel, polls predicted a neck-to-neck finish between Prime Minister Benjamin Nethanyahu's Likud Party and the Zionist Party. Instead, Netanyahu ended up the undisputed winner.

In Greece, polls suggested the Greek bailout referendum would be a close call. Instead, Greeks rejected the proposed bailout conditions by a whopping 22 percent margin.

Mixed record

What then do pollsters' failures around the world mean for predictions being made concerning the US presidential primaries and general election?

The recent record in the United States is mixed, explained Charles Franklin, co-founder of pollster.com and the director of the Marquette Law School Poll. "In 2012, the average across all polls indicated the correct winner in every state except Florida, which was very, very close."

But last year, he added, "all the polls were wrong in the Kentucky governor's race, indicating the wrong winner. Yet a few weeks later, the polls were correct in the Louisiana governor's race and very close to the actual vote split."

Low participation

Polls in the United States and elsewhere increasingly face a technological problem that can affect their reliability: The decline of landline phones in favor of mobiles. Traditionally, polls had been conducted mostly via landline phones, particularly in the United States where the person receiving a call also often has to pay part of the call's charges.

With people increasingly switching from landline to mobile phones they become harder to reach as they usually don't show up in pollsters' databases, said Donald F. Kettl, a professor of public policy at the University of Maryland. "Moreover, it's gotten increasingly easy to screen calls, and that's especially the case for mobile phones."

Smartphone Handy Display Hülle Schutzfolie
The rise of mobile phones gives pollsters a headacheImage: Imago

This technological switch, coupled with a growing reluctance to participate in polls, has dramatically reduced the number of people willing to answer pollsters' questions. As a result, according to a 2012 Pew study, the average response or completion rate has fallen to just 9 percent.

"A poll with a 10 percent response rate and no use of follow-up technology is worthless," said Michael Steele, a statistics professor at the University of Pennsylvania.

Charles Franklin who runs the Marquette Law School Poll does not think that a low response rate on its own invalidates a poll. "Low response rate alone does not matter, as long as the reason for non-response is unrelated to political preferences," he said.

What worries Franklin is that polls could be biased against younger and less educated people who tend to be more difficult to reach. That potential bias, he said, can be addressed statistically. But Franklin acknowledged that it would be preferable to have more people from those groups participate in polls.

Not all polls are created equally

These factors make clear that not all polls are created equal when it comes to evaluating the current horse-race nature of US elections.

"To understand a poll, you have to know the exact questions and the exact way in which the sample was drawn," said Steele. "The most useful polls are those that are repeated, so despite whatever ambiguity there may be in the question, one gets some information by comparing time T to T+1."

Predictions of the current presidential primary races face an additional problem, especially on the Republican side, said Kettl.

USA Wahlkampf Republikaner Diskussion der Kandidaten
The large number of candidates makes Republican race hard to gaugeImage: Reuters/R. Hill

"Large numbers of candidates running in small states with polls having small numbers of respondents and many candidates clustered within the range of error," he added.

Asked whether he still trusts polls, Kettl said: "I don't trust the polls. But like everyone else, I follow every one I can find."