Believe it: the Presidential polls were accurate

Editor’s note: Ruth Whitney is founder and CEO of InVeritas, a global public affairs firm. Opinions, commentary and other essays posted in this space are wholly the view of the author(s). They may not represent the opinion of the owners of Talk Business & Politics.
–––––––––––––––
Regardless of their political views, many Arkansans, like other Americans, were surprised Donald Trump was elected President November 8. They were surprised because the election outcome differed from the expectations news reports had created. A common refrain is people were surprised because “the polls were wrong.”

Before you decide to discount all polls going forward, or decide polls can’t be trusted to help make informed judgments, let’s look at the evidence.

In reality, the national polling was pretty accurate. Reporting about the polls was not. Much of the erroneous reporting was based on a misunderstanding about what polls can and cannot do, a mistaken approach to analyzing poll results, and a mistaken belief that all polls are equal.

National polls are estimates of the national popular vote, so we need to know the national popular vote to evaluate the performance of the polls. At the time of this writing, the votes that have been counted show the following:

More than a million votes still have not been counted, and others may change because of recounts. Most votes not included in these totals will come from provisional votes cast on Election Day and votes cast by mail. Experts believe Clinton’s popular vote margin will grow since most uncounted votes are from states where she did well.

Not all polls are the same. They collect voter responses in different ways and they are conducted at different times.   In addition, the sources that collect polls use different procedures for averaging poll results. For this analysis, we include only polls reported in the last week of the campaign and calculate a simple average of their results using the Huffington Pollster compilation.

Averaged together, the polls of the final week showed Clinton winning more votes than Trump, but as a whole, they somewhat over-estimated her margin (2.1 versus 0.9). The accuracy of the polls varied according to the way interviews were collected. Most of the overall “miss” is due to the inclusion of internet polls. Polls that use live interviewers and polls that use a combination of interactive voice response calls (IVR) and on-line interviews were remarkably accurate.

If the polls were accurate, why were people surprised that Trump was elected?

First, although most polls are of the national popular vote, that’s not how we elect Presidents. Trump did not win the most votes, but he will become President because the states he won have more votes in the Electoral College than the states won by Clinton.

Second, all polls are estimates based on interviewing a sample of voters rather than all voters. Most polls included in our analysis had a sampling (or estimation) error of about 3.0 points. In 95 out of 100 cases, the results of those polls should be within three points (plus or minus) of the results one would obtain by interviewing all likely voters. The candidate estimates provided by the live interview polls were easily within that range.

Third, some of the STATE polls were less accurate than the national polls. This is particularly true of state polls that used internet interviewing. But it is important to note that the Presidential election was exceedingly close in many states; in eight states, the margin will be less than three points. As a result, the difference in the candidates’ actual totals in these states will be closer than the margin of sampling error.

Fourth, polls include some respondents who are undecided or at least say they are. This is even true of polls in the final week of the campaign. But much of the reporting suggests that a candidate LEADING in a poll is being predicted to win. This is not what polls do. Polls estimate the vote at the time they are conducted. If most undecided voters move toward the trailing candidate, it can appear the polls were wrong – even if they are not. From exit polls, we know that late deciders did tend to break Trump’s way – although not enough to help him win the national popular vote.

We use information to make decisions and good polls provide accurate and useful information. If you have a persistent pain, your physician might suggest an x-ray. The fact that x-rays do not show everything is no justification for saying “I don’t believe x-rays.”

The same is true for polls. Polls do not show everything – like predicting what undecided voters will do. But when properly conducted and properly analyzed by trained professionals, the information polls provide remains an important tool in making informed judgments in politics and business.