‘Nothing is guaranteed’: Berkeley Haas School of Business study finds polls not as accurate as expected

Voting
Josh Kahen/File
Despite the 'media frenzy' over the failure of polls to accurately predict the results of the 2016 election, this study affirms previous journalism that found 2016 was not actually an outlier, according to study co-author Aditya Kotak.

Related Posts

With a presidential election around the corner, election polls that aim to predict the outcome may be less accurate than expected, according to a UC Berkeley study.

The study from Haas School of Business researchers surveyed data from 1,400 polls over 11 election cycles and found that elections polls report 95% confidence but are accurate only 60% of the time.

“With the election next week, this finding is more important than ever,” said recent UC Berkeley alumnus and co-author of the study Aditya Kotak in an email. “Our findings show that polls that are released around now will only capture in its margin of error the true election outcome 60% of the time.”

There are two major takeaways from the findings of the study, according to Kotak. The first is that this research provides a way to adjust interpretation of polls given the time between when they are conducted and when an election will take place.

While election polls report a margin of error, they do not typically take into account time before an election as an influencing factor of that uncertainty, according to Kotak.

Although polls serve as “snapshots of the electorate,” they have a larger amount of error than what they report, Kotak said in the email. The second takeaway from the study offers clarity on the actual level of error and helps people “calibrate” the way they interpret polls, according to Kotak.

The 2020 Democratic primaries in the fall of 2019 inspired Kotak’s study. Given the large amount of time left before the presidential election, he wondered how he should adjust his confidence in the polls. Co-author and campus professor Don Moore’s work on overconfidence also helped to inspire the study, Kotak added.

“I noticed that the margins of error were often cast aside as a footnote and that not much focus was put on them in forecast methodologies,” Kotak said in the email. “I thought it’d be perfect to bring both together and work on this project by looking at poll accuracy over time through the lens of the margin of error, which provides a basis for how overconfident the polls are.”

The study confirmed the team’s hypothesis that polls grow more accurate over time — which Kotak added “makes sense” considering that more people make up their minds the closer it gets to Election Day.

What Kotak found surprising about the study, however, was that they found 2016 presidential polls had around the same accuracy levels as other years. Despite the “media frenzy” over the failure of polls to accurately predict the results of the 2016 election, this study affirms previous journalism that found 2016 was not actually an outlier, Kotak added.

“While there are several other factors affecting the election and more robust models that can handle this uncertainty, it’s helpful to remember nothing is guaranteed,” Kotak said in the email.

Audry Jeong is a research and ideas reporter. Contact her at [email protected] and follow her on Twitter at @audryjng_dc.