With the 2024 presidential election just weeks away, new data from pollsters about the presidential election is seemingly being released every day. In the early period of the election, these polls depicted a variety of scenarios, but are now mostly focused on the general election matchup: the Republican nominee, former President Donald Trump, and the Democratic nominee, Vice President Kamala Harris.
While Harris gained an early momentum in the polls upon replacing President Joe Biden as the Democratic nominee, the race remains a closely contested toss-up, particularly in battleground states like Georgia and Pennsylvania. But the recent back-and-forth on this data begs the question: How accurate are political polls?
As recent years have proved, polling is often, sometimes heavily, incorrect. Case in point: Polling generated by HuffPost on Election Day 2016 concluded that Hillary Clinton had a 98% chance of beating Trump. However, while the polls that year were dramatically wrong, most polling throughout the 2020 election cycle correctly predicted that Joe Biden would defeat Trump. So given that polling accuracy has been on both sides of the coin, how much trust should the public place in polls?
Subscribe to The Week
Escape your echo chamber. Get the facts behind the news, plus analysis from multiple perspectives.
SUBSCRIBE & SAVE
Sign up for The Week’s Free Newsletters
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
From our morning news briefing to a weekly Good News Newsletter, get the best of The Week delivered directly to your inbox.
How are political polls taken?
Polls are used “because posing questions to every member of a community would be costly and logistically difficult,” according to a report from CalTech. However, a “sample size of just 1,000 to 1,500 people can be enough to estimate national opinion in the United States with a high level of accuracy.”
A common method used in polling is random sampling, in which pollsters “draw a random sample of participants from computer-generated lists of landline and cell phone numbers (increasingly, polls are also being conducted online),” CalTech said. This “gives everyone in the population an equal chance of being included or not.” A good poll “needs to be a microcosm of the group” being surveyed, Anthony Salvanto, the director of elections and polling at CBS News, said to CBS. The groups “could be made of voters, Democrats, Republicans, people of different genders, age groups and education levels.”
But “different polling organizations conduct their surveys in quite different ways,” the Pew Research Center said in 2020. Pew noted that CNN and Fox News conducted polls over the phone, while CBS News, Politico and The Associated Press conducted polls using various online techniques.
How accurate are these polls?
Heavy criticism has been levied at polls in recent years, particularly in the last few rounds of presidential elections. Even in 2020, when most polls correctly predicted Biden’s victory, the “national surveys of the 2020 presidential contest were the least accurate in 40 years,” Politico said.
Does this mean that polls just aren’t accurate? Not always, but they can present a different picture than reality. This is largely because “the real margin of error is often about double the one reported,” Pew said. Many polls typically have a margin of error less than 3%, which “leads people to think that polls are more precise than they really are.” But this margin “addresses only one source of potential error: the fact that random samples are likely to differ a little from the population just by chance.”
There are at least three other identifiable sources of data errors that can come from poll taking, Pew added, but most polls don’t calculate these metrics into their margins of error. The differing approaches in how polls are taken can also have “consequences for data quality, as well as accuracy in elections.” As a result, the actual margin of error in most historical polls is closer to 6% or 7%, not 3%, said a 2016 study from The New York Times. This represents an error range of 12 to 14 data points, the Times said.
Nonetheless, polls can still be valuable and paint a widespread picture of Americans’ feelings — and they are still sometimes on the money. Polling during the 2022 midterms was “historically accurate,” FiveThirtyEight reported. This is partially because pollsters began “increasingly weighting surveys based on whom respondents recall voting for in a previous election, in addition to adjusting for standard demographics such as race and age,” the New York Times said.
This method has long been used to calculate polling in other countries, but is only recently gaining widespread usage in the United States. After the 2016 election, it was also found that pollsters underrepresented less-educated voters, which heavily skewed poll results. Since then, pollsters have “adopted education as an additional survey weight, and a cycle of accurate polls in 2018 seemed to reflect a return to normalcy,” the Times said.
Also notable is the percentage difference between the accuracy of polls versus the outcome of an election, which can often skew in opposite directions. Most polls “report a 95% confidence interval,” but the “actual election outcome only lands inside that interval 60% of the time — and that’s just a week before the election,” said Don Moore, a behavioral professor at the University of California, Berkeley. Going further out from an election, the “hit rates fall even harder.”
And while polling can’t determine anything with certainty, it can “provide a nuanced picture of what a country, state or group thinks about both current events and candidates — and how that is changing,” Texas A&M University political science professor Kirby Goidel said. But at the end of the day, there is “far less precision and far more punditry than forecasters admit,” Justin Grimmer, a professor of public policy at Stanford University, said to Politico. Experts “certainly do not know if small fluctuations in the probability of a candidate winning represent anything other than modeling error or meaningless random variation.”