Earlier this month, results from a SurveyUSA poll showed that 51% of likely voters in California’s upcoming recall election would vote to remove Governor Gavin Newsom from office. It was the first poll to show a majority favoring his removal and led to a dramatic shift in polling averages.
Before these results, polling averages calculated by the politics and data website FiveThirtyEight showed a 7-point margin favoring keeping Newsom in office. But with SurveyUSA’s data, the margin narrowed to less than one point. As of August 17, the latest date for which we have data, the margin has inched up to 1.2 points, with 48.8% for keeping Newsom in office and 47.6% for removing him.
FiveThirtyEight’s polling averages are perhaps the most sophisticated data-based method of assessing the state of the recall race. The numbers come from a statistical model that aggregates individual poll results into two averages — one for keeping Newsom in office and another for removing him. The website has been producing these averages since mid-July but incorporates polls that go as far back as January 2021.
FiveThirtyEight also tracks averages for whom Californians would choose as a replacement if Newsom is recalled. In their latest numbers, Larry Elder leads with an average of 19%, followed by Kevin Paffrath at 9% and John Cox at 6%. The other 43 candidates on the ballot have averages below 5%.
When creating these averages, FiveThirtyEight doesn’t just take a simple average of all the results. They evaluate the quality of each poll and weight the results accordingly. Data from a high-quality poll can have a large influence on the overall average, while those from a low-quality poll may have little effect on the average.
So what makes a poll high-quality or low-quality? According to Nathaniel Rakich, a senior political analyst at FiveThirtyEight, three key factors go into determining the quality of a poll: the number of people polled (the more people, the better), how recently they were polled (the more recent, the better) and the rating of the pollster, or organization, that conducted the poll.
This rating, which FiveThirtyEight has assigned to roughly 500 pollsters, is based on each pollster’s methodology and historical accuracy. Of the 12 pollsters that have results for the recall election, most have A or B ratings, and just one has a C/D rating.
But an A-rating doesn’t guarantee that a poll is accurate, especially for a special election. In general, the most accurate polls are for presidential elections, and lower-office election polls, like those for the Senate, House and governor, are not as accurate. Special election polls are even less accurate.
Some people may dismiss polling data altogether because of how off some were in 2020. But a FiveThirtyEight analysis found that the overall average error in 2019 to 2020 was only slightly worse than previous years’ averages. And considering the voting changes and general unpredictability brought on by the pandemic, it’s not surprising that survey errors were high. The 2018 midterm polls were actually unusually accurate.
One of the biggest challenges for pollsters is accurately predicting voter turnout. Turnout for special elections is historically low. Since 2010, turnout for nearly all special elections in California has been below 40%. Moreover, election day for this race is on a Tuesday in September, in an odd year — not like the usual first Tuesday in November.
Many pollsters, like SurveyUSA, report findings on “likely voters,” as opposed to registered voters. This categorization is based on survey questions that ask how likely a person is to vote and their voting history. According to registration data from July of this year, 47% of the state’s registered voters are Democrats, while 24% are Republicans. If Newsom can get more of these registered Democrats to vote, that could save him from being recalled.
One reason to expect higher turnout is that all registered voters will automatically receive a ballot in the mail — an extension of the vote-by-mail program from the 2020 presidential election. This contributed to an uncharacteristically large voter turnout in 2020.
“I think that there’s a very wide variance in what you might expect in terms of who is going to turn out in this election. That provides extra challenges for the pollsters,” said Rakich.
Because of these challenges, Rakich notes that we may see larger-than-usual polling errors. He also points out that FiveThirtyEight’s averages are not the same as forecasts, which FiveThirtyEight calculates for presidential and midterm elections. The polling averages do not factor in information like the partisanship of California, the quality of the replacement candidates or the fact that Newsom has raised three times as much money as all of his top Republican challengers combined. If FiveThirtyEight were to forecast the election outcome, these factors might tip the projections more heavily towards Newsom.
Therefore, while FiveThirtyEight’s current polling averages show a close race, it’s not clear whether that will actually play out on election day. “It’s kind of a snapshot in time of just the polls and not anything else,” said Rakich.
Over the next three weeks, Rakich expects several more pollsters to add their data into the mix. You can find those results and see how they affect the overall average on FiveThirtyEight’s page.
For more information on the recall election and how to vote, check out the Chronicle’s ongoing coverage.
Nami Sumida is a San Francisco Chronicle data visualization developer. Email: [email protected] Twitter: @namisumida
