How much can you trust political polls?

As every election shows, Americans more and more have sorted themselves into circles that minimize contact between people with differing views and life experiences. Boundaries of race and ethnicity have become more porous, but those of class, education, religion and politics have strengthened and interlocked.

In that environment, polls provide one of the few opportunities to get outside our silos and learn what others think.

But can they be relied on?

Mistrust of polls has a long history, but concerns ramped up after 2016, when polls largely failed to forecast Donald Trump‘s presidential victory. And while surveys did well in predicting the outcome of the 2018 midterm, most polls once again underestimated Trump’s share of the vote in 2020, missing by an average of more than 4 percentage points.

One of the big, unanswered questions is whether the errors in 2016 and 2020 reflect something particular to Trump — supporters who consistently refuse to answer surveys, for example — or a broader decline in accuracy as the share of Americans who respond to polls has declined into the single digits.

The last couple of weeks have provided two new pieces of evidence — one offering a very positive verdict on polling accuracy, the other pointing to more problems. Let’s take a look.

Mixed evidence on accuracy

Truly being able to verify the accuracy of a poll isn’t easy. If a poll finds that slightly more Americans prefer chocolate to vanilla ice cream (YouGov, July 2020), for example, no outside ice cream authority can determine conclusively whether the poll got it right. (Sales figures, for example, might partly reflect marketing efforts or price, as opposed to preference.)

The same goes for political topics. Repeated polls have shown President Biden’s job approval dropping over the last two months. Most recently, a Pew Research Center survey released Thursday showed his net standing with the public moving negative for the first time, with 44% of the public approving and 53% disapproving.

The fact that numerous polls by nonpartisan survey organizations show the same downward trend (although they differ on the share of the public that gives Biden a positive grade) helps bolster the case for accuracy. But, again there’s no objective, outside standard by which to determine whether the polls are right.

The share of the U.S. population that has gotten vaccinated against the coronavirus is a different story. The Centers for Disease Control and Prevention tracks that number as shots go into arms — as of Thursday, 387,821,704 had been delivered, and 76.7% of the adult population (64% of the total population) had received at least one dose. The existence of that sort of data means vaccination rates provide a rare opportunity to check on the accuracy of national polls.

Turns out, they’ve done well.

Pew recently looked at data from 19 survey organizations that conducted 98 public polls from the end of last year through June asking people their vaccine status. The individual polls, on average, came within 2.8 percentage points of the CDC figures, they found. More than 1 in 5 came within 1 point.

Polls that overestimated the rate largely canceled out those that underestimated it, so an average of all the surveys came within 0.3 percentage points of the CDC figure, Pew found.

That’s a lot better than recent election polls have done. But elections pose tricky problems.

Pollsters don’t know exactly who will vote. Often voters don’t know for sure, either. Timing also complicates judgments about accuracy. If a poll taken a week before the election shows a candidate getting 52%, and she ends up getting 56%, does that mean the poll got it wrong, or that the candidate gained ground in the final week?

That was a big issue in California’s recall election. In July, several surveys, including the poll by UC Berkeley’s Institute of Governmental Studies done in cooperation with the Los Angeles Times, showed Gov. Gavin Newsom with only a slight lead among likely voters, largely because a lot of Democrats weren’t sure whether they’d vote.

By late August and early September, those same surveys showed a big shift — Democrats apparently had woken up to the importance of the recall, and Newsom had gained a strong lead, the polls found.

Even so, most polls missed the size of Newsom’s victory by a lot. In the presidential race, polls underestimated the Republican vote. This time, it was the Democratic vote that they missed.

The final average of polls by the FiveThirtyEight.com website found Newsom winning by just under 16 points. The average on the Real Clear Politics website which includes a slightly different mix of polls, came in at 14.5 points. In reality, with about 1 million ballots yet to count out of just over 13 million cast, Newsom’s win stands at almost exactly 25 points.

“That’s huge, it’s way bigger than the polling miss in 2020 or 2016,” FiveThirtyEight’s founder, Nate Silver, said on the site’s recent politics podcast, “but no one cares” — largely because people perceive the polls as having been right when they call the result correctly.

Notably, the surveys done by California’s two most experienced polling operations — the Berkeley poll and the one done by the Public Policy Institute of California — did significantly better than the average.

PPIC’s final poll, finished just over two weeks ahead of the election, showed Newsom winning by 19 points. Berkeley’s final recall poll, conducted a week later, found Newsom winning by 22 points.

The numbers continued to show Newsom gaining ground while those final polls were under way, said Mark DiCamillo, the veteran pollster who oversees the Berkeley survey.

A couple of out-of-state polls, by Emerson College and Suffolk University, both based in Boston, also beat the average, but by and large, polls with less experience in California did worse.

That’s no accident, said Mark Baldassare of the Public Policy Institute who, like DiCamillo, has been polling in California for four decades.

“It’s really challenging to come into a state and do polling” if you’re not familiar with the place and its voters, he said.

The Berkeley and PPIC polls use very different methods. Both, however, poll in both English and Spanish, a step that not all national surveys take, but one which is essential to an accurate result in the state. The Berkeley survey, which benefits from a huge sample — roughly 9,800 voters for the final recall poll — divides the state into eight sub-regions and aims to have a representative sample in each, to get more accurate results, DiCamillo said.

In the 1970s and 1980s, when pollsters could count on getting responses from almost half the people they called, that sort of elaborate weighting of results wasn’t necessary, DiCamillo said.

Today, “you’re not going to get more than 5%” response, he said. “That’s really the difficulty.”

It’s still true that most of the time, especially on national surveys, the best way to get an accurate picture is to look at the averages and avoid obsessing about individual surveys. But the California results provide a reminder that not all polls are equal, and sometimes, especially on state polls, experience may help.

A new border crisis

The Biden administration has been cracking down on Haitian asylum seekers trying to enter the U.S. through Mexico, deporting many back to Haiti and generating an uproar from pro-immigrant parts of the Democratic Party.

Some of the Border Patrol’s tactics, however, have made administration officials uncomfortable. Biden on Friday denounced some of the actions taken at the border, calling them “horrible,” as Chris Megerian reported. “It’s outrageous,” Biden said. “I promise you, those people will pay.”

Molly Hennessy-Fiske and Andrea Castillo report that for many Haitians, color-coded tickets passed out by U.S. officials determine their fate — release into the U.S., detention in a Border Patrol camp or removal.

Civil rights leaders decried the treatment of Haitian migrants at the southern border, Erin Logan reported.

Democrats and young voters

Young voters turned out in force for Democrats in 2018 and 2020. One big issue for the party now is whether they will stick around, Janet Hook reported in the first of three stories on the huge generation of young voters who are reshaping U.S. politics. Biden, the oldest president in American history, is trying to bridge a canyon-sized generation gap, she wrote.

Republicans haven’t given up on young voters, but they’re swimming against a powerful tide, Hook wrote.

The latest from Washington

Talks aimed at finding a bipartisan compromise on police reform have gone nowhere for months, and this week, Democrats officially declared the effort dead. Nolan McCaskill talked with Rep. Karen Bass (D-Los Angeles), one of the lead Democratic negotiators, about why the push for reform legislation failed.

House Democrats plan to vote on Friday on a bill that would write protections for abortion rights into federal law, effectively overturning a host of state antiabortion statutes and regulations.

As Jennifer Haberkorn wrote, the measure is expected to pass the House, but stall in the Senate. Sen. Susan Collins who has campaigned as a supporter of abortion rights, told Haberkorn she won’t support the abortion-rights bill, saying it goes too far.

Federal officials are reshaping their efforts against the COVID-19 pandemic, including reshuffling how they distribute vaccines, as they struggle with resistance in some parts of the country and prepare for booster shots, Chris Megerian and Anita Chabria reported.

On Friday, Biden hailed the decision by the CDC to approve booster shots for Americans 65 and older and others who are especially at risk. “We have the tools to beat COVID-19 if we come together as a country and use the tools that we have,” he said.

Vice President Kamala Harris met Thursday with Indian Prime Minister Narendra Modi in a “coming-of-age” moment for the Indian diaspora, Noah Bierman reported. At the meeting, Harris noted her family ties to India as she gently prodded Modi on human rights.

The Federal Reserve signaled plans to start reversing some stimulus programs, Don Lee reported. Policymakers at the central bank expressed confidence in the economic recovery, with about half indicating that they expect rate increases to begin next year, while the rest pointed to 2023.

The latest from California

Bill Gallaher, a wealthy and politically active developer, spent $1.8 million in an effort to recall Sonoma County Dist. Atty. Jill Ravitch from office. The effort failed spectacularly, getting just 22% of the vote, and it points to the need to reform the state’s recall law to keep it from being abused as a political weapon, Mark Barabak wrote.

Caitlyn Jenner brought fame to her run for California governor, but ended up with less than 1% of the vote. Robin Estrin and Faith Pinho look at why her effort failed.

Scott Lay, a chronicler of California politics with a must-read newsletter, died this week at 48. Seema Mehta’s obituary of Lay explains his impact on state politics.

The state’s new laws that greatly limit single-family zoning in California will make a major change, George Skelton writes.

Sign up for our California Politics newsletter to get the best of The Times’ state politics reporting, including full coverage of the Newsom administration and the latest action in Sacramento.