After another election in which the polls and polling-based forecasts erred pretty badly, you might expect political writers and pundits to have learned their lesson. You might expect them to approach further polling numbers—the ones that come out just after the election, instead of just before it—with a fair degree of skepticism. You would be wrong.
SUBSCRIBE
It took less than 24 hours after voting concluded on November 3 for analysis to start pouring in on how various demographic groups had changed allegiances since 2016. A CNN headline promised to show “how voters shifted during four years of Trump;” one from Vox offered to explain why “Trump made gains with Black voters in some states.” Other outlets offered even more precise parsings of the electorate: “Over half of those whose family income was more than $100,000 a year supported the president,” claimed The Financial Times, “compared with 45 percent in 2016.”
It’s easy to assume that all these data-driven judgments, delivered after Election Day, are somehow epistemologically distinct from the faulty pre-election forecasts. In fact, they are mainly based on a national exit poll conducted by Edison Research, and no more free of systematic bias or methodological ineptitude than, say, a statewide poll of Ohio from October. Exit polls are a lot like regular polls, only worse. That was especially true this year, when capturing voters who had cast ballots early through the mail required calling them up weeks before the election. In a sense, the 2020 exit poll was just another pre-election survey.
Even in normal times, exit polls are plagued with sampling bias, meaning different groups are not all equally likely to respond. This leads to a misleading picture of who actually voted. College graduates and young people, for example, tend to be overrepresented. As the political scientist Robert Griffin notes in The Washington Post, this seems to have led to this year’s exit polls heavily underestimating the share of the electorate representing white people without a college degree—just as the exit polls did in 2016. Compounding the problem is the fact that an exit poll has to match the results of the election; if not enough respondents report voting for Trump, for example, the pollster will have to adjust the input from various groups to bring up Trump’s share of the vote. This often means the groups that are already overrepresented get the biggest adjustments—which might explain this week’s dubious reports that 55 percent of white women broke for Trump.
To be fair, articles like the ones I’m citing tend to include disclaimers about the limitations of exit polls. But then they plow ahead with the analysis anyway, rather like personal-health reporting that is based on a study of a dozen healthy undergrads placed on a two-week diet. The more honorable disclaimer would be to not write the article in the first place. “Exit polls are garbage,” said Lee Drutman, a political scientist and senior fellow at New America. “Any smart person at this point knows not to make any judgments about the electorate from exit polls, because the sampling methodology is just totally off.”
Not all analyses of the electorate have been based on exit polling. The New York Times, for example, has been publishing data visualizations that plot the degree to which every county in certain swing states shifted left or right between 2016 and 2020, along with explanations of what it means for voter behavior. (Example: “Hispanic Voters Deliver a Texas Win for Trump,” a finding based on the fact that counties with higher proportions of Hispanic residents shifted more heavily toward the GOP.) There’s a lot that can be gained by this sort of geographic analysis, but it still has pitfalls. The first is obvious: all the votes aren’t counted yet. As of this writing, a fair number of Texas counties still have counted less than 90 percent of their ballots. If the endless election process we’re living through has taught us anything, it’s that even one or two percent of the ballots can dramatically shift the results. California and New York State appear to have millions of votes left to count. It makes little sense to draw firm conclusions about the Black and Hispanic vote, especially, before hearing from those two states.