Consider this the “don’t get cocky” moment for Democrats high-fiving over polling out of Wisconsin, Florida, and Ohio where their Senate candidates appear to be doing better than they probably are.
The issue of survey research bias has been present since the inception of public opinion polls but the issue has gotten worse and worse every year. Whether it’s the issue of landline phones versus cell phones or certain demographics of voters simply being less inclined or more inclined to answer the phone for a pollster, it’s a very imperfect science often used to drive the news rather than reflect actual opinion.
In recent years, polls coupled with election “forecasts” have created algorithms to spit out numbers such as this farce in 2016 courtesy of Nate Silver’s FiveThirtyEight:
The underlying problem, of course, was state-level polls over-estimating Hillary Clinton’s chances.
Writing at The Upshot, part of the New York Times data-analysis team, Nate Cohn (another data nerd named Nate) is looking at recent polling trends and seeing similar warning signs to those of 2020 when polling was off by several percentage points and over-estimated Democratic vote share by close to double-digits in some cases:
Wisconsin is a good example. On paper, the Republican senator Ron Johnson ought to be favored to win re-election. The FiveThirtyEight fundamentals index, for instance, makes him a two-point favorite. Instead, the polls have exceeded the wildest expectations of Democrats. The state’s gold-standard Marquette Law School survey even showed the Democrat Mandela Barnes leading Mr. Johnson by seven percentage points.
But in this case, good for Wisconsin Democrats might be too good to be true. The state was ground zero for survey error in 2020, when pre-election polls proved to be too good to be true for Mr. Biden. In the end, the polls overestimated Mr. Biden by about eight percentage points. Eerily enough, Mr. Barnes is faring better than expected by a similar margin.
The Wisconsin data is just one example of a broader pattern across the battlegrounds: The more the polls overestimated Mr. Biden last time, the better Democrats seem to be doing relative to expectations. And conversely, Democrats are posting less impressive numbers in some of the states where the polls were fairly accurate two years ago, like Georgia.
The gist of Cohn’s analysis is that state-to-state, polling bias varies greatly. Some states poll better than others and examining previous cycles can show glaringly different results.
As noted above, in a state like Georgia, for example, Republican Herschel Walker has recently started to see an uptick and the race appears to be tipping in his favor after months of trailing incumbent Democrat Raphael Warnock. Georgia also happened to be more accurate in 2020 in terms of calling the close election between former President Donald Trump and Joe Biden.
The conclusion, from Cohn’s research, is that faulty state polls in areas like Wisconsin, and others, could be a “mirage” which is also driving this narrative that Biden has somehow become a popular figure once again and Democrats might not be doing as bad as they expected:
It raises the possibility that the apparent Democratic strength in Wisconsin and elsewhere is a mirage — an artifact of persistent and unaddressed biases in survey research.
If the polls are wrong yet again, it will not be hard to explain. Most pollsters haven’t made significant methodological changes since the last election. The major polling community post-mortem declared that it was “impossible” to definitively ascertain what went wrong in the 2020 election.
There are also other forms of bias that can be the result of which poll respondents actually decide to respond to a survey. Since the Dobbs decision, some of the people now choosing not to respond are probably skewing the results to an extent:
Since the Supreme Court’s Dobbs decision on abortion, some pollsters have said they’re seeing the familiar signs of nonresponse bias — when people who don’t respond to a poll are meaningfully different from those who participate — creeping back into their surveys.
Brian Stryker, a partner at Impact Research (Mr. Biden is a client), told me that his polling firm was getting “a ton of Democratic responses” in recent surveys, especially in “the familiar places” where the polls have erred in recent cycles.
Democrats, a broad group that generally is more responsive to polls than Republicans, are now twice or three times as likely to jump at the opportunity to express their opinion to a pollster. If that’s the case, says Cohn, it doesn’t necessarily mean polls are totally screwed, but it does mean that the underlying data might not accurately represent the electorate on Nov. 8.
The fundamentals of the midterm elections haven’t changed. Biden remains unpopular despite statistically irrelevant upticks in his approval. Inflation and/or the economy remain the top concerns of voters.
Furthermore, polls indicate (ironic, I know) that very few voters have any faith in the so-called Inflation Reduction Act to do anything that improves their household economic situation in any way.
Republicans still have the heavy favorite to take the House. The Senate is far closer to being a toss-up than it is to being a lean-Democrat scenario. If state polling is as bad as Cohn suggests it could be, then election night will be a roller coaster.
