What Happens to Polling if the Polls Are Wrong in a Big Way Again

What Went Wrong With Polling? Some Early Theories

Pollsters thought they had learned from the errors of 2016. Information technology's possible that they did, and that this ballot reflects new issues.

Nov. 10, 2020

Request for a polling post-mortem at this stage is a little fleck like asking a coroner for the cause of death while the body is still at the crime scene. You're going to have to wait to conduct a total autopsy.

But make no mistake: It's not too early to say that the polls' systematic understatement of President Trump'south back up was very similar to the polling misfire of iv years ago, and might have exceeded it.

For now, there is no easy alibi. Afterward 2016, pollsters arrived at plausible explanations for why surveys had systematically underestimated Mr. Trump in the battlefield states. I was that state polls didn't properly weight respondents without a higher degree. Some other was that there were factors across the scope of polling, like the big number of undecided voters who appeared to interruption sharply to Mr. Trump in the final stretch.

This year, there seemed to be less cause for concern: In 2020, most state polls weighted by education, and there were far fewer undecided voters.

Just in the terminate, the polling error in states was near identical to the miss from 2016, despite the steps taken to ready things. The Upshot's handy "If the polls were as wrong as they were in 2016" chart turned out to be more useful than expected, and it nailed Joe Biden'due south one-betoken-or-less leads in Pennsylvania, Georgia and Arizona.

	Terminal 2020 poll avg.	2020 polls with 2016 mistake		2020 Result
U.South. †	+viii Biden	+6 Biden		+5 Biden
N.H.	+11 Biden	+7 Biden		+7 Biden
Wis.	+10 Biden	+four Biden		<1 Biden
Minn.	+10 Biden	+four Biden		+7 Biden
Mich.	+8 Biden	+4 Biden		+iii Biden
Nev.	+6 Biden	+8 Biden		+iii Biden
Pa. †	+v Biden	<1 Biden		+i Biden
Nib. 2*	+5 Biden	+9 Biden		+7 Biden
Maine 2*	+3 Biden	+9 Trump		+seven Trump
Ariz.	+three Biden	+i Biden		<ane Biden
Fla.	+two Biden	<1 Biden		+iii Trump
N.C.	+2 Biden	+3 Trump		+1 Trump
Ga.	+2 Biden	<1 Biden		<one Biden
Ohio	<1 Trump	+6 Trump		+8 Trump
Iowa	+1 Trump	+five Trump		+8 Trump
Texas	+2 Trump	+four Trump		+six Trump

The national polls were even worse than they were four years ago, when the industry'south about highly respected and rigorous survey houses generally found Hillary Clinton leading past four points or less — close to her 2.ane-signal popular-vote victory. This year, Mr. Biden is on runway to win the national vote past around five percentage points; no major national live-interview telephone survey showed him leading past less than 8 percentage points over the terminal month of the race.

The New York Times/Siena College polls were also less authentic than they were in 2018 or 4 years ago. In 2016, the terminal ii Times/Siena polls were amidst a very minor group of polls to show Mr. Trump tied or alee in Florida and North Carolina. This time, nearly all of the Times/Siena surveys overestimated Mr. Biden to most the aforementioned extent as other surveys.

In the months ahead, troves of information will help add together context to exactly what happened in this ballot, similar concluding turnout information, the results by precinct, and updated records of which voters turned out or stayed dwelling house. All of this information can be appended to our polling, to smash down where the polls were off most and help betoken toward why. Only for now, information technology's still too soon for a confident answer.

In the broadest sense, there are two ways to interpret the echo of 2016's polling error. One is that pollsters were entirely wrong about what happened in 2016. As a result, the steps they took to address it left them no better off. Another is that survey research has gotten even more challenging since 2016, and whatever steps pollsters took to ameliorate afterward 2016 were canceled out by a new prepare of problems.

Of these 2, the latter interpretation — existent improvements canceled out by new challenges — may brand the most sense.

"I recollect our polls would have been fifty-fifty worse this yr had nosotros employed a pre-2016 methodology," said Nick Gourevitch of Global Strategy Grouping, a Democratic polling firm that took steps to better correspond Mr. Trump's supporters. "These things helped make our data more conservative, though conspicuously they were not enough on their ain to solve the problem."

Mind to 'The Daily': About Those Polls…

Joe Biden may have won the ballot, only the margin of victory was much closer than the experts predicted. Why?

transcript

transcript

Heed to 'The Daily': About Those Polls…

Hosted by Michael Barbaro; produced by Jessica Cheung and Michael Simon Johnson; and edited by Lisa Chow and M.J. Davis Lin.

Joe Biden may have won the ballot, but the margin of victory was much closer than the experts predicted. Why?

michael barbaro

Hey, information technology's Michael. This episode has been updated to clarify our analysis of the Latino vote in the election.

From The New York Times I'm Michael Barbaro. This is "The Daily."

Today: What the results of the election taught us about the American electorate, and almost the polling that in one case again failed to correctly sympathize information technology. A conversation with my colleague Nate Cohn.

It'due south Tuesday, November 10.

And so Nate, nosotros want to take two conversations with you lot. The outset is about what we learned from the vote, the results of this election, now that the race has been called. And 2nd is what we learned nigh why the polls were so off again, including our own polling hither at The Times, and kind of have a reckoning on that forepart.

nate cohn

Sounds good.

michael barbaro

Right, I'm sure it's going to be a total pleasure for you.

nate cohn

It will be.

michael barbaro

Permit's offset with the voters and how this celebrated turnout for both candidates broke down. And maybe the most intriguing grouping in this campaign, then far in the results, are Latino voters. So tell us what the final-ish data is telling y'all about Latino voters?

nate cohn

Well, I suspect that many listening right now have probably heard that Donald Trump fared quite well among Latino voters. You learned that pretty speedily on ballot dark when the results from Miami-Dade canton in Florida come in. And now that we take seen more detailed results from elsewhere in the state, I think we can safely say that Latino voters actually swung to Donald Trump. Estimates vary about Latino support for Trump in 2016, simply information technology'south somewhere in the 25 to 30 percent range.

This twelvemonth, we are still waiting for all the numbers, just we're expecting that to look more like 35 percent. And, there'south been important focus this year on the fact that the Latino vote is not a monolith — hence the divided back up for Biden and Trump.

Simply what'southward remarkable virtually this year is that when you look at the results and then far, this increase in support for the president is consistent all over the country. Information technology'south truthful in the agricultural regions of California, like the Royal Valley. Information technology's true in the border towns along the Rio Grande. It's truthful, every bit you know, in Cuban areas like in Miami-Dade Canton. But it's also in Puerto Rican areas or around Orlando and Kissimmee. It seems like it'south fifty-fifty true in the northern cities like Philadelphia or Milwaukee, where Latino voters are unremarkably the very about reliable for Democrats. And the magnitude of the improvement for the president is really significant. There are counties forth the Rio Grande where Trump picked up 50 points.

michael barbaro

l points amid simply Latino voters?

nate cohn

Yes, I mean, you tin't know for sure that there isn't some contribution of white voters in these counties, of course.

Merely these are places where Latino voters make up the overwhelming majority of the electorate. Nosotros can say the aforementioned affair for these Latino enclaves in Philadelphia or in Milwaukee. The magnitude of the gains there is not as staggering as it is among the Rio Grande, just it'due south still really stark: 10, 15, 20-point gains for Donald Trump.

michael barbaro

So, Nate, how do you outset to explain what is now a much broader miracle than perhaps many of united states thought information technology would exist on election nighttime? Is this the story of Democrats declining to capture this vote? Or is this the story of Donald Trump succeeding in capturing it? Is it both? How are you lot thinking near this?

nate cohn

Yeah, the first matter I would just say is I call up this is a place where reporting on the ground will be really helpful. The polls struggle with Latino voters. On a national survey, you only get 100 Latino voters. It's difficult to aggregate a lot of Latino respondents to do the kind of deep dives that nosotros tin can do on larger subgroups like white voters, for instance, who I'm sure we'll go to. Equally a general framing signal, I would say that this election was about Donald Trump. It was a referendum on the president. Joe Biden did everything he could to brand certain it was about the president. He stayed in his basement equally the Trump entrada oft noted. And so I am inclined to presume that when we see big shifts that it reflects attitudes about the president, not Joe Biden.

michael barbaro

So we should read into this that the president successfully appealed over the final year or so to this group of voters?

nate cohn

That's right. 1 affair I would point out is that there was always strong evidence that the economic system was the president's stiff conform in this ballot. And when you look at the results, non but for Latino voters, but also for other non-Hispanic voters, information technology does seem to me that the president made his largest gains in less affluent areas. It would not surprise me if that reflects the greater salience of the economic system for economically vulnerable people, compared to people who might accept the privilege to vote on other issues like cultural bug or the president'due south conduct in function and then on.

A 2d ascertainment I would brand is that to me this was not an election near clearing in the same fashion that the 2016 election was about immigration. Trump's pledge to build the wall. His comments on his very first twenty-four hour period he sought the presidency when he said that Mexicans were rapists and criminals and so on. Hillary Clinton in 2016 really focused on immigration as one of her major critiques of the president. And I wouldn't say that was almost so true in 2020. I don't think immigration even came up in the get-go presidential argue. So I think you can imagine that immigration became less salient to not just Hispanic voters, simply all voters. Just that was particularly consequential among Hispanic voters, because I remember we tin can at least hypothesize that many Latino voters in 2016 were skeptical of the president in role because of the way he talked almost this effect.

michael barbaro

So the issue that might have hurt President Trump with Latino voters — immigration — was far less prominent this fourth dimension than four years ago. And the thing that would potentially assistance him the most with Latino voters — the economy — did seem to piece of work in his favor, insofar as Latino voters identified him with economic prosperity, especially pre-pandemic economic prosperity?

nate cohn

Yeah, that'south right. And one final indicate I would brand is that the president has ever washed really well among white voters without a degree. His schtick has had appeal for that demographic grouping. And I don't call up that'due south just most policy. I also recall it'due south about his conduct. I retrieve that the Trump human action has entreatment to a lot of people who haven't previously been terribly receptive to Republican politicians in the by.

And I think that if nosotros're honest, that it's not e'er obvious why the president's appeal to white working-class voters on the economic system, or in terms of his carry and temperament, wouldn't also have appeal to working-class voters who are non-white, whether they're Black or Latino.

So I could cobble together a theory that sort of says that many of the things that appeal to white working-grade voters about Donald Trump may appeal to Hispanic voters besides. But in 2016, that was obscured by his position on clearing. And once immigration was taken out of the motion picture, mayhap that gave the president a belated opportunity to make the same sort of gains amid working-class Latino voters in 2020 that he was able to make amidst white working-class, traditionally Autonomous voters, in 2016.

michael barbaro

Information technology's interesting, Nate, that you made that parallel, considering every bit you were describing what's happening here, it occurred to me that if you are the Autonomous party, the terminal major grouping of voters that were mayhap taken for granted in the past decade or so were white working-class voters, especially in the Midwest. Autonomous leaders thought those voters were unquestioningly theirs. I wonder if that's the right way of thinking virtually how the Democratic party saw Latino voters in 2020 — that there was an assumption that they would be with Joe Biden. And they merely were non to the same degree.

nate cohn

I think that's right. I think there were a lot of people in this land who became Democrats during an era when the Democratic Party was the party of working people and the Republicans were the party of the rich and business concern interests, who over the terminal 4 years accept come to encounter the president as a different kind of Republican.

And the president has denied the Democrats some of their traditional reward among working-course voters of all races as a result. This was not a campaign about privatizing social security. It wasn't a entrada nigh the minimum wage and so on. Instead, the Democrats have advanced a sort of idealistic liberal bulletin with obvious resonance at least among higher-educated voters. And it didn't have the same resonance amid working-class voters. And Democrats in the past have gotten away with information technology past falling back on their traditional strengths amid these groups, long-standing allegiance that has perhaps been eroded at present the Republicans have put forward a more populist candidate.

michael barbaro

OK, Nate, what about Black voters? What did we larn near this population from the results of Biden's victory?

nate cohn

I think that at that place were a few — I would brand a few points about what happened with Black voters. Ane is that it seems to me that although Black turnout increased, it did not increase the same extent equally it increased amid non-Black voters. And so as a result, the Black share of the electorate seemed to decline.

michael barbaro

The number of Black Americans who voted did increment. Only the number of everybody else increased only a footling bit more?

nate cohn

That'due south right. So you may recall in 2016, the turnout in places like Milwaukee and Philadelphia and Detroit was downwardly. It was down by so much that Hillary Clinton narrowly lost. In 2020, the turnout was up significantly, enough that if you could go back in time with this Black turnout, Hillary Clinton would have been the president. But yet, the white turnout elsewhere in these states increased by even more, such that if you lot could go back in time and increment white turnout in 2016 to the same extent, Donald Trump would and then come back and be president once more.

michael barbaro

Wow, that is actually fascinating. You're saying if yous practical the increment in both the Black and white vote that occurred this yr back and so, you'd get the aforementioned result.

nate cohn

That'south right. In the end, the Black versus white turnout dynamic did not change in a way that would accept allowed the Democrats to prevail 4 years ago.

michael barbaro

And among the Blackness voters who turned out, what percentage supported Biden versus President Trump? Considering I accept the sense that the president did somewhat meliorate with Black voters than was expected.

nate cohn

I retrieve that's right to an extent. I recollect that this is, over again, a case where we'll need to see the final data before we boom this down. Just let's just say that Biden won roughly 90 percent of the Blackness vote and that Donald Trump won effectually 10 percent of the Blackness vote. Requite or take, we'll see what the terminal numbers are. That will be better for Donald Trump than in 2016. And if I were forced to explain information technology, I would chalk up a like explanation to what I told you with Latino voters or some combination of the president's economical appeal and and so on. It was able to help him skin off only a little bit. Simply I don't desire to overthink it also much. This is a pretty small shift all things considered. And Democrats go along to control the overwhelming support of Blackness voters.

michael barbaro

Nate, if your theory is authentic and what connects the improvement by Donald Trump amid Latino and Black voters is his economic message, what's interesting nearly that, of form, is that the economy is doing terribly correct now considering of the pandemic. So it means that there's a vestigial amore for the president'due south success a yr agone, the manner the stock market did before the pandemic, the way chore growth was going before the pandemic. Merely it's remarkable that he's still getting credit for that given where things are right at present.

nate cohn

Here are the points I would make. I, you lot're totally correct. I recollect that the president does withal get credit for the style he handled the economic system during ordinary circumstances. And people can rationally believe that the president was a good steward of the economy when coronavirus wasn't in the picture. And therefore, he'll be a good source of the economic system once coronavirus is out of the moving picture.

And the second thing I would say is that I think that he also has gotten some credit from people for the way he handled the economic system during the coronavirus. The stimulus packet was actually pop. The Democrats take not criticized the way he'south handled the economy during the coronavirus. They've just criticized his treatment of the coronavirus. And even on his handling of the coronavirus, the main criticism of him is that he'south too eager to reopen the economy.

Then information technology's a little bit complicated to make people go through the logic of, here's someone who wants to reopen the economy. And therefore is on the side of getting people dorsum to work, and and then on. Only he's actually bad for the economy because that step that's facially good for the economic system volition contribute to the spread of the coronavirus to force shutdowns, which hurt the economic system. That's — I do recall that nosotros take that logic for granted in a way that I'one thousand not sure we should wait of ordinary people.

michael barbaro

Off-white bespeak. OK, so I now desire to plow to a grouping that nosotros accept been talking near a lot throughout this campaign, a group that was predicted to potentially exist decisive in this election, and that was suburban voters. Polls predicted that many of the suburban voters who had voted for Trump four years ago would swing to Biden. Did that end up happening?

nate cohn

That'south the thing that concluded up happening. And information technology was just enough for him to get over the tiptop in the northern battleground states as well as in Arizona and in Georgia. Across the land, Joe Biden did ameliorate in suburban areas than Hillary Clinton did four years ago. His gains were largest in traditionally Republican suburban areas, like Atlanta, like Dallas, like Jacksonville, Indianapolis, Kansas City, Grand Rapids, Michigan — places that are non total of liberal suburbanites that have been voting Democratic for a really long time, like Westchester Canton or the suburbs of D.C. and Philadelphia.

And don't go me wrong, Biden did make gains in a lot of those traditionally Democratic suburban areas. Only the biggest gains were in the areas where 10 or 15 years ago, the Republicans dominated among the suburban vote. And now a lot of traditionally Republican, flush white voters in those areas are saying, wait a second, this isn't the Republican Party I signed upwards for. And at present it has swung pretty decisively to the Democrats.

michael barbaro

And practise we know why there was this shift? It feels like that word you merely used, affluence, may exist an chemical element of it. These are voters who can financially afford to cast a vote based on whether they like Trump or Biden.

nate cohn

Yeah, I definitely recollect in that location's something to that. I think that in that location are a lot of rich white people in the suburbs of Dallas and the suburbs of Atlanta, who've been voting for their tax cuts for a very long time. But Donald Trump crossed a line with them in terms of his personal conduct. And they are willing to vote on what they feel and think almost him as a person in a way that peradventure other people are not necessarily and then inclined to do.

I also do recall there is a policy element of it likewise. I think that clearing has ever been popular. And gratis trade has always been popular amidst a big chunk of conservative voters in suburbs across the Southward. And the president has departed from traditional Republican views on some of those bug in a way that may at least somewhat complicate the idea that this is strictly a vote confronting their policy views. But I do think it's truthful that information technology'south a lot easier for someone making $200,000 a twelvemonth to vote against their perceived economical interests than it would be if you brand $20,000 a year.

michael barbaro

OK, then finally, what about white non-college educated voters, many of whom are in rural America, not suburban? So we demand to be talking about ex-urban or rural voters. This is traditionally seen as the president's base of operations. This is the grouping that Biden had hoped he would get-go to poach. How successfully did he do that? How well did President Trump defend this base of operations for himself according to the data that yous have seen?

nate cohn

Donald Trump defended his base of operations here into a far greater extent than whatsoever of the pre-election polls predictable. As far every bit I can tell, Joe Biden didn't make any gains in rural America among white voters since 2016. And as you alluded to, the whole premise of choosing Scranton Joe was that if the Democrats nominated a white moderate with some populist appeal, that you could win back some of the voters who backed Barack Obama in 2012, supported Donald Trump in 2016. And that just didn't happen. And in many cases, Donald Trump extended his gains. There are places in rural Iowa and rural Ohio, where Donald Trump did even improve than he did four years ago. And that this is the exact opposite of what the pre-ballot polls said. So something went very wrong in the public opinion research there.

michael barbaro

Well, we will go to that. Just at the end of the twenty-four hours, if I'chiliad putting all these pieces together the style I call back you have intended for me to, it feels similar suburban voters end up existence the most important group to shift and shift in the direction of Joe Biden. Is that right?

nate cohn

That's absolutely right. We have known for a very long time that voters had deep reservations about Donald Trump. We know that he only received 46 percent of the national vote in 2016. We know that he was the least popular candidate in terms of favorability ratings when he was elected. And the president had four years to try and address those shortcomings.

It seems that his handling of the economy and his performance on the job was enough to persuade some number of more economically vulnerable voters, or maybe just voters in full general — even higher on the economic spectrum — to support his re-election despite the personal reservations that they've probably had about him for years.

Just it also seems that in nominating Joe Biden — someone who is adequately broadly appealing and promised to unify the country and to deed in a mode more conforming of the office — that Joe Biden was able to consolidate a significant number of voters who have had longstanding reservations about the president, specially college-educated voters, peculiarly traditionally Republican tilting independent voters in the suburbs effectually many of our largest cities. And that, to me, tells y'all a very odd and interesting story. It says that the economic system can really help the president as we've already known. But there are limits to it.

michael barbaro

We'll be correct dorsum.

archived recording 1

Let'south take a await at our new poll. And the headline number is this. This is a wow. The largest lead of the race for Joe Biden.

archived recording 2

Joe Biden with a 14-indicate pb nationally over President Trump.

archived recording iii

10 points.

archived recording iv

12-point lead over Trump.

archived recording 5

The largest atomic number 82 of the race for Joe Biden by 16-point spread. Joe Biden leads in Michigan. He leads in North Carolina. He leads in Pennsylvania. He leads in Wisconsin.

archived recording 6

Joe Biden is leading President Trump by 11 points in Wisconsin in the latest New York Times Siena College poll.

archived recording seven

Joe Biden is in the hunt in these races that have been going cherry of late in this state. So imagine Joe Biden'southward ballot night could besides include North Carolina, could possibly include Florida. This is not an out-of-the-question map hither for Joe Biden, given the state of the race we know.

michael barbaro

So Nate, at present for the root canal department of this conversation. Y'all cover polling for The Times. And you are involved in how polling itself is conducted for and past The Times along with our partners. And we all know, at this point, that at that place was significant polling error this year. How would you describe the level of polling error in 2020?

nate cohn

I would consider this to exist far worse than 2016.

michael barbaro

Actually?

nate cohn

Far worse. In terms of the difference betwixt the final poll results and the bodily results, the difference is non that much worse than 2016. That'south pretty comparable. But pollsters this twelvemonth do not have the same excuses that they had 4 years agone. And I recall that the error is much more systematic and portrays much more than cardinal problems with the attempt to correspond the electorate than the polling did four years ago.

michael barbaro

And I desire you to give u.s.a. some examples simply so we are on the same page. When you lot say the polls are worse this fourth dimension than 4 years ago, what exercise you mean by that?

nate cohn

Maybe information technology's easier to start with 2016 for a second. The polls did a good job of representing white voters without a caste, in that they showed white voters without a caste being great for Trump against Hillary Clinton. But they did not have enough of those voters. And although that is a serious problem in polling — to not take enough of a certain demographic group — information technology is a common trouble in polling. And it is a fixable trouble, because you can give more weight to voters from that demographic group. And in the post-election assay four years ago, pollsters constitute that when they gave more weight to that group, that the polls did pretty well. In 2020, pollsters did that. They gave more weight to white working-class voters. But they were no closer to the upshot, even though we know that the aforementioned technique four years agone would have brought the polls closer to the result.

michael barbaro

Then I just want to be articulate on this. You're proverb what makes the error in 2020 greater and more grievous than 2016 is that we entered 2020 having made meaningful adjustments to our polling methodology, all polls, that come to the presidential ballot. And yet we still had super off polls despite those reforms and adjustments?

nate cohn

Non only were they super off, though, they were just as bad. And what that means is that while iv years agone, the polls showed Donald Trump doing really well amidst white voters without a degree, and had too few of them, this yr the polls had the correct number of white voters without a degree. Only they showed Joe Biden doing way better amidst this grouping than he actually did. So for all this ballot, nosotros've been saying white voters without a caste who voted for Donald Trump in 2016 appeared to accept come dorsum for Joe Biden. And that didn't happen.

michael barbaro

OK, so how exercise you lot explain the level of error that y'all just ticked through after all the corrections fabricated four years ago to endeavor to ensure that this would never happen?

nate cohn

I am going to try and answer your question. Just before I do that, I practice want to say that information technology is too soon to take a definitive autopsy on this. I'm going to accept that analogy a little further. Merely the trunk is still expressionless at the scene of the crime. It has not made information technology to morgue notwithstanding for u.s.a. to go into the details and dig in to see exactly what went on here. I mean, but we know the polls went really wrong. But it'due south going to exist a bit until we've nailed downwardly exactly why. Then I'm happy to speculate.

michael barbaro

But assuming you're a crime scene investigator and you're there and yous're trying to figure information technology out, what do you know and so far?

nate cohn

Yeah, I can just toss out some ideas. And that'south all they are at this stage. We have seen over the last iv years, a huge increase in political participation on the left. We also know that political participation correlates with participating in polls, which brand sense. You get called. And you lot either choose to take the survey or non. And if that interests you lot considering you're interested in politics, you are likelier to take a poll. This could mean that Democrats became a little more likely to take a poll. That's one possibility.

Another possibility is looking on the other side that Trump voters are less likely to answer to surveys than they were 4 years ago. Perhaps, the president's attacks on the media and institutions have gradually eroded their trust in surveys. Maybe even the 2016 issue itself had a role in diminishing their willingness to participate in surveys, considering the polls were off past so much that now they don't trust them and don't want to play. We make sure in the state of Pennsylvania that 41 percent of our interviews are with Republicans or whatsoever the number was in that area. Which Republicans picked upwardly the phone? Were they the ones that were actually enthusiastic about the president in rural areas? Or were they the people who are also registered every bit Republican just are no longer fans of the president?

I think that nosotros have to conclude that we got the ones who are more than likely not to be fans of the president. That'due south non to say nosotros didn't become enough of people who supported the president, but plenty of a difference compared to reality that you nudge all of the polls in i direction.

michael barbaro

Well, Nate, nosotros've had four years of the president speaking nearly the organizations that bear polling in a very specific way. And that would be The New York Times, which conducts polls. That would be ABC News. That would be CNN. That would exist CBS. And he has described them as the enemy of the people. He has described them every bit false media. So I don't take a hugely difficult time agreement why his supporters would be skeptical of answering our calls.

nate cohn

No, I think that four years of the president advancing that message could accept had a negative effect on the polls. I call back that'southward totally possible. I don't have any proof of that. But what'southward important near that theory is that it could potentially explicate why the polls have gotten worse since 2016. That, to me, is the key part of any theory if you really want to try and dig into what'due south happening here.

michael barbaro

And so, Nate, are we left with the kind of inevitable conclusion here, subsequently ii presidential elections that pretty much committed the aforementioned sin perhaps for different reasons, that election polling — pre-election polling — is broke?

nate cohn

That'southward a great question. And I think it depends on the level of precision that y'all expect out of information technology. And there is a spectrum of ways that you can interpret polling errors. I possibility is that the polls are imprecise, but they're still useful. They'll never be able to blast down whether Joe Biden is going to win Georgia or lose information technology by iv or win it by four. Similar they're just non that good. Simply they're skilful enough to flag that something happened in Georgia, and that'due south useful to united states.

Another possibility is that they're so imprecise that they're no longer useful. It'south like all right, we got these polls showing Biden upwardly eight points. Just in our political era, the range of possible outcomes in national elections is pretty tight. In my lifetime, the whole range of results goes from what? Bush plus two in 2004 to Clinton plus eight in 1996. And so if the amount of mistake in a poll is basically equal to the range of possible results, they may tell united states of america something. But they're just not that useful. In whatsoever given election, either side can win. And you can't dominion out anything. And you didn't become that much out of information technology.

3rd possibility is that the polls are so incorrect that they're counterproductive. They're non simply useless. But they give an actively misleading moving picture of the country that we live in. That would be actually bad if true.

michael barbaro

OK, so now I demand to know where practise you autumn on this spectrum? Imprecise but directionally useful; useless but we can live with them; really bad and misleading and counterproductive? Considering it feels similar for a lot of people, number three is where nosotros may be living right now. Where is your head?

nate cohn

I remember in that location are definitely elements of the polling error this year and in 2016 that autumn in to the counterproductive category. In Ohio and Iowa, Joe Biden decided to spend late parts of his campaign, final stop, in Cleveland on the day before the ballot. That'south outright counterproductive. It'due south likewise counterproductive for readers in the electorate. There are cases where political activists brand decisions to spend money on longshot Senate races similar South Carolina Senate, or the Kansas Senate race, or the Alaska Senate race where people simply put their energies into the wrong spot. There'south besides a toll on people's trust in institutions. Polling is an inherently uncertain thing. And so nosotros do look to some extent that readers capeesh that polls can be off. And it shouldn't undermine the credibility of other things that we have to say or do, which could exist more than based on the firmest facts.

Just I call up that it does undermine the credibility of our ability to communicate to readers that nosotros empathize the departure between something that we know to be true and something that is a best approximate.

michael barbaro

But you didn't answer my question. Where exercise you autumn?

nate cohn

I'm really — look, I think that —

michael barbaro

Information technology sounds similar you're torn.

nate cohn

I really am torn. I call back the alternatives to public polling are pretty bad. I mean, we're only stuck to talk to our neighbors and our agreeing friends. We wouldn't really take whatever idea of what'due south happening the rest of the country potentially. And so that'south tough to accept. Let's revisit this after the autopsy. I hateful, if nosotros go through the data and we find that there are things that we can fix, then we'll make those changes and evaluate whether we think we're still in the imperfect simply useful category. And and then nosotros might continue to do some amount of public polling in the hereafter.

But if we conclude that we don't think we can fix these things, and then nosotros accept a really hard choice, which is whether to abandon the enterprise altogether — which has some of import costs, because if you don't accept a read on the attitudes of the electorate, I think nosotros're left with alternatives that are not very skilful. And I think it does affair to empathize where the American people are at. I think information technology'southward important to the mode this democracy works. Simply if we can't get there, so we can't get in that location. And we would have to re-evaluate what nosotros'll do going frontwards.

michael barbaro

Well, Nate, I look forrard to hearing what you learn. Cheers very much for your time.

nate cohn

Thanks for having me.

michael barbaro

We'll be right back.

Here'due south what else you lot need to know today.

archived recording (senator mitch mcconnell)

I desire to spend a few minutes this morning talking about what we saw last calendar week, where we are now, and where our great land volition go from here.

michael barbaro

In a speech from the capital on Mon, Senate Majority Leader Mitch McConnell threw his support behind President Trump'south refusal to concede the election and declined to recognize President-elect Joe Biden's victory.

archived recording (senator mitch mcconnell)

President Trump is 100 percent within his rights to expect into allegations of irregularities and weigh his legal options.

michael barbaro

McConnell mocked Democrats for calling on Trump to accept the results of the election, saying that many of them had never recognized Trump's victory four years ago. That message prompted a scolding from Senate Minority Leader Chuck Schumer.

archived recording (senator chuck schumer)

Joe Biden won this election off-white and square. Republican leaders must unequivocally condemn the president'south rhetoric and work to ensure the peaceful transfer of ability on January 20th. But too many, including the Republican leader, have been silent or sympathetic to the president'south fantasies.

michael barbaro

And the drugmaker Pfizer announced that an early assay of large-scale human testing has found that its coronavirus vaccine is more than 90 percent constructive in preventing infection — a highly promising result that could make it a leading candidate for federal approval. The clinical trial is not complete, and the issue could change. But so far, the company said, no serious safety concerns have been observed. Pfizer said they would inquire the F.D.A. for emergency authority for the vaccine later this month.

That's information technology for "The Daily." I'm Michael Barbaro. See y'all tomorrow.

The explanation for 2016's polling error, while non necessarily consummate or definitive, was not contrived. Many land pollsters badly underrepresented the number of voters without a college degree, who backed Mr. Trump in huge numbers. The pollsters went back to their data after 2016, and found that they would accept been much closer to the election result if they had employed the standard instruction adjustments that national surveys have long used. An Event analysis of national surveys constitute that failing to weight past education cost Mr. Trump about 4 points in polling support — enough to cover much of the 2016 polling error. Other pollsters had similar findings.

But this time, educational activity weighting didn't seem to help. State and national polls consistently showed Mr. Biden faring far better than Mrs. Clinton did among white voters without a caste. Last week's results fabricated it articulate that he didn't.

Over all, the final national surveys in 2020 showed Mr. Trump leading by a margin of 58 percentage to 37 percent amid white voters without a degree. In 2016, they showed Mr. Trump ahead by far more than, 59-30. The results by county propose that Mr. Biden made few gains at all among white voters without a degree nationwide, and even did worse than Mrs. Clinton's 2016 showing in many critical states.

In contrast, the 2016 polls did show the decisive and sharp shift among white voters without a degree, merely underestimated its effect in many states because they underestimated the size of the grouping. Many state polls showed college graduates representing one-half of the probable electorate in 2016, compared with most 35 percentage in census estimates.

The poll results amid seniors are some other symptom of a deeper failure in this year's polling. Unlike in 2016, surveys consistently showed Mr. Biden winning past comfortable margins among voters 65 and over. The concluding NBC/WSJ poll showed Mr. Biden upwards 23 points among the group; the final Times/Siena poll showed him upward by x. In the final account, there will be no reason to believe whatever of it was real.

This is a deeper kind of error than ones from 2016. It suggests a fundamental mismeasurement of the attitudes of a large demographic grouping, not merely an underestimate of its share of the electorate. Put differently, the underlying raw survey data got worse over the last 4 years, canceling out the changes that pollsters made to accost what went incorrect in 2016.

It helps explicate why the national surveys were worse than in 2016; they did weight past pedagogy four years ago and have made few to no changes since. It as well helps explain why the mistake is so tightly correlated with what happened in 2016: It focuses on the same demographic group, even if the underlying source of the fault among the group is quite different.

Polling clearly has some serious challenges. The industry has always relied on statistical adjustments to ensure that each group, like white voters without a degree, represents its proper share of the sample. But this helps only if the respondents yous achieve are representative of those you don't. In 2016, they seemed to be representative enough for many purposes. In 2020, they were non.

So how did the polls become worse over the last iv years? This is mainly speculation, but consider just a few possibilities:

The president (and the polls) hurt the polls. At that place was no existent indication of a "hidden Trump" vote in 2016. But maybe there was i in 2020. For years, the president attacked the news media and polling, among other institutions. The polls themselves lost quite a bit of credibility in 2016.

Information technology'southward hard non to wonder whether the president's supporters became less likely to respond to surveys as their skepticism of institutions mounted, leaving the polls in a worse spot than they were four years agone.

"We at present have to accept seriously some version of the Shy Trump hypothesis," said Patrick Ruffini, a Republican pollster for Echelon Insights. Information technology would exist a "trouble of the polls simply not reaching large elements of the Trump coalition, which is causing them to underestimate Republicans across the lath when he's on the ballot."

(This is different from the typical Shy Trump theory that Trump supporters don't tell pollsters the truth.)

A related possibility: During his term, Mr. Trump might accept fabricated gains among the kinds of voters who would be less likely to respond to surveys, and might have lost additional ground amid voters who would exist more likely to respond to surveys. College education, of form, is merely a proxy for the traits that predict whether someone might back Mr. Trump or respond to a poll. There are other proxies besides, like whether you trust your neighbour; volunteer your time; are politically engaged.

Another proxy is turnout: People who vote are likelier to take political surveys. The Times/Siena surveys go to great lengths to reach nonvoters, which was a major reason our surveys were more favorable for the president than others in 2016. In 2020, the nonvoters reached by The Times were generally more favorable for Mr. Biden than those with a track record of turning out in recent elections. Information technology'southward possible that, in the end, the last data will suggest that Mr. Trump did a amend chore of turning out nonvoters who backed him. But it's also possible that we reached the wrong low-turnout voters.

The resistance hurt the polls. Information technology's well established that politically engaged voters are likelier to respond to political surveys, and information technology's clear that the election of President Trump led to a surge of political engagement on the left. Millions attended the Women's March or took part in Black Lives Thing protests. Progressive activists donated enormous sums and turned out in record numbers for special elections that would have never earned serious national attention in a different era.

This surge of political participation might have also meant that the resistance became likelier to respond to political surveys, controlling for their demographic characteristics. Are the "MSNBC moms" now excited to take a poll while they put Rachel Maddow on mute in the background? Like about of the other theories presented here, there'due south no difficult evidence for it — just it does fit with some well-established facts nigh propensity to reply to surveys.

Prototype

Credit... Janie Osborne for The New York Times

The turnout hurt the polls. Political pollsters have oftentimes assumed that higher turnout makes polling easier, since it means that there'south less doubt about the limerick of the electorate. Maybe that's not how it worked out.

Heading into the ballot, many surveys showed something unusual: Democrats faring better amongst likely voters than among registered voters. Usually, Republicans hold the turnout edge.

Take Pennsylvania. The final CNN/SSRS poll of the country showed Mr. Biden up past 10 points amidst probable voters, but by merely five among registered voters. Monmouth showed Mr. Biden up by vii amidst likely voters in a "high-turnout" scenario (which it ended upwards being), simply by five points among registered voters. Marist? It had a pb of 6 points among likely voters and five points among registered voters. The ABC/Washington Post showed a seven-point lead for Mr. Biden amongst likely voters and a 4-bespeak lead amid registered voters.

It's even so too soon to say whether Republican turnout shell Democratic turnout, simply information technology sure seems possible. In Florida, the one country where we do have difficult turnout information, registered Republicans outnumbered registered Democrats by well-nigh two percentage points amid those who actually voted, even though Democrats outnumber Republicans among registered voters past near 1.v points in the land. Here, in that location is no incertitude that the turnout was better for the president than the polls suggested, whether they're individual polls or the concluding Times/Siena poll — which showed registered Republicans with an edge of 0.7 points.

If Mr. Trump fared meliorate amid likely voters than amid registered voters in Pennsylvania, a key misfire on the estimate of turnout could very speedily explain some of the miss.

Unlike the other theories presented here, this ane can exist proved false or true. States will eventually update their voter registration files with a tape of whether voters turned out in the election. We'll exist able to run into the exact limerick of the electorate by party registration, and nosotros'll too be able to see which of our respondents voted. Perhaps Mr. Trump'south supporters were likelier to follow through. We might start to become data from North Carolina and Georgia in the adjacent few weeks. Other states might have longer.

The pandemic injure the polls. Recollect those Times/Siena polls from October 2019 that showed Mr. Biden narrowly leading Mr. Trump? They turned out to be very close to the bodily result, at least outside of Florida. They were certainly closer than the Times/Siena polls conducted since.

It wasn't just the Times/Siena polls that were closer to the mark farther alee of the ballot. Results from pollsters in February and March look only about dead-on in hindsight, with Mr. Biden leading by about 6 points among registered voters nationwide, with a very narrow pb in the "bluish wall" states, including a tied race in Wisconsin.

One possibility is that the polls were just as poor in October 2019 as in October 2020. If so, Mr. Trump actually held a clear lead during the wintertime. Maybe. Another possibility is that the polls got worse over the terminal yr. And something really big did happen in American life over that time: the coronavirus pandemic.

"The basic story is that subsequently lockdown, Democrats just started taking surveys, considering they were locked at dwelling house and didn't have anything else to do," said David Shor, a Democratic pollster who worked for the Obama campaign in 2012. "Near all of the national polling mistake can be explained by the post-Covid jump in response rates amongst Dems," he said.

Circumstantial evidence is consistent with that theory. We know that the virus had an effect on the polls: Pollsters giddily reported an increase in response rates. High-powered studies showed Mr. Biden gaining in coronavirus hot spots, seeming to confirm the supposition that the pandemic was pain the president.

But if Mr. Shor is correct, the studies weren't showing a shift in the attitudes of voters in hot spots; rather, it was a shift in the tendency for supporters of Mr. Biden to respond to surveys.

Adding to the intrigue: There is no evidence that the president fared worse in coronavirus hot spots, contrary to the expectations of pundits or studies. Instead, Mr. Trump fared slightly improve in places with high coronavirus cases than in places with lower coronavirus cases, controlling for demographics, based on the preliminary results by canton and so far. This is well-nigh obviously true in Wisconsin, one of the nation's current hot spots and the battleground country where the polls underestimated Mr. Trump the most. The final polls in Wisconsin — including the final Times/Siena poll — showed Mr. Biden gaining in the state, fifty-fifty equally polls elsewhere showed Mr. Trump making gains.

Don't forget the Hispanic vote. There's one state in detail where the polls were much worse in 2020 than in 2016: Florida, where Mr. Trump fabricated huge gains among Hispanic voters.

What happened in Miami-Dade County was stunning. Mr. Biden won by just 7 points in a county where Mrs. Clinton won by 29 points. No pollster saw the extent of it coming, not even those conducting polls of Miami-Dade County or its competitive congressional districts.

Virtually polls probably weren't fifty-fifty in the ballpark. The final Times/Siena poll of Florida showed Mr. Biden with a 55-33 atomic number 82 among Hispanic voters. In the final business relationship, Mr. Biden may barely win the Hispanic vote in the land.

What happened in Miami-Dade was non only about Cuban-Americans. Although Democrats flipped a Senate seat and are leading the presidential race in Arizona, Mr. Trump fabricated huge gains in many Hispanic communities beyond the country, from the agricultural Imperial Valley and the border towns along the Rio Grande to more urban Houston or Philadelphia.

Many national surveys don't release results for Hispanic voters because any given survey usually has only a pocket-size sample of the group. It will exist some time until the major pollsters postal service their results to the Roper Center, a repository of detailed polling data. Then nosotros'll exist able to dig in and see exactly what the national polls showed among this group.

But if the Florida polls are whatsoever indication, it'south at least possible that national surveys missed Mr. Trump's force among Hispanic voters. It seems entirely possible that the polls could have missed by 10 points among the group. If true, information technology would account for a small but pregnant office — maybe ane-quaternary — of the national polling fault.

These are the initial guesses. Other theories will sally. In time, to the extent they tin be, all of them will be put to the test. And so nosotros'll know more than we practise at present, and can revisit this question.

coronahenceall.blogspot.com

Source: https://www.nytimes.com/2020/11/10/upshot/polls-what-went-wrong.html