25%, 1st Place
10%, 5th Place
6%, 5th Place
Average: 13.7%, 4th Place
Hmmmmmm. Does the truth really lie somewhere in between the data points? Or are one of the extremes more on target?
Polling averages exist for a good reason. Surveys have a margin of error. It’s too easy to read too much into one result. The standard measure is the Real Clear Politics average. If you read a piece about poll numbers, and an average is referenced, it’s likely theirs. Standards are good too. Common frames of reference make writing a piece like this easier.
RCP doesn’t count every poll though. They grab all the most-respected surveys and some of the others. There’s good overlap with the polls certified by the DNC to count toward debate qualification. FiveThirtyEight publishes a wider range of polling. If they don’t list it, it doesn’t exist. They also have a poll-rating system.
Outlier results are more common from lower-rated providers, but that doesn’t mean they’re always wrong. Sometimes they’re correct, but in a different way than they thought. AKA, two wrongs can indicate a right.
Let’s visit 2016. Hillary. The Bern. A razor thin Iowa result. South Carolina extinguishing Bernie’s fire. The final RCP average in Iowa was very accurate. Clinton led by 4%, but Martin O’Malley held 4.3%. With the 15% viability rule—a candidate needs to reach 15% at each individual caucus location in order to count—O’Malley’s support was guaranteed to get redistributed. Bernie got most of it and the final result was an effective tie.
Score one for the averages. The road to get there was a bit more fraught. Outliers abounded. Both Gravis Marketing and the Loras College poll frequently showed Sanders trailing Clinton by 20 or 30 points, even when many other surveys had them within single digits.
Two weeks ahead of the caucus, CNN showed Bernie up by 8. Loras had Hillary at +29. These pollsters were viewing two alternative universes. Gravis began showing a tighter contest, with their final survey a few days ahead of the vote recording Clinton +11.
Gravis was wrong. Loras was wrong. Embarrassingly so. Or were they?
Loras didn’t poll in New Hampshire. Gravis did. When they surveyed in mid-January, the result was Bernie +3. This was on the low side. CNN got +27 at the same time. Then Iowa voted.
Polling has a bunch of elements. The front of mind one is how the hell to get millennials to answer their cell phones to get a representative sample. But there’s more. Do you pay attention to whether someone says they’re going to vote, or if they actually voted last time? How do you weight the sample between genders, ages, ethnicities, party/independent registrations?
That’s all a somewhat educated guess. If the shape of the poll and the form of the electorate differ by too much, the survey results will mislead. And you don’t know if you were correct until voting starts. In a general election the world finds out at once. Primaries and caucuses take place over extended time. After each set of results, pollsters move their expectations of who will show up.
Gravis did this. Their survey taken in New Hampshire, immediately after Iowa, had Bernie up 16. His poll numbers among other organizations measuring the Granite State had not moved by very much in the intervening couple of weeks. What changed was how a pollster interpreted their internal results before distributing to the public.
The final RCP average was Sanders +13. He won by 22. Gravis was less incorrect than the median pollster. Most organizations skipped Nevada. South Carolina was the next contest with plenty of polling sample.
Gravis published a result for South Carolina right after New Hampshire voted. By now, they’d learned Bernie does better than they would expect. So they adjusted even more. Their published survey had Hillary +18. This was the most pro-Sanders poll of any shown by RCP.
The final RCP average had Clinton ahead by 27.5. She won by 47.5. Gravis was wrong by almost 30 points. After regularly underestimating the Bern, they gave him way too much credit. Other pollsters made smaller versions of the same error.
Except one. Clemson University released a late poll. It was their first attempt in several months. They didn’t survey other states (Clemson is in SC). What did they find?
Nobody else had her ahead by more than 29. The outlier was the only correct result. They had a scenario. They stuck with it regardless of what others were publishing.
Gravis was the most bearish on Bernie the first time they published in South Carolina. They had her ahead by 70 in the summer of 2015. It turns out they were on to something. If you adjust where the national race was at the time of that poll, and figure what you’d have expected in February 2016, the result is very close to what South Carolina voters did.
Clemson’s first poll was one of several surveys taken between mid-October and mid-November 2015 that gave Clinton a massive lead. Their average was exactly what transpired a few months later. But among the pollsters who released results in February, Clemson was the only one to publish something in that range again.
Gravis was first to spot Hillary’s extreme strength in South Carolina. They and Loras correctly saw Bernie was susceptible to big defeats when the pool of voters turning out weren’t ideal for him. Just because they were way off in Iowa didn’t mean it couldn’t happen elsewhere.
So Gravis made two wrongs. Too far one way in Iowa, too far the other way in South Carolina. But it masked a greater right they uncovered. Clinton had a huge advantage among African American voters over 35, and all Democratic voters over 60. Not only would this plague Bernie in southern states, but he also underperformed in California and New York.
When the electorate was young enough and/or white enough, particularly in a caucus state that favored the candidate with motivated volunteers, he excelled. Otherwise he lost. Often badly.
I’m not sure the polling rights and wrongs mattered that much in 2016. While individual state polls were regularly wrong, the national polls were often correct. Hillary won about 10% more of the national vote. Her final RCP average was 11%.
It will matter way more in 2020. This isn’t a two-person contest. Early poll numbers impact debate qualification. Push funding numbers. Surprises in early voting states will remove some candidates and launch others forward. If 2016 is any indication, relying on the state poll averages as being more accurate than the outliers is dangerous. Especially if pollsters themselves begin herding their results to cover for previous errors. Even if those errors were just noticing the right thing at the wrong time/place.
There’s no earthly reason to think Pete Buttigieg was really at 25% in Iowa the week after the debates. His previous best number there was 17%, prior to Kamala Fest, prior to the white South Bend officer shooting a black citizen under very suspicious circumstances. Also, both best numbers are from Change Research.
They’re rated a C+ pollster by FiveThirtyEight. The 10% is from David Binder Research. They don’t have a rating. Suffolk University is a B+. This was the 6% result. All three surveys were taken at the same time. RCP only shows the Suffolk survey. This is the only one that counts for debate qualification.
Mayor Pete’s next best Iowa numbers are both 14%. One from Selzer & Co (A+) in early June. The other from our pals at Gravis (C+) in mid April. Nationally, Buttigieg is at or a bit below his early June numbers. He’s doing noticeably worse than in mid-April.
You’d expect both of these to wind up similar or lower if they took another survey now. Remember, the average of the most recent surveys is 13.7%, with one pulling the other two up significantly. Best logic would indicate he’s at 11-12% in Iowa right now. The data points us here from all sides.
But that 25% number, one arrived at by stacking the deck in Buttigieg’s favor:
A. More of his type of voter—educated and upper income—his skew toward white voters helps him in all Iowa surveys
B. Where two thirds of the voters considering him make him their first choice.
This may not be what will happen. It is something that can happen. Instead of viewing these surveys as a prediction, or even a measurement of where things are, it’s often better to see them as roads voters can potentially travel next winter.
And, just because the road isn’t taken by Iowans doesn’t mean it won’t get used by Californians, or Texans. In the particular case of Mayor Pete though, either Iowans or New Hampshirites need to fall for him. Even Change Research has him at 6% in South Carolina.