How to Read a Political Poll Without Being Had
A thousand people, a three-point margin of error, and a headline written like a photo finish. What the numbers under the numbers actually say.
Every political poll you will see between now and election day is a survey of roughly a thousand people, dressed up as the voice of five million. That is not a scandal. Done properly, a thousand-person sample is genuinely informative, and the polling companies are mostly careful about how they do it. The trouble starts after they hand the numbers to a headline writer.
Start with the margin of error, which the fine print will state as about plus or minus 3 points. That figure applies to the whole sample, at the standard level of confidence, and it means a party reported at 4.8 percent might really be at 3.9, or at 5.7. Notice what that does to threshold drama. A minor party hovering “just under 5 percent” is not just under 5 percent. It is somewhere in a fog bank that happens to contain the number 5, and the poll cannot tell you which side of it the party is standing on. The margin gets worse again for subgroups, so any confident sentence about what women under 40 or rural voters think is being carried by a couple of hundred respondents at best.
Next, the movement problem. A party goes from 34 to 32 and the coverage reports a slump, a blow, a warning shot. Two points is inside the margin of error. Statistically, nothing is known to have happened. The honest headline would be “Party’s Support Somewhere Between Fine and Slightly Less Fine, Probably,” which is why you will never see it. The signal is in the trend across many polls from different companies over months, not in the twitch between one poll and the next.
Two more habits of the trade are worth knowing. Polls exclude the undecided before calculating the percentages, and the undecided are routinely one voter in ten or more, so every party’s reported number is inflated by the removal of the people who might yet decide against them. And the preferred Prime Minister question, beloved of news bulletins, measures name recognition and incumbency more than anything else. It has never been a reliable predictor of an election result. It is a familiarity contest wearing a leadership costume.
None of this makes polls worthless. It makes them instruments with a stated tolerance, like a bathroom scale that is honest to within a kilo. The pollsters publish the tolerances themselves, in the methodology notes, which is more than can be said for some of the coverage built on top of them. So read the average, not the outlier. Read the trend, not the twitch. And when a bulletin announces that the government would or would not be re-elected “if an election were held today,” remember the one thing known for certain: it is not being held today. In a system where 5 percent is the difference between a caucus and a hobby, the difference between what a poll says and what a headline says it says is worth your attention.