Politicians as well as journalists take political polls very seriously. Losses and gains of a few percentage points are overly analysed and commented on.
This interactive explanation shows you why polling results should be taken with a grain of salt. In a lot of cases decimals will prove to be meaningless and changes of less than 2 percentage points shouldn't be subject to much analysis.
And all of this without a single statistical formula. That is a promise.
Imagine PollLand, an imaginary country. PollLand is a democracy, has a population of 1 million and has 8 political parties. Every 1000 PollLandians are represented in the graphic on the right by a circle. PollLandians don't like sharing their political preferences.
But on election day, everyone in PollLand shares his or her political preference.
Nice! As you can see the political landscape in PollLand is fragmented. It's almost like a Jackson Pollock painting!
The votes are in, let's take a look.
Ok. That makes it a little easier to count the votes.
The Reds party was the clear winner of the elections in PollLand, with almost 32 % of the votes.
Elections are only held every couple of years. To measure trends in political preferences in between elections, speciallized polling companies conduct surveys, often commissioned by newspapers and tv stations.
Let's simulate that. At the same time we conduct a thought experiment: we assume every PollLandian still has the same political preference as during the last election. So the ideal poll should generate results equal to the elecion results.
Ok, let's go!
The polling company asks people for which party they would vote if elections would be held at that moment.
With the button below, you can add the answer of one interviewed PollLandian to the poll. After the first ten answers are in, our experiment can continue.
All right. But you noticed that surveying is hard work. Let's take a better look at the results so far.
The first results are in, but as you can see there are big differences between the polling percentages and the election results. Let's focus now on these differences
In this experiment we suppose that political preferences at the time of the survey are equal to those on election day. Every difference between polling and election results is completely due to the chance and uncertainty that come with surveys.
Wow, those differences are quite large. Let's survey some more people to get more confident results.
Let's speed things up a little. With the button below, you can survey 10 people at once and their answers to the poll. This time we aim for at least 100 surveyed PollLandians. You can still add people to the poll one by one if you wish, to see the influence of adding 1 person on the polling results.
Nice job! Already surveyed 100 people. Did you notice that the differences between the polling and election results became smaller when you added more voters?
Let's take it 1 step further: now you can add 100 voters at once. This time we aim for 1000 voters in the survey, a number typically used for political polls in PollLandia and many other countries.
Great: you completed a whole survey! But significant differences between polling percentages and the election results persist: for this difference is still .
Despite all your hard work to survey 1000 people, significant differences between what you measured and the real political preferences among the PollLand population persist.
Those differences can be completely attributed to chance: even in a survey of 1000 people voters for some parties will be overrepresented and voters for other parties will be underrepresented.
We'll explore this a little further by conducting 8 polls at exactly the same moment, with exactly the same number of surveyed people, with the same method and among the same group of people. So it's reasonable to expect the results of all 8 polls to be the same. All deviations are completely due to chance.
Use the button below to generate 8 polls, of 1000 people each.
Nice. Let's check the differences with the real percentages among the PollLandians for all 8 parties.
Quite some differences, right? And all just the result of chance.
Of the 64 polling results, only were correct up to one decimal. So, giving importance to small differences in polling results is doesn't make much sense. Displaying decimals misleadingly gives the impression of accuracy.
times the deviation was greater than 2 percentage points, the largest difference was percentage point. Sheer chance can play a big part in seemingly big changes.
When reporting on polling results is done well, the expected error is mentioned (the 'margin of error' or the 'confidence'). Now you know where this is coming from: randomness generates noise in the polling results. So keep that in mind when you read or write about polling results.
Here are some useful links in case you want to visualize uncertainty in polling results: