Accurate poll reporting -- I don't know

A comment on the Dim Post made me wonder what proportion of people polled in political opinion polls don't know, don't respond, or otherwise opt out of making an explicit call for a party.

Thanks to Lyndon Hood who pointed me at TV3 poll results, I was able to find the following data, which is very interesting.

Poll results as reported in headlines
  Feb 2009 April 2009 Aug 2009 Oct 2009 Dec 2009 Feb 2010 April 2010
Labour 27 30 29.3 27.2 30.8 29.6 33.8
National 60 56 58.1 59.9 55.2 56.3 52.1
NZF 1.6 1.3 1 1 1.5 1.5 1.5
Green 7 6.1 7.5 6.9 7.8 7.3 8.2
ACT 1.2 2.1 1.4 1.7 1.8 1.6 1.6
MP 2.1 2.9 1.2 2.4 1.7 2.4 1.9
Base 949 947 911 917 893 910 874

Poll results with percentages reflecting don't know and won't vote
  Feb 2009 April 2009 Aug 2009 Oct 2009 Dec 2009 Feb 2010 April 2010
Labour 25.6 28.4 26.7 24.9 27.5 26.9 29.5
National 56.9 53.0 52.9 54.9 49.3 51.2 45.5
NZF 1.5 1.2 0.9 0.9 1.3 1.4 1.3
Green 6.6 5.8 6.8 6.3 7.0 6.6 7.2
ACT 1.1 2.0 1.3 1.6 1.6 1.5 1.4
MP 2.0 2.7 1.1 2.2 1.5 2.2 1.7
Don't know/Won't vote 5 5.3 8.9 8.3 10.7 9 12.6

The first table shows percentages of the people who gave a preference. The "base" row is the number of people out of 1000 who did so. That's why the percentages sum roughly to almost 100%. I say "almost" because I omitted the tiddlers for brevity's sake, who never account for more even 2% of those who gave a preference in any of those polls.

That "base" row is very interesting. Lots of people aren't included in those headline percentages. In the last poll, more than 1 in 10 people didn't state a preference.

So I made the second table. It restates the data as percentages of the whole 1000 polled, showing those who responded don't know or won't vote as well.

What you can see is that if you report the results the way the news media typically do, it looks as though in April 2010 the coalition parties -- National, ACT and the Māori Party -- still had majority support. Their supposed percentages add up to 55.6%. But if you report the true figures, you can see that they've only got 48.6% -- a clear minority.

I think it's fine to omit the don't knows or won't votes if they're an insignificant number of people, but once they hit more than 10% it surely is worth reporting. For instance, in that last poll, there are more people who don't know or won't vote than there are Green, Māori, NZ First and ACT combined.

Apart from that, this gives a very misleading impression of support for the government. They weren't supported by a majority of voters at all.

As I find myself saying a lot these days, it's very disappointing, though not surprising.

Posted on Monday, May 31 2010

Tags: news mediastatisticspolitics

  1. From Giovanni on Monday, May 31 2010 #

    However 48.6% is still an absolute majority of the people who expressed a preference, and that's how the election works. So I think it makes sense to adjust the percentages, and then add the number of undecideds, as they do in the US. It's quite interesting that the number is currently at its highest - I wonder if it reflects a typical trend at this point of the electoral cycle. One would be tempted to read it as disaffection with the govt coupled with lack of any great enthusiasm for the alternatives on offer.

  2. From Giovanni on Monday, May 31 2010 #

    (When I say "add the undecideds", I mean note how many they were, not actually add them to the tally of course!)

  3. From Stephen on Monday, May 31 2010 #

    48.6% is still an absolute majority of the people who expressed a preference, and that’s how the election works.

    That’s a very fair point. I still think people are getting a skewed impression if the undecideds etc are left out though. Surely when they more than double, that’s a story.