The Social Affairs Unit

Print Version • Website Home • Weblog Home


Use the buttons below to change the style and font size of our site.
Screen version     Print version:   
February 01, 2007

Confessions of a pollster - Harry Phibbs explains why working for a polling company as a teenager has left him with a deep scepticism about opinion polls

Posted by Harry Phibbs

Ever since working for a polling company as a teenager, Harry Phibbs has been sceptical about opinion polls.

As a teenager I used to work for an opinion poll company, NOP Market Research, whose offices were in Southampton Street, just off Covent Garden. I think I was paid 3.00 an hour plus unlimited free coffee and often I would ring up names from sheets torn out of telephone directories. Usually I was working at weekends or in the evenings and so mostly I would be conducting an "omnibus survey" where several topics would be thrown together in one survey thus lowering the cost for the clients because of economies of scale.

Perhaps every tenth name would be contacted. After a while we would be given notes saying "No more C2s" or "No more women" or "No more over 65s". This was to ensure those taking part in the survey were representative. To achieve this, once a quota had been reached no more could be included. This meant the start of the conversation could be very awkward. Naturally a woman answering the phone might take offence if I asked to speak to her husband on the grounds that her opinions were of no relevance. Or that a pensioner willing to take part was turned down. Naturally, one did ones best to explain but there often remained a residual tone of suspicion and resentment on the other end of the phone line.

More frequent, however, was the opposite problem of people not wishing to take part. Among unrecorded but typical responses pollsters get are:

I'm just trying to get the kids to bed.

Is that you Malcolm? Stop messing about.

Do I get paid?

Can you call back another time?

How did you get my number?

Often it would occur to me that the clients lacked any sense of reality in expecting a typical consumer to show such a detailed interest in their product:
And would you say are very satisfied with the colour of the packaging, fairly satisfied, neither satisfied not dissatisfied, fairly dissatisfied, or very dissatisfied?
This experience means when it comes to political polling I have always been suspicious of those who claim that polls underestimate the support of Conservatives because Conservatives are ashamed of their allegiance and lie or refuse to answer.

If the polls underestimate Conservative support it is more likely to be because Conservatives refuse to answer the survey at all. Often for Conservatives other matters aside from politics take precedence. They want to go to Church, or spend time with their families, or make money, or go hunting or read a P. G. Wodehouse novel.

Conservatives might be more busy but also more inclined to regard such matters as private. Those on the Left perhaps tend to be more pushy about their beliefs - more inclined to go on demos and wear badges - and also tend to be less hard working and thus have more time to answer questions from random callers.

But whether Conservatives are dishonest and ashamed or hard working and reserved, the polls raw data underestimate their number and a variety of adjustments are now put in place to correct for this.

The real shy respondents to polls are the shy don't knows. Rather than admit to being "don't knows" they will often plump for any middle option, especially if it represents the status quo. Ask the question about anything: Should we have more, or less, or leave things as they are? The leave things as they are option will hoover up mass support from those who don't have a clue about the topic you are enquiring about.

One of the things Party officials like to examine is the breakdown of how their support is made up. A leaked memo from Labour Party Chairman Hazel Blears warned that the Tories under David Cameron have opened up a huge "gender gap" with a big lead in opinion polls among women voters.

By definition this also leads to some equivalent concerns for the Tories who have been preoccupied by gaining more support from women. What are the Tories doing to close their gender gap amongst men? Perhaps they need an A List of male candidates to reach out to British males and show their concerns will be met? More pictures of David Cameron drinking pints of beer in pubs?

A further note of caution over polls is that it is easier for people to say they would buy a new product or vote for a political party than to actually do so. When it comes to elections this is another factor that seems to help the Tories, the more so with Council elections and Euro elections when there is a lower turnout. I remember during the 1999 Euro Elections The Guardian had a poll specifically asking people how they were going to vote in those elections shortly before they took place. The poll showed Labour 22% ahead. In the actual election the Tories were 7% ahead.

A more recent example of opinion pollsters being gloriously confounded was over the referendum to establish a regional assembly on the north east of England. It was defeated by 78% to 22%. The opinion polls had indicated huge support. One showed a 51% Yes vote, 19% would vote No, with the rest undecided.

The sheer staggering scale of these examples show how the pollsters can get it so wrong that the caveats are really washed away. We hear about there being a "margin of error" of two or three per cent either way. We hear about polls being a "snapshot" not a predictor and how the situation can change with a subsequent shift of opinion. In other words the claim is made that they were accurate in stating the opinion of people at the time - but that we changed our minds before polling day.

So are polls useless? Not entirely. I think the interesting part is the trend. Even if the methodology has changed, the trend will still be interesting. Ignore the bit that says

the Tories are five points ahead.
Listen out for the bit that says:
They are three points up on last month.
Harry Phibbs is a journalist.


Comments Notice
This comments facility is the property of the Social Affairs Unit.
We reserve the right to edit, amend or remove comments for legal reasons, policy reasons or any other reasons we judge fit.

By posting comments here you accept and acknowledge the Social Affairs Unit's absolute and unfettered right to edit your comments as set out above.
Comments
Post a comment








Anti-spambot Turing code







Creative Commons License
Except where otherwise noted, this site is licensed under a Creative Commons License.

The Social Affairs Unit's weblog Privacy Statement