I have long thought that the “Yes” vote in the Scottish referendum, all other things being equal, would be around 35-38%. This is a rough guesstimate based on how people say they will vote, minus those who drift away from change late in any such campaign. Essentially, it assumes that the “don’t knows” will break heavily for “no”, given the uncertainty factor – a classic example being the (frankly much less life-determining!) Australian Republic referendum.
Polls move up and down. As Mr Salmond is a master operator, “yes” was always likely to gain ground slowly. Here’s the real thing: what people tell pollsters is very different from what they actually do in the privacy of the polling booth! Years of practice by pollsters have seen a system of “weighting” developed which means they can predict UK General Elections almost precisely. However, no such comparator exists for a one-off referendum of this nature. There is no truly reliable system of “weighting”.
Thus, nothing I had seen in the polls swayed me from that view. Until I was pointed to one in The Scotsman by my cousin, who lives in Falkirk. This one, known as the ICM Wisdom Index, was almost certainly the most reliable poll I have seen on the subject – and it put the “Yes” vote at a staggering 47%.
It is an apparent outlier versus my own analysis and indeed any other reliable polls, but here is why it must not be discounted. The poll did not ask how people intend to vote; but rather what they think the outcome will be. Polls asking what they think the outcome will be are vastly more reliable the ones asking people how they will vote. There are lots of reasons for this.
When asked by pollsters how they themselves will vote, the information received merely tells them what that single person likes to think they will do (or even what that single person thinks the pollster would expect them to do or admire them for doing). It does not tell us what that single person will do; nor does it tell us any more that one person’s view.
On the other hand, when asked by pollsters what they think the overall outcome will be, people can in the first instance be more honest – this is not about them alone any more, but about their reading of the overall population. More importantly still, it does not give us an indication of what a single individual will do, but rather what they perceive their entire social circle, community and even country will do – essentially, it gives us information about lots of people for the price of one.
Polls asking people what they think will happen at an election or referendum have been demonstrated countless times to be more accurate that polls asking people what they think they themselves will do (except where decades of tested weighting may be applied, which is not the case for the Scottish referendum) – for example, unlike “normal polls”, the same ICM Wisdom Index consistently and correctly ignored “Clegg Mania” in the run-up to the 2010 UK General Election.
Still, it is not perfect of course. Proponents of the ICM Wisdom Index thought the AV referendum would be close (it wasn’t remotely). There is still the element where the respondent does not want to be thought a fool by the pollster. It is safer to tell a pollster that you think it will be a close vote in favour of the likely winner than to say you think the underdog will win or that the likely winner will win big – hedging on the close vote means you won’t be too wrong either way!
Interestingly, this is why bookies are often more accurate predictors of elections or referendums than pollsters – as bookies base prices based on bets placed by punters on what they think will happen collectively rather than on what they will do individually. On this basis, most bookies still have a “yes” vote at 7/2 – which is more in line with my initial instinct that the “yes” vote will be closer to 37% than 47% – to be precise, the bookies’ spread reckons a “yes” vote at 41%. But on the above evidence, if I were a betting man, at 7/2 I’d be tempted…