Scottish independence polls send conflicting messages

Is the result of Scotland’s independence referendum on a knife-edge, or are pro-union campaigners comfortably on course to save the 307-year-old political union with England?

With less than four weeks to go until the referendum on September 18, the state of battle remains frustratingly unclear, according to Charlie Jeffery, vice-principal for public policy at the University of Edinburgh. “It depends on which polls are right,” Professor Jeffery says.

Opinion surveys are sending conflicting messages. While pollsters have consistently found support for Scottish independence trailing, they have differed greatly on the size of the pro-union campaign’s lead.

The latest poll by YouGov puts support for a No vote at 57 per cent to 43 per cent for Yes when “undecideds” are excluded – a relatively safe 14-point margin. But a survey completed the same day for Panelbase put the gap at a nail-biting 52 per cent to 48 per cent.

And judging which result is more likely accurately to reflect the Scottish electorate is extremely difficult, given the particular challenges posed by an event as unique as a constitutional referendum.

An expected high turnout means many people will vote on September 18 who would not usually be seen in a voting booth – or found on the lists of the internet panels that many pollsters use to judge the electoral mood. This makes weighting samples to ensure they are politically and socially representative of the electorate even more of an art than usual.

“Producing accurate figures is still akin to hitting a moving target while standing on the back of a moving pickup truck at dusk,” noted one contributor to PoliticalBetting.com.

In a sign of the difficulties and sensitivities involved, Peter Kellner, president of pollsters YouGov, issued a public criticism in July of the weighting techniques used by Survation, a rival polling group that had been reporting higher support for independence. 

At issue was the weighting of poll participants according to how they voted in past Scottish elections, an issue complicated by the fact that many people tend to quickly forget which party they backed.

Survation dismissed the criticism, defending its approach to weighting and saying the issues highlighted by Mr Kellner did not have much impact on its overall results.

But polling experts accept that representing the electorate in a referendum is a challenge.

“There remains a chunk of voters who flatly refuse to tell us anything at all on their electoral behaviour – and it’s these [representing about 10 per cent of the total Scottish electorate] who hold the true balance of power in this campaign,” wrote Martin Boon, director of ICM Research, in The Scotsman newspaper.

“It’s a tricky one,” says Ivor Knox, managing director at Panelbase, which has reported some of the highest levels of support for independence.

But Mr Knox says the difference between polling companies should not be overstated. It has narrowed over time he says and the pollsters using online panels have recently clustered near 45 per cent support for Yes when undecided voters are excluded.

In depth

Future of the union

Scotland will decide in a referendum to be held on September 18 whether or not to end the 307-year-old union with England

Further reading

“Given margins of error accepted at around 3 per cent, the four online pollsters are not really that far apart,” Mr Knox says, adding that it is likely that all the referendum surveys will start to converge as voting day draws near.

One approach to the uncertainty is to combine surveys into an aggregate result. The Financial Times’ Tracker Poll for example suggests support for Yes at nearly 43 per cent when undecideds are excluded.

Such an approach helps distinguish trend changes from outlier results, confirming, for example, that support for independence grew in the early part of this year before falling back again.

The latest three polls have suggested that support for the Yes campaign is rising again, though at a rate psephologists say may be too slow to secure victory.

But Mike Smithson, editor of PoliticalBetting.com, says punters should remember apparent consensus can be misleading.

“There are no precedents for this referendum,” Mr Smithson says. “Being on the average is not necessarily the best way to go.”