The problem with opinion polls, in Hong Kong and elsewhere

The Conservative Party's wide, clear-cut margin of victory in the British general election surprised many people as the party claimed its first outright majority since 1992, despite what many opinion polls were predicting. The inaccurate result has led the British Polling Council to set up an inquiry to find out why the final polls were so wide of the mark after pollsters dramatically understated the Conservative share of the vote.

There have been suggestions that it was due to the "shy Tory" effect; voters were perhaps unwilling to admit publicly they supported the party and, indeed, online pollsters routinely found higher levels of support for the Conservatives than did telephone polls. Another possible explanation is that the Labour Party failed to turn those who pledged support into actual votes on the day.

Here in Hong Kong, the government is frantically trying to win over public support for its reform package, with a massive TV campaign, advertisements in the MTR, and officials on the ground appealing to the people. However, the latest joint university polling results seem to show support is waning - 42.3 per cent supported the package and 40.9 per cent opposed it. An earlier poll showed that 46.7 per cent supported reform and 36.7 per cent opposed it. It would seem that more people still support the package than are against it, but the gap seems to be narrowing since the government began its campaigning. But how much can we trust the results?

All such results are basically derived from phone polls. The polling organisation usually uses a standard, territory-wide phone sample. A computer randomly generates the numbers for pollsters to call. Within each household reached on a landline, an interview is sought with a person living in the household who is 18 or older. When respondents are selected in this way, at random, every adult has an equal chance of being part of the sample, which is typically about 1,000 adults. Sometimes, more detailed subgroup analysis is provided but with a smaller sample size, the accuracy of the results is reduced. Some weighting of respondents might be applied, for age and gender usually, to match the demographic characteristics of the adult population according to data from the Census and Statistics Department.

So, could the problems experienced in the UK also affect our polling results? Those contacted by phone may be less willing to admit they support the reform proposal. And among those who did express support for the proposal, would they actually cast a vote in a 2017 territory-wide election? One of our research studies on the post-80s generation, in 2011, showed that less than 40 per cent of those aged 18-30 had registered to vote and their turnout rate in the previous Legco election was less than 50 per cent. Even though a high proportion of young people oppose the government's reform proposal, according to phone polls, if a referendum were held today, I wouldn't be surprised to see different results to those in the poll.

Conducting a poll properly is as much an art as it is a science. The questions need to be simple and clear, with no ambiguity, and there should be no leading or loaded questions, in order to achieve an unbiased response. The timing of the survey is important, too. If a poll is conducted during the day, it can be difficult to get hold of members of the working population, who might have different views from those who are at home. The response rate is also very important, and anything less than 50 per cent should be viewed cautiously.