The ritual dance of the annual company survey

Human beings have, on average, fewer than two legs. This may seem like a startling thing to read in a newspaper, but it says more about the fallibility of statistical analysis than it does about the distribution of limbs on human beings. Because there is at least one person who actually has fewer than two legs, and no people with more than two, the average number is less than two. Not much less, but less.

This is one of the great things about statistics - accurate and interesting remarks that are completely meaningless.

Once every year or so, our bank illustrates this point by conducting an employee survey. A consultant is hired to find out things about morale and how pleased everyone is to be working there. We answer multiple-choice questions about things like whether 'the bank respects me as an individual' or 'the management of the bank understands my concerns as an employee'.

Our answers are given in shades of agreement between 'strongly disagree' and 'strongly agree', including the enticing option of 'no opinion'. At the end of the process, the consultant will present results that show that the staff are generally happy and think that management are doing a good job.

There are three reasons that the results always come out this way. First, people who can't stand their jobs and think management are a bunch of monkeys generally don't bother filling in surveys. Second, despite assurances to the contrary, there is always a lingering suspicion that our answers might not be completely anonymous, so we tend to give favourable responses. And the final and most important reason is statistics itself.

Statistics is a very complicated field, one which most people don't realise that they don't understand. The ways in which the results of a survey can be misinterpreted or even misrepresented through statistical analysis are endless.