Damned lies or statistics? Use your brain
Mark Twain single-handedly helped discredit an entire mathematical discipline in the popular mind by saying: 'There are three kinds of lies: lies, damned lies and statistics.' There is little doubt that data is frequently manipulated by politicians, lobbyists and commercial interests for their own ends; and the media often accepts their claims uncritically.
Ironically, it was precisely during Twain's lifetime that modern statistics emerged, in the west at least, as a revolutionary science that laid the foundations for empirical observations in the hard sciences and transformed social philosophy into modern social sciences.
The new science of uncertainty teaches practitioners to correct for errors, eliminate biases, detect fraud and apply mathematical rigour to observational data. It is the opposite of what Twain had in mind. Ken Alder, a US historian, argues it helped create the modern notion of professional scientists, as opposed to gentleman-savants.
It is interesting in this context that a heated debate has broken out in the exciting new field of social neuroscience that turns on its statistical validity. These days, it is impossible to avoid reading in the news the latest findings from the field. By using a type of brain scan called functional magnetic resonance imaging (fMRI), its practitioners claim to capture images of brain activities that correlate with various emotions, conscious states, and mental activities, from memorising a poem to solving an equation.
Researchers have claimed to find mental differences between those who vote for Democrats and Republicans in US elections; and between men and women in experiencing jealousy, handling social rejection, and reacting to grief and anger. Areas in the brain associated with imagining will light up when we tell a lie; others when we make moral decisions. Social neuroscience promises much and seems, on the surface, to deliver much. Alas, MIT graduate student Edward Vul and his supervisors have recently dropped a bombshell, raising questions about the goals and methods of the new science; how the public can learn or be misled by its claims; and the use and abuse of statistics in science in general.
The controversy hinges on what those brain correlations really mean, if they exist. The resolution of digital images is defined by pixels; that of fMRI is by voxels, which cover a tiny volume of brain tissue. The brain scans 'light up' by the amount of haemoglobin in the blood, whose flows indicate mental activities. They are, if social neuroscience is right, objective data of ephemeral subjective feelings or mental states.