Can chocolate help you lose weight?

By Alan Yu

No, it can't - at least, it's very unlikely - but that didn't stop some media from reporting the results of a bad study

By Alan Yu |

Latest Articles

Here's a story to remind you not to believe everything you read, even if it's in a newspaper.

Last week, science reporter John Bohannon explained on a blog - at - that he did a scientific study - and did it very badly on purpose. He concluded that chocolate can help people lose weight, and then watched as reporters around the world accepted what he said was true and published his results.

This is important because you often read about scientific findings in newspapers and other media. Stories may say that salt, fat or sugar are bad for you, and so on. But how do you know you can believe what you are reading?

One way is to work out whether or not those findings are based on good research. In this case, reporters showed just how easy it is to get the media to write about research which is clearly bad.

Last December, Bohannon agreed to work with some documentary filmmakers about how easy it is to get people to believe bad science about weight loss. They did everything you would do in a study: paid people to try out different diets for three weeks, collected data, and tried to figure out if the diets worked.

However, it was a very bad study, as Bohannon explains. There were only 16 people taking part, and he says researchers do not take a study seriously unless it has at least 30 subjects. Even worse, if you take enough measurements and do not have a lot of subjects, you can get a false positive, which is a result that proves what you want - but only for that small group of people.

He compares the measurements to lottery tickets: if you buy enough tickets (or make enough measurements), you are more likely to get the result that you want to get.

Also, the scientific journal that published the study did not go through peer review, which is when other scientists look at the study to see if it was done properly. Bohannon had raised this issue before. Some magazines do not go through the usual review process, and will publish a study, even a very bad one, as long as you pay them. Bohannon is not an expert in this area of science, and if the journal had wanted to check that, they would have found out easily.

Finally, the journalists who reported the study did not talk to other scientists to check whether the study was a good one or not.