• Sat
  • Dec 27, 2014
  • Updated: 2:09pm

Why so many medical 'breakthroughs' turn out to be phantom findings

PUBLISHED : Sunday, 02 February, 2014, 2:07am
UPDATED : Sunday, 02 February, 2014, 2:07am

After years in which a handful of scientists raised the alarm about biomedical studies that cannot be independently confirmed, top US science officials are acknowledging the crisis of "irreproducibility".

The problem suggests its possible much of the research literature has been compromised by widespread incompetence or downright deception.

"We have to take this seriously," Dr Francis Collins, director of the National Institutes of Health (NIH), the country's largest funder of basic biomedical research, said. He cited the waste of time and money spent trying to build on studies whose results are a mirage or whose methods are too opaque for others to follow.

In an essay in the journal Nature, Collins warned that "the checks and balances that once ensured scientific fidelity have been hobbled" and outlined steps NIH will take to combat the "non-replication" problem.

US President Barack Obama's Council of Advisors on Science and Technology recently held a meeting on reproducibility. The council heard from researchers who have investigated the crisis and from journal editors.

Collins said high-impact journals "are looking for papers that are groundbreaking and surprising" - the kind most likely to report results that turn out to be phantoms. Pharmaceutical, biotechnology and medical diagnostic companies have become increasingly concerned about scientific papers whose claims hold the promise of leading to new medications or diagnostic tests, but that they can't reproduce.

Scientists at biotech pioneer Amgen, for instance, reported in 2011 that they could confirm only six of 53 landmark studies in cancer biology. Researchers at pharmaceutical giant Bayer announced in 2012 that only 14 of 67 attempts to confirm claims in oncology, women's health and cardiovascular disease succeeded. Officials at Novartis and AstraZeneca told a recent cancer meeting they found the same problem.

"Whether in diagnostics or pharma, you hear the same story," said Mary Lou Gantzer, chief executive officer of BioCore Diagnostics: "'We saw something interesting in a paper, and we couldn't reproduce it.'"

NIH is now encouraging journals to devote more space for scientists to detail the methodology of their experiment. That way, there is no excuse for omitting mention of a crucial step or "secret sauce".


For unlimited access to:

SCMP.com SCMP Tablet Edition SCMP Mobile Edition 10-year news archive



This article is now closed to comments

Having been in the clinical area for over 30 years and worked at great hospitals and bad hospitals I have seen the inability of clinicians to follow a clinical protocol that was not theirs to the letter and often doctors do not realize the quality of care that the leading centers employ at all levels of care to get the results they wish to get. I recall in the 1980's that The Beth Israel Hospital in Boston was spending 50% more on each patient for quality outcome. No one discussed this too much because the doctors there clearly knew they could get what they needed to do the job right. That is not the case in many municipal and for profit medical centers. How could you compare Brigham and Women's Hospital and Bellevue Hospital for cancer outcomes? It is not a "secret sauce". It is quality care after surgery, after and during chemo and radiation and the ability to treat people with the exact technique not the following institutions take on it to produce similar results. The field is light years away from understanding this. They just don't want to challenge the bean counters in the US to allow them to staff properly and buy equipment to copy the leads technology to a tee. The administrations of the US healthcare systems are more about billing systems then medical systems so for the doctors to take them on to change the landscape is professional suicide. I am not a doctor but I have manged departments and seen this first hand.


SCMP.com Account