The Tale of The Missing Trials

In this blog I thought I’d tell the tale of the missing trials. I had no clue about this until coming across Ben Goldacre on TedTalks, leading to a mild obsession with his work. I would highly recommend his work to everyone, it really does open your eyes and is accessible for the non-science buff. As you will see, this issue has been neglected for decades and does real harm to patients. I will, however, give some glimmer of hope (just to make sure it’s not all doom and gloom) based on recent progress from the World Health Organisation (WHO) that came earlier this year.

There are currently 257,799 clinical trials underway across the globe in over 200 countries. Huge numbers of volunteers, scientists, clinicians and administrators are working hard to test new medicines for almost every disease you can think of (the full directory can be accessed here). The pharmaceutical industry is pouring billions into these trials and other research efforts every year, most estimates conclude it costs around $1 billion to develop a new drug from start to finish, and this figure is on the rise. These trials have the potential to improve millions of lives and positive results, rightly, generate excitement. These trials are quickly published in the big journals and seized upon by the media. Just Google “wonder drug” and you will be inundated with some encouraging recent outcomes and some exaggerated hype!

But what about the unsuccessful studies and trials? The trials that show no effect, or little effect, or that the treatment is no better than the others that are available? What happens to the data from those trials? The simple answer is that they just don’t get published. It goes missing. Sometimes it is actively hidden by pharmaceutical companies and the European Medicines Agency. I don’t have time to run through all the various legal battles people at the Cochrane Collaboration and AllTrials have fought to gain access to this missing data but it certainly is not easy to get hold of some negative results.

Credit: AllTrials Campaign (https://f1000research.com/articles/5-2629/v1)

Knowledge of publication bias (the technical term for the tendency to not publish negative results) stretches back to the 1980s. One recent study looked at all the Food and Drug Administration (FDA, US) approved clinical trials conducted on anti-depressants; 36 of these were negative and 38 had positive outcomes. Of the 36 that had negative results, 3 were published and of the 38 positives, 37 were published. This should worry you as it means a hugely distorted picture is presented not only to the public, but to our doctors! (As a bit of a funny side note, there have been studies looking at publication bias in the publishing of studies looking at publication bias).

Just to reiterate: if trials aren’t published they can’t be read by you and me and, perhaps a little more importantly, by the doctors that prescribe us medicines!

Lorcainide. You’ve probably never heard of this drug; I certainly hadn’t. In 1980, a tiny trial was done looking at the effectiveness of lorcainide as a treatment for people who have had a heart attack.  The study found that lorcainide didn’t have much benefit and there was an increased rate of death in the treatment group compared to the control (this was thought to be down to chance). This trial wasn’t published. In the next decade, drugs similar to lorcainide were developed, known as antiarrhythmics, and licensed – they also caused an increased rate of death, but this wasn’t realised until 1989. The result of the first study going missing in action was an estimated 100,000 unnecessary deaths in the USA. The lorcainide study was eventually published in 1993 with the authors recognising it as a stark example of the dangerous effects of publication bias.

Credit: James Lind Library (http://www.jameslindlibrary.org/cowley-aj-skene-a-stainer-hampton-jr-1993/)

So, what has been done? And what is being done? The FDA Act passed in 2008 in the USA required that all registered trials must be published within 12 months. This was widely hailed as having fixed the problem but there are 2 main issues. Firstly, this does nothing to address all the thousands of hidden trials conducted before 2008. Secondly, the pharmaceutical industry effectively shrugged their shoulders; just 1in5 trials (168/738) were published within the legal time period.

Now for the hope. The WHO issued a statement – with 17 signatories including the Wellcome Trust and the Gates Foundation – that outlined a plan to tackle this huge, systemic problem. Crucially the WHO statement covers all trials and is clear on where results should be reported – in peer reviewed academic journals.  It says all trial results must be reported on the registry where the trial was registered to allow for effective, quick auditing. If these processes are followed, and real penalties imposed for non-compliance we could have reached a turning point.

The tale of the missing trials is currently at a bit of a cliff-hanger, let’s hope it ends with a happily ever after.

Credit: Journeying Beyond Breast Cancer (https://journeyingbeyondbreastcancer.com/2014/12/29/a-rough-guide-to-spotting-bad-science/)