The case of the missing neurological drug trials remains shrouded in mystery. Nearly 48,000 people took part in these trials for new drugs for multiple sclerosis, stroke, Alzheimer disease, migraine, epilepsy, insomnia, and Parkinson disease.
We don’t know what happened to them. That means we’re at least partly in the dark about the drugs they took, too.
More about what this case does reveal shortly. First, though, some perspective: our 48,000 people are the tip of an iceberg.
At least a couple of million people participate in new clinical trials of drugs each year [*]. They are making a massive contribution to knowledge. But a lot of that knowledge doesn’t reach the outside world.
Perhaps the best way to know just how much, would be using all trial approvals by ethics committees as the basis, with an extensive search for sources of data published a few years after they finish. But a 2014 systematic review by Christine Schmucker and colleagues only found 2 studies that concentrated analysis like this on trials – and those trials were in the 1990s [*].
There has been a lot of water under the bridge since then. The year 2004 was a game changer. That’s when the International Committee of Medical Journal Editors said they would only publish trials that had been registered, flushing many more trials into the open – at least partially.
I found one study of trials based on ethics approval and comprehensive search since that review. This one is only about drug trials. Out of 574 trials that gained ethics approval in 2007 in the Netherlands and actually started, 58% were published in journals, and 10% of the rest had results posted in public trial registries [2016]. The authors didn’t hunt for data reported in regulating or assessment agency reports.
Scary, though, isn’t it?
We know that the results of clinical trials are routinely, but also legally withheld from doctors, researchers and patients…[W]e’ve been failing to fix the problem for over three decades now.
Ben Goldacre (2015)
Goldacre campaigns about this through AllTrials and more.
But let’s go back to those neurological drug trials.
The new data this month comes from Amanda Hakala and colleagues, from the universities McGill and Ottawa. It builds on earlier work from 2015.
In this recent study, they compared publication rates for trials of completely new drugs that had been under development at about the same time. Trials go through phases, with early small trials exploring safety, progressing towards the randomized controlled trials companies need to get the drug approved by a regulatory agency like the FDA in the US or the EMA in Europe. In several countries, after that there is another evaluation by a different agency to determine if the drug will actually be recommended for clinical use and publicly funded.
Hakala looked for new drugs for 7 neurological conditions approved by the FDA between 2005 and 2012: there were 8. Then they looked for drugs that they called “stalled”. These were drugs that were traveling the same path at the same time, but hadn’t been approved by the FDA. To be in that group, the drug had to have reached that later stage of randomized trial (phase III) needed for an FDA application in the same period. There were 28 of them.
They found 163 registered trials for the approved drugs, and 203 registered trials for the “stalled” ones. The successfully approved drugs were far more likely to have published some results: 90% of those trials were published versus 54% of the trials for the “stalled” drugs. Yikes! [Update] Only some of that is full published papers, it was worse. One of the authors, Jonathan Kimmelman, wrote in a comment on this blog post:
At 5 years follow-up after trial completion, the proportion of “licensed drug” trials that were published was 55%. For “stalled drugs,” the proportion was 22% (!!).
Even when a drug gets abandoned, we need to know what the trials found. Safety and effectiveness information is obviously critical, since someone else might try to develop the same drug or one very similar to it. But that’s not the only reason: we can learn more about the weaknesses of theories about drugs and the disease, and weaknesses in how we study them [2015].
What’s more sometimes, a drug that fails for one purpose turns out to be important for another: think of AZT, for example. It was developed for cancer, but abandoned in the 1960s because it didn’t work. But it became the first drug used to fight HIV/AIDS.
Opicapone is an interesting case in the neurological drug trials study. It may be “stalled” in this study because it’s not approved by the FDA. But it was approved in Europe in June 2016. It’s for Parkinson disease, and it was developed by a Portuguese company.
Hakala found 20 trials, but they don’t point us to them or the published articles [but that’s now on its way]. My very quick and dirty look at the EMA’s assessment report [2016] found 16 trials getting at least a mention: 12 phase I trials, 2 phase II, and 2 phase III trials.
Opicapone information was also included by both of the European evaluation agencies I looked at: England’s NICE summarized the evidence, Germany’s IQWiG did a formal assessment (available in German only as of writing). It got a positive write-up from NICE and a thumbs down from IQWiG – with both making their decision based on the same one of the two phase III trials.
This cartoon addresses a key reason for these different perspectives on this drug:
NICE reports on the basis of comparison to placebo and non-inferiority or equivalence.
Non-inferior means it has to be “no worse” than something else that’s used for the same purpose. For Germany, the bar is set high on the quality, type, and amount of evidence needed to come to a conclusion. The drug has to add something useful to people’s options, especially if it’s more expensive – it has to have fewer adverse effects, for example, or be more effective for at least some people. [Disclosure: I worked at IQWiG in its early years, including participating in developing its methods and standards.]
For assessments like these, IQWiG is strict on making sure no trial relevant to their questions is hidden: if they can find any sign of a missing trial, the drug will be rejected for public funding in Germany. A powerful incentive for companies to reveal all.
And what about the US? The Portuguese company that developed Opicapone (brand name Ongentys) has sold US rights to a company that
…plans to meet with the [FDA] after a technology transfer to discuss getting the drug to market.
The companies are hoping to hold that meeting before year’s end [2016]…[A]nalysts expect the agency would require an additional clinical trial for Ongentys before greenlighting the treatment in the U.S., and therefore a new drug application wouldn’t come until 2020.
And FDA approval for new drugs now hinges on trials being registered in the registry, ClinicalTrials.gov – and results being available in the registry within 12 months of the trial finishing (unless an extension is granted). As of April 2017, these rules can be enforced, with a range of serious penalties possible.
The NIH now applies the rules to all trials it funds, too. The AllTrials campaign has been reporting more funders going in this direction this month: Canada and then a joint statement from 9 major non-government funders internationally.
Here’s hoping that 2017 proves to be a real game changer, too. We badly need to know more, not just about the drugs we use, but the ones that were abandoned, too. Over the years, millions of people have participated in trials whose results are invisible. That’s not what they signed up for.
~~~~
Disclosure: I worked at IQWiG, the German Institute for Quality and Efficiency in Health Care, from its start in 2004 to 2011.
I wrote a comment on PubMed Commons, encouraging the authors of the Hakala study to release more data. [Update 27 May]: Jonathan Kimmelman responded here, and the data will be coming – thanks, Jonathan! This post was updated in two places in response to his comment – firstly, on this question of data. And secondly, to correct an original statement relating to published papers – the data I included were for any public results, not necessarily full papers – and to add the data on published papers.
The cartoons are my own (CC-NC-ND-SA license). (More cartoons at Statistically Funny and on Tumblr.)
How many people sign up to new drug trials each year?
Here’s the basis for my statement that millions of people join new drug trials every year.
PhRMA, an association of drug manufacturers, estimated that 1.1 million people enrolled in 6,199 industry-sponsored drug trials alone in 2013.
Based on trials added to Clinicaltrials.gov (which aren’t only new trials and aren’t all the trials in the world), there were about 25,000 new trials in 2016.
A trial can have more than one type of intervention – for example, comparing a drug to a surgical procedure. As of 18 May, 2017, Clinicaltrials.gov reported this breakdown of 196,214 interventional studies:
118,353 Drug or biologic (60%)
58,401 Behavioral and other interventions
21,132 Surgical procedure
23,273 Devices
The average number of participants in the trials of new neurological drugs in Hakala’s study was about 130, and in Hakala’s 2015 study of a wider group, it was 501 to 760. The mean in PhRMA’s report was 177. Given how wide this variation is, a lot more data is needed to get a clear idea of how many people join drug trials each year.
60% x 25,000 x 130 = 1,950,000 people
60% x 25,000 x 760 = 11,400,000 people
The 2 older studies of trial publication rates based on all ethics approvals
One of those doesn’t seem to have differentiated trials that weren’t started from ones that didn’t finish and publish (trials in 1994 and 1995 [2004]). The other only looked at journal publication and relates to 451 trials between 1988 and 1998 [2008].
(Back to top)
* The thoughts Hilda Bastian expresses here at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.
Source: PLOS
