Most experimental drugs fail before they make it through all the tests required to figure out if they actually work and if they're safe. But some drugs get fairly far down that road, at the cost of hundreds of millions of dollars, based on poorly conducted studies at the outset.
Medical researchers reviewing this sorry state of affairs say the drug-development process needs serious improvement.
Consider drugs that are being developed to treat ALS, or Lou Gehrig's disease. In the past decade or so, nine potential drugs have been tested in people who have this degenerative nerve disorder. Not one has been effective.
So Steve Perrin, who runs the ALS Therapy Development Institute in Cambridge Mass., decided to take a close look at the mouse studies that had initially suggested these drugs were promising.
"We tried to replicate those findings precisely by talking to the authors and trying to repeat the experiments in an identical fashion," Perrin says. "And what we found was that we couldn't replicate a lot of the experiments."
Perrin says that if scientists had been more careful with their initial mouse studies, they would have realized that these drugs were never good candidates and that it made no sense to try them in people.
Scientists who did those early studies would have known that — if they had done the studies correctly, Perrin wrote in the journal Nature. Instead, it ended up wasting a huge amount of time and potentially hundreds of millions of dollars.
It's also a rough deal for the patients who have diseases like ALS, which can progress rapidly to paralysis and death, if they choose to try an experimental treatment.
"Patients have one shot on goal," he says. "If it doesn't work for them, they've lost that one shot. They might not qualify for the next trial that looks promising, or they may lose their battle with the disease."
And this is not just a problem for ALS. It's true for cancer research as well.
C. Glenn Begley worked at the drug company Amgen in Thousand Oaks, Calif., to develop cancer drugs. He also noticed a lot of faulty animal studies in the early stages of drug development. His rule of thumb was that 60 percent of them were no good.
But when he actually sat down to review more than 50 studies that had come in over the transom, "I was frankly shocked to find that the number was more like 90 percent of papers that we were unable to reproduce."
Because Amgen and other big pharmaceutical companies know that many animal studies are dubious, they always redo them before deciding whether to go ahead with human studies. So the company did not end up wasting hundreds of millions of dollars or years on a dead end.
But that's not always the case.
"For small companies that have limited resources, they probably do go ahead with studies that ultimately will burn out," Begley says. (Since he did that analysis, he has moved to Tetralogic Pharmaceuticals in Malvern, Pa.).
Begley says this problem doesn't apply to drugs that make it all the way through the approval process. But these early errors clearly slow the search for new drugs. He says it's not that scientists doing these tests are deliberately sloppy.
"Fundamentally, it's the way we as human beings operate," he says. "We really want results to be positive. We really want to discover something that's going to improve human health."
So scientists are too eager to seize on hopeful results, and not skeptical enough about their own work.
Drug studies in human beings take that into account by making sure scientists running the tests don't know who's getting the drug and who's getting the comparison pill. Begley says that same standard should be applied to animal studies. Scientists doing this research need to be rewarded for getting it right, he says, not just for getting it published.
It will cost more in the short run to do it right, but that will avoid wasting time and money in the long run, Perrin says. "We can't possibly keep living under the same system that we have been for the last decade or so."
Copyright 2021 NPR. To see more, visit https://www.npr.org.
