Repeatability of clinical trials discussed at Economics Seminar
By Jenn Tyborski
There’s a reason most pharmaceutical drugs on the market have a million side effects — most published research results are false, according to Borko Jovanovic, associate professor of preventive medicine at Northwestern University.
Hosted by the Roosevelt University Economics Seminar Series, Jovanovic explained why he believes most published results are not true. Within clinical trials, the issue is a large controversy, as researchers want to be the first to document a finding, even if it means hiding negative results.
“We want to be successful,” Jovanovic said. “We want to have interesting results, but we’re bound by these errors which we admit from the very start.”
Throughout the lecture, Jovanovic provided numerous examples to explain his argument. One example was an essay from 2010 by Jonah Lehrer, who used the Eli Lilly’s Zyprexa case on schizophrenia.
Initially, people who took the drug demonstrated a large drop in symptoms. However, as the drug was re-tested by others, “the effect seemed to be waning across the board.”
“We pride ourselves in medical school that we’re special and better than others, so if it happens occasionally it’s an honest mistake,” Jovanovic said. “Unrepeatability has become an issue, and some [scientific results] are repeatable, but not all are repeatable.”
Throughout the discussion, there appeared to be an overarching theme. Clinical trials only last about two to three years at most, according to Jovanovic, and at the end of that time, people — and the government — want to know those results immediately.
This impatience for a drug to be fully tested is why we often see the effectiveness of the drug decrease over time. Jovanovic cited three main points for the decrease in effectiveness.
The first point was the claim that drug research was conclusive with only a limited number of subjects, and only had an initial error margin of five percent.
Second, negative research is not reported because of investigators and journal editors.
And last, research contradicting the paradigm of the moment is largely absent.
Stephen Ziliak, professor of economics at the university, stepped in to help explain Warren’s ideas to students.
“What he’s talking about is a problem in the statistical sciences,” Ziliak said. “What they’re doing is confusing statistical significance with medical or biomedical or economic significance. Statistical significance is neither necessary nor sufficient for proving what Dr. Jovanovic is calling a ‘positive’ result.”
Why rush the results then, when researchers know there’s a strong chance they’re wrong? According to Jovanovic, one incentive is for the researchers to publish first and pressure from the government. When a trial is registered through the clinical trials government website, it is tracked from start to finish.
As Jovanovic puts it, once you register, you’re in it for life. Upon the reported completion of the trial, especially if your trial is funded by government grant, they’ll want results.
“If you finish a trial, they’ll go after you and say, ‘Show us the data. Show us the data,’” Jovanovic said. “If you don’t like the data, you walk away from it — move to North Carolina. They’ll find you. Point is, you have to answer to the outcomes — whatever they are.”
Attending the seminar was first year graduate student Sonali Gandhi. Although she was attending for extra credit in a class, she had interest in the topic as a future medical student.
“It was interesting,” Gandhi said. “I didn’t think about this part before, just getting a small sample size just to get the results you want to get published. I learned a lot.”
Although Jovanovic teaches clinical trials — his only class — he spends most of his time working with physicians, fellows, residents and “people who are pretty accomplished, who have beepers [and] two or three phones.”
The lecture gave Jovanovic the opportunity to speak to a group for a change. According to Jovanovic, he “missed talking.”