While researching my recent April Special Report on Neuroscience, I listened in on a webcast of a pediatric neurologist and patient advocate, which—as I quickly learned—was really targeted at lay media.
As I followed the conversation, though, I was struck by the vehemence with which the clinician explained the pivotal difference between over-the-counter cannabinoid extracts and prescribed medications.
Specifically referencing the placebo effect, he suggested the best way to avoid spurious results was to use “the gold standard of scientific methodology.” That is to say: randomized, double-blinded, placebo-controlled clinical trials.
I don’t doubt that his goal was to convey that the apothecary elixirs had not passed through the same scientific rigor of prescription meds, that people needed to exercise caution when considering salvation in a vaguely labelled bottle.
By the same token, I wondered how aware he was of the scientific mythology he was perpetuating to his lay audience.
Yes, it's true that randomized, double-blind, placebo-controlled studies are the gold standard. But a couple of decades of science and medicine writing have shown me there’s little “gold in them thar hills.”
And two recent publications suggest few clinical specialists find this issue problematic, including the oft-quoted FDA.
In the British Medical Journal, Vinay Prasad and colleagues at Memorial Sloan Kettering and Oregon Health & Science University conducted a comparison between the National Comprehensive Cancer Network (NCCN) clinical practice guidelines and FDA-approved drugs, looking specifically for differences in approved indications as well as approval conditions.
They examined 47 new chemical entities approved by the FDA in 2011-2015, which covered 69 cancer indications.
To these indications, the NCCN guidelines added another 44 recommendations, ranging from the removal of prior or concomitant treatment to novel drug combinations and, in one-third of cases, treatment for a wholly different malignancy. The researchers then examined the cited support, if even available (36 percent offered no evidence).
They discovered less than one-quarter of recommendations were based on randomized controlled trials, and only 16 percent on Phase 3 studies.
And over a 21-month follow-up, only six of the 44 additional NCCN recommendations (14 percent) received FDA approval.
“Although it is tempting to conclude that extrapolating a drug’s benefit to a different malignancy (based on biologic rational) is more egregious than extrapolating a drug’s benefit in a previous line of treatment, the reality is both are empirical questions that involve recommending a drug for which cost and toxicity is certain, but benefit unknown,” the authors suggested.
Prasad and colleagues acknowledge that they did not look for independent support for the additional NCCN recommendations.
Prasad wasn’t about to let the FDA off the hook, however, as he and Mayo Clinic School of Medicine’s Emma P. DeLoughery challenged the regulatory agency on recent full approvals of oncology treatments based on single-arm studies, editorializing in Annals of Oncology.
As they suggest, regular approval is effectively the domain of treatments that have shown improvements in overall survival (OS), quality of life (QoL) or surrogate endpoints well established to correlate with those endpoints. In contrast, therapies that show likely clinical benefit without clear OS or QoL improvements receive accelerated approvals, which are contingent on subsequent trials.
For the authors, however, many FDA approvals fall outside of this paradigm, including the recent full approvals of dabrafenib-trametinib in BRAF V600E mutation in metastatic non-small cell lung cancer (NSCLC) and crizotinib in ROS1 rearranged metastatic NSCLC. They listed 11 similar approvals since 2008.
“We believe granting regular approval based on surrogate end points in single-arm trials raises four concerns for public policy,” they wrote, which included the validity of the surrogates, the validity of the results without confirmatory studies, and questions about the comparative and cost-effectiveness of the therapies without randomized trials.
“While granting regular approval based on surrogate end points may appear to achieve the goal of making promising oncology drugs available sooner, it does so at the price of evidence,” DeLoughery and Prasad warned. “Once a manufacturer obtains FDA approval for an indication, there is less incentive to continue testing the drug. Indeed, there is a disincentive, as favorable results will not lead to any further benefits for the manufacturer, while unfavorable results may lead to loss of market share.”
Yes, this is 13 of goodness knows how many therapies approved in that decade, but there is a case to be made for caution.
In the rush to get novel therapies to market and expand the indications of existing therapies, and in the rush to get patients into clinical trials (more on that later), are the healthcare establishment and broader industry taking unnecessary risks? More importantly, in loosening the brakes even a little, who stands to lose most?
And as we market the miracle of clinical trials, have we perhaps imbibed our own Kool-Aid?
Randall C Willis can be reached by email at willis@ddn-news.com
