In a new article in the Journal of Clinical Epidemiology, researchers examined drug regulatory agencies’ decision-making processes and their use of clinical trials when approving an extended-release medication used to treat attention deficit hyperactivity disorder (ADHD). The researchers found that clinical trials were missing in over half of the examined applications. They conclude by offering suggestions for improvements to the drug regulatory system, such as calling for total transparency, including all relevant trials in decision making, among other recommendations.
The authors, led by Kim Boesen of the Nordic Cochrane Centre, write:
“To our knowledge, this may be the first report to systematically investigate whether drug regulatory agencies make decisions based on complete or select samples of clinical trials. In our cohort of 13 extended-release methylphenidate applications for ADHD in adults, we identified missing trials in 7 (54%) applications.”
Drug regulatory agencies, such as the US Food and Drug Administration (FDA), base their decisions on whether a new drug will be approved on clinical trials submitted by pharmaceutical companies. However, industry sponsorship by pharmaceutical companies results in these trials being commonly selectively published in medical journals.
The outcomes of the trials are also frequently selectively reported. As a result, there can be differences in the studies published in medical journals, which are accessible to the general public, versus the raw data submitted to drug regulators, with the data omitted in published reports often including information related to patient-reported outcomes and harms of the medication.
In the European Union (EU), legislation was passed that requires accurate reporting of the results of drug trials. However, a recent study found that more than half of new clinical trials in the EU did not comply with this legislation.
Moreover, in addition to ethical concerns about how results are reported, drug companies’ approaches to recruiting participants in their clinical trials have also been criticized, as they exploit poor individuals who are desperate for money as “guinea pigs.”
Boesen and colleagues write:
“In 2008, it was reported that 31% of trials of antidepressants included in new drug applications submitted to the US Food and Drug Administration (FDA) were never published in medical journals. Most of these unpublished trials did not favor the antidepressant and were considered ‘negative’ by the FDA.”
Although international standard requires all pharmaceutical companies to submit a document when applying for marketing authorization, including information related to quality assurance, clinical trial data, and non-clinical studies, such as toxicology studies, studies have found that this is not always the case.
For example, a review of drug regulators in Europe and the United Kingdom found that antidepressant trials were incomplete and internally inconsistent. Additionally, a review of a drug for influenza found that the European Medicines Agency (EMA) and the FDA “largely ignored” the largest drug trial in their approval processes.
In the current study, researchers examined publicly available drug regulatory documents related to extended-release methylphenidate for adult ADHD. The drug regulatory agencies included in this review spanned across the US, Australia, Canada, Japan, and Europe.
The researchers found that 7 (54%) trials were missing out of 13 drug applications they reviewed. When they reached out to the drug regulatory agencies to inquire about the missing trials, the FDA, Health Canada, and the Australian Therapeutic Goods Administration (TGA) did not directly answer their questions. The German Bundesinstitut für Arzneimittel und Medizinprodukte (BfArM), on the other hand, confirmed that they did not examine a trial that was missing.
Boesen and colleagues highlight the potential negative impact of missing trials through an example where the FDA approved an ADHD drug based on two trials sponsored by Janssen Pharmaceuticals. The FDA’s medical reviewer raised concerns about the dangers of the drug, highlighting potential cardiovascular risks, and suggested that a third trial be completed as a condition for the drug being approved.
However, this recommendation was dismissed by the FDA Team Leader and Division Director, as they did not believe it was feasible or ethical to complete additional clinical trials due to the length of time and logistics it would require. Additionally, although there was information on four additional placebo-controlled trials submitted along with this application, the FDA chose not to request these trials.
Moreover, the same ADHD drug was approved by Health Canada based on one of those trials and was approved by the TGA based on two of those trials, with the TGA also listing data related to harmful effects of the drug from a third Janssen-sponsored study.
Interestingly, the drug application was rejected by a British drug regulatory agency based on all three of the Janssen trials, with one of the reviewers commenting:
“The totality of the data is therefore weak, with one successful, one borderline failure, and one clearly failed trial.”
These examples emphasize the importance of including all trials in decision-making processes. As in the case of the last example, the addition of the third “clearly failed” trial influenced the British regulatory agency’s decision to reject the drug.
Additionally, the language used in the requirements for new drug applications is largely broad and unclear, leaving it open to interpretation. For example, both the FDA and EMA have guidelines that state that all drug trials that are “pertinent” to the proposed drug should be included in the application. Health Canada uses the language “all pivotal trials,” which again leaves pharmaceutical companies the room to construe that in ways that will make their applications more favorable to regulatory agencies.
The lack of clarity in the language used across drug regulatory agencies may be explained by the language used by the agency upon which regulators rely, the International Council for Harmonisation (ICH), whose guideline states that trials should be included which are “pertinent to the efficacy of the medicinal product.”
Boesen and colleagues offer several explanations for why some clinical trials are not included in public documents, such as drug companies deliberately deciding to exclude them and the fact that drug regulatory agencies do not explicitly request all trials be submitted.
Additionally, they point out that regulatory agencies do not complete their own reviews of drug trial databases but instead rely on the application material, regulatory agencies do not always make sure that what is listed on the application has actually been submitted, and regulatory agencies themselves may decide to exclude certain trials based on their own determinations.
Limitations of the study include its reliance on public documents and a small sample size.
The authors conclude with several recommendations for improvements to the drug regulatory system, such as greater transparency the inclusion of all relevant drug trials, which could be addressed in part by regulatory agencies doing their own systematic reviews of drug trial databases, clarifying application requirements to avoid loopholes, accountability for drug companies that do not submit all data, and increased partnership between drug regulatory agencies internationally.
Others have suggested that a “registered reports” model for peer review and publication be adopted, where prior to the study even beginning, the research questions and criteria would need to be clearly defined and approved by peer reviewers before the drug trial could commence, which could prevent potential rule-bending after the data has been collected.
Boesen, K., Jørgensen, K. J., Gøtzsche, P. C. (2021). Clinical trials were missing from regulatory documents of extended-release methylphenidate for ADHD in adults: A case study of public documents. Journal of Clinical Epidemiology, 7(3). https://doi.org/10.1016/j.jclinepi.2021.10.027 (Link)
Interested readers can find a quick overview of clinical trial reporting rules in the US and weak FDA enforcement here:
Here more on EU rules:
A more in-depth analysis is here, with focus on the UK:
Useful background on why this issue is so important for patients here: