In a recent article, researcher Lisa Cosgrove and her colleagues question the quality of evidence behind a recent systematic review of long-acting injectable antipsychotics (LAIs). LAIs are drugs injected into one’s body and can maintain effects for four weeks.
The initial review, authored by Taishiro Kishimoto and colleagues, stated that LAI antipsychotics were more effective than oral antipsychotics in reducing hospitalization and relapse. However, on closer scrutiny, Cosgrove and colleagues found that the evidence was of low quality, riddled with bias, and vulnerable to industry corruption. They then provide recommendations on how to reduce bias and improve the quality of evidence when systematic reviews are conducted by authors who have industry ties.
“In order to enhance the quality and trustworthiness of systematic reviews, authors need to consider more carefully the minimum methodological rigor they believe is needed to accurately answer a specific question. We argue that this systematic review included weak study designs and failed to adequately consider the effects of risks of bias on the results of included studies.”
There has been an ongoing debate about whether LAIs are more effective than oral antipsychotics. MIA previously covered this debate between Cosgrove and Kishimoto and their colleagues, where some of these issues were raised and addressed.
There are serious concerns about patient rights and safety with LAIs, so investigations that find that LAIs are no more effective than oral antipsychotics are especially pertinent to gauge harm and adverse effects. In addition, antipsychotic drug studies tend to have significant reporting bias, with many obscuring adverse outcomes.
Psychiatry has been long plagued by influence from the pharmaceutical industry. More recently, similar criticisms have been levied against other medical sciences. There are obvious financial conflicts of interest that can undermine the integrity of research and more subtle implicit biases that might lead researchers to spin their abstracts or misinterpret data. In addition, researchers have noted deceitful practices common in clinical trials that can manipulate outcomes.
The World Health Organization’s Mental Health Gap Programme conducted the systematic review under investigation. This program has not escaped critique by researchers who question its claims of universality about mental illnesses and the technology it uses to diagnose and intervene.
Cosgrove and colleagues note that this review has significant clinical consequences for patient care and rights and has already received attention as both clinicians and researchers tend to trust systematic reviews. It is thus essential that the evidence these reviews are built on is of superior quality.
Systematic reviews are supposed to comprehensively and robustly comb through relevant studies. However, if the studies under examination are biased or of poor quality, the review can reflect similar bias, and its conclusions can be equally flawed.
The authors recommend that it is crucial that low-quality RCTs (Randomized Controlled Trials) and studies that are at risk of bias (RoB) should undergo sensitivity analyses, especially since it has been repeatedly found that industry-sponsored studies tend to favor industry treatment. A recent Cochrane review directly implicated systematic reviews, finding that reviews authored by those with financial conflicts of interest tend to be of lower quality and favor the industry.
Cosgrove and colleagues critique numerous decisions made by the authors of the systematic review that led to favorable conclusions about LAIs.
First, the studies of the most dubious quality, the pre-post studies, produced the most favorable results for LAIs. The more rigorous Randomized Controlled Trials produced the least favorable results. The review researchers should have explicitly highlighted this for clinicians to exercise informed decisions.
More importantly, patients need accurate and relevant information to make informed decisions about their treatment. This includes knowing about its possible adverse effects. Kishimoto and colleagues did not focus enough on the harm that LAIs can produce or the fact that the number needed to treat in order for one patient to show benefit is 539. Cosgrove and colleagues write:
“The authors of this systematic review on LAIs vs. oral formulations did provide this information in the article’s online appendix, which indicates a better adverse effect profile for oral formulations than LAIs. However, the authors misrepresent these data in the body of the paper, stating, “LAIs showed no significant difference from oral antipsychotics regarding most adverse events” (Kishimoto et al. 2021, 388). It is unlikely that most readers of this systematic review will examine the data contained in the appendix, and thus readers may get a more positive impression of the risks associated with LAIs, an impression that is inconsistent with the data.”
Kishimoto and colleagues failed to consider the extent to which many studies were not properly randomized, their allocation concealed, and their outcomes blinded. Only 19% of the RCTs included were properly randomized. A sensitivity analysis that excludes investigations with a high risk of bias would have been helpful but was not performed. As MIA had previously reported:
“While in the appendix it was reported that in RCTs, adverse effects related to LAIs had a worse profile than oral antipsychotics, this fact did not feature in the authors’ conclusion and interpretation.”
Lastly, the review measured disease-centric outcomes (decreased hospitalization or relapse) and not patient-centric, thus ignoring patient choice, desires, or needs. Disease-centric outcomes are known to be more industry-friendly. As the importance of shared decision-making with clients has become apparent, this decision to ignore what patients value is egregious.
Cosgrove and colleagues also point to the significant extent to which the authors of this review are tied to the pharmaceutical industry. They write:
“One author was an employee of a company that manufactures antipsychotics, and two others are shareholders/stock option holders of a company, LB pharma, that aims to develop new LAIs (Business Wire 2020) … Based on data from ProPublica’s “Dollars for Docs” and the Open Payments database, the two US authors received payments from the industry of a combined total of USD $1,808,001 from 2014-2020 (the last year data are available on these databases) … Of that total, $560,772 was received from the companies that manufacture LAIs.”
While these ties do not automatically mean that researchers intentionally manipulated their data, such relationships can create implicit bias and influence how the data is interpreted, what might be highlighted (weak low, quality pre-post studies), and what is left out (harm profiles and patient-centered outcomes).
Since the FDA or the European Medicines Agencies do not regulate systematic reviews, the lack of transparency is deeply concerning, as Kishimoto and colleagues have stated that they will not make their data available for sharing.
The authors lay out several recommendations if unbiased systematic reviews have to be conducted by people with ties to the industry.
First, reviewers must be more transparent and open about their uncertainty and highlight weak evidence since these results have direct consequences for clinical guidelines and thus patient care.
Second, systematic reviews need to enhance their independence from industry conflict. For example, Cochrane requires at least two-thirds of the review team to have no conflicts of interest, but PRISMA guidelines, followed by Kishimoto, do not have this requirement.
Third, reviewers need to be more discerning about the studies they allow into their review. In this case, the low-quality pre-post studies favored LAIs and underestimated the risks involved.
Lastly, it is essential to focus on patient-centric outcomes; studies that focus on the lived experience of patients have highlighted some of the adverse effects of antipsychotics that are overlooked in traditional study designs.
Cosgrove L, Mintzes B, Bursztajn HJ, D’Ambrozio, G, Shaughnessy AF (2022). Industry effects on evidence: A case of long-acting injectable antipsychotics Accountability in Research. DOI: 10.1080/08989621.2022.2082289 (Link)