Evaluating the relationship between financial conflicts of interest among systematic reviews of authors of alprazolam use and the nature of the results and conclusions of these reviews

. Through an unprecedented marketing and promotional campaign--which included over 40 all-expenses-paid symposia to over 5000 healthcare providers, the distribution of branded fishing hats and toys, and the doubling of its national sales force--Purdue Pharma increased its sales from $48 million in OxyContin’s inaugural year to almost $1.1 billion in 2000, a stifling over 2000% increase in the span of just four years. As is evident, pharmaceutical manufacturers and the financial incentives they offer have the ability to heavily influence the proliferating rate at which drugs of abuse are prescribed³. While opioids are certainly dangerous, there is growing concern about benzodiazepines⁴.

Since Benzodiazepines have been introduced to the market in the US in the 1960s, they quickly gained a reputation for misuse and abuse⁵. Of the benzodiazepines, one formulation, alprazolam, has proved itself to be more dangerous than the others⁶. Alprazolam is the most commonly prescribed benzodiazepine, as well as the most commonly prescribed psychotropic medication in the United States with approximately 48 million prescriptions dispensed in 2013⁷. According to Hughes et al.⁸ who analyzed the 2015 National Survey on Drug Use and Health carried out by the Substance Abuse and Mental Health Services Administration (SAMHSA), an estimated 6.6 percent of the US population aged 12 or older used alprazolam products in 2015⁶. Not only is it commonly prescribed and used, but it is also commonly misuse⁸.

Alprazolam was the 2nd most common prescription medication, behind opioids, and the most common benzodiazepine to be involved in drug misuse related Emergency Department visits in 2011⁹. Of the 17.6 million alprazolam users, 4.1 million people abused alprazolam (23.3%)⁸. This is alarming once it is compared to the 6.6 million users of other tranquilizers of which only 202 thousand people reported misuse (3.1%)⁸. Despite many studies highlighting alprazolam’s misuse liability, it still remains the most commonly prescribed benzodiazepine in the US⁸.

Investigation into the cause of alprazolam’s continued misuse warrants an investigation into the frequency of FCOIs in authors of systematic reviews (SR) relating to alprazolam use. SRs have the most significant influence on medical knowledge and are often considered the gold standard for clinical practice guidelines. In the era of evidenced-based medicine, treatment plans can be heavily influenced by SRs. With such an important role in clinical care, all effort should be made to minimize FCOI in SRs. Since FCOIs are a reducible cause of bias, this study aims to determine the associations between the funding and the findings of SRs involved with alprazolam.

Methods

The present study did not meet criteria for human subjects research¹⁰, and was therefore not subject to approval and oversight by the institutional review board. The PRISMA reporting guidelines were used¹¹. The analysis was not pre-registered on a publicly available platform, such as OSF, so the results should be considered exploratory.

Data Sources
Cochrane guidelines¹² were used to conduct a systematic review of the literature to identify SRs and meta-analyses comparing alprazolam to at least one of the following comparators: placebo, SSRI, TCA, and/or other benzodiazepine. A comprehensive search was conducted on May 4, 2020, by entering the keyword “alprazolam” in the search query boxes of the following electronic databases: PubMed, Scopus, and Embase. Initial results were filtered by “systematic review” and “meta-analysis” which returned 25 PubMed and 397 Embase results. Scopus did not offer a filtering function and returned 2,671 results.

Study Selection
The screening of studies was carried out independently by two investigators (C.D. and B.N.). To qualify for inclusion, the study must: (1) be a SR or meta-analysis; (2) include at least one study in its review which compares alprazolam to placebo, SSRI, TCA, and/or other benzodiazepine; (3) be available in English; and (4) involve human data. Returned results from PubMed, Scopus, and Embase were downloaded to the Rayyan web application (rayyan.qcri.org/) where the investigators initially screened each article by title and abstract alone. Through Rayyan, the decision-making of each investigator for each article remained hidden from the other investigator until both investigators completed their independent screen. Articles that could not be definitively included or excluded based on title and/or abstract alone were then examined by full text. Disagreements of inclusion following this independent screening were then resolved by discussion and mutual consensus between C.D. and B.N.

Data Extraction
Data was independently extracted by both investigators with the use of a pre-coded form. The following data was extracted from the full-texts of all studies meeting the criteria for inclusion: (1) name of each author, (2) the authors’ institutional affiliations, (3) verbatim funding and FCOI disclosure statements, (4) whether authors included their own primary studies in the review, (5) favorability of results for alprazolam, (6) favorability of results for benzodiazepines, (7) favorability of discussion and/or conclusion for alprazolam, (8) favorability of discussion and/or conclusion for benzodiazepines, and (9) whether the article contained risk of bias evaluations.

As part of our data extraction, the results section of each study meeting our criteria for inclusion underwent two different evaluations. Firstly, they were rated as favorable for alprazolam if more than 50% of the alprazolam studies included in the SR favored alprazolam, unfavorable for alprazolam if more than 50% of alprazolam studies included in the SR favored the comparator, or mixed if the SR did not report complete results or when the results did not fit the categories of favorable or unfavorable for alprazolam as defined by the 50% benchmark that we had predetermined. The same protocol was repeated for each article in determining the results section’s favorability for benzodiazepines. Each results section was rated as favorable for benzodiazepines when more than 50% of the studies involving benzodiazepines favored benzodiazepines, unfavorable for benzodiazepines when more than 50% of studies involving benzodiazepines favored another drug class, or mixed if neither definition was met.

In addition to the results section, the discussion and conclusion sections of each study meeting our criteria for inclusion were also evaluated for favorability of alprazolam. A discussion and/or conclusion was rated as favorable for alprazolam if either stated that alprazolam was superior to comparator, unfavorable for alprazolam if either stated that comparator was superior to alprazolam, equivocal for alprazolam if either called for more data, or not applicable for alprazolam if neither discussion nor conclusion specifically addressed alprazolam. The same protocol was repeated to determine each discussion and/or conclusion’s favorability for benzodiazepines as a drug class.

During our data extraction, using Cochrane’s criteria¹², we also rated four domains of risk of bias for each study meeting our criteria for inclusion as present, absent, or uncertain: (1) replicable inclusion criteria, (2) blinded fashion screening and extraction, (3) comprehensive search, and (4) additional measures employed for reducing bias. An article’s overall risk of bias was considered high if less than 3 of these criteria were present.

Following data extraction of each article, the investigators proceeded to independently search for undisclosed financial ties in the lifetime of each author using the following electronic search engines: (1) Open Payments (open.payments data.cms.gov), which catalogues all payments made to US-based physicians from pharmaceutical and device manufacturers, (2) ProPublica’s Dollars-for-Profs (projects.propublica.org/dollars-for-profs/), which lists self-disclosed FCOIs by principal investigators of National Institutes of Health grants, (3) Google Patents (google.com/?tbm=pts), which archives both active and inactive patents across the world, and (4) the US Patent & Trademark Office (patft.uspto.gov/netahtml/PTO/search-bool.html), which indexes patents filed specifically in the US. In addition to the above search modalities, the investigators also independently reviewed the FCOI disclosure statements of other articles published by each author that could be accessed through PubMed by searching the author’s full name. In our binary decision of whether FCOI is present or absent for each author, we agreed to include any form of manufacturer payment greater than $0 (excluding any payment in the form of food and beverage), filings of patents related to medicine, and any self-disclosures of ties to manufacturers throughout the lifetime of the author as FCOI being present. Following this comprehensive review of financial ties, the investigators then met to discuss and reconcile any discrepancies of their independent findings.

Results

The systematic search identified 3,093 results. After adjusting for duplicates, the abstracts and titles of 2,829 articles were screened for a first evaluation. Of these, 2,788 did not meet the inclusion criteria and were excluded. Main reasons for exclusion were publication type (i.e. case report, conference abstract, randomized-control trial), drug of study (i.e. anticonvulsants, anti-psychotics, herbal supplements), study design (i.e. comparison to non-pharmacotherapy, investigation of pharmacokinetics or pharmacodynamics), and study population (i.e. rats, mice). Of the remaining 41 studies, two full texts could not be retrieved and an additional two studies were excluded for wrong study design. Therefore, a total of 37 studies met the criteria of inclusion for this present review (see Figure 1). Of the 37, 15 articles, or 40.5%, were rated to have a high risk of bias. (Insert Figure 1 about here)



Among the 37 studies, there were 159 authors and 169 authorships. Of these 159 authors, 42 authors, or 26.4%, had some form of FCOI in their lifetime (i.e. consulting fees, speaking fees, sponsored travel and lodging, research funding, and patent revenue) (see Table 1; Insert Table 1 about here).




Of the 159 authors, 40 listed institutional affiliations with the US, 24 with Italy, 14 with Canada, 13 with the UK, 12 with Australia, 10 with the Netherlands, 8 with Germany, 7 with Japan, 6 with Spain, 5 with China, 5 with Brazil, 4 with Austria, 4 with Turkey, 2 with South Africa, 2 with Thailand, 1 with Portugal, 1 with Spain, and 1 with Singapore. Of the 42 authors we discovered to have lifetime FCOI, 13 were affiliated with US institutions.

Of the 37 studies included in this review, 15 results sections were favorable for alprazolam, 8 were unfavorable, and 14 were mixed. Of the 15 articles which had favorable results for alprazolam, 6 articles, or 40%, had at least one author with an FCOI. Of the 15 articles which had favorable results for alprazolam, 6 of these articles also had favorable discussion/conclusion sections for alprazolam. Of these 6 articles, 3 articles, or 50%, had at least one author with an FCOI. None of these authors’ FCOIs were disclosed. Of these 3, 1 article was also rated as having a high risk of bias.

Of the 37 articles, when looking exclusively at the discussion/conclusion sections, 8 articles had discussion and/or conclusions which were favorable for alprazolam, 4 which were unfavorable, 9 which were equivocal, and 16 which did not address alprazolam. A large number of those discussions and/or conclusions which did not address alprazolam belonged to studies which focused on non-DSM diagnoses such as fibromyalgia and essential tremor, for which first-line drug treatments of those diagnoses (i.e. pregabalin, primidone) were given greater attention. Of the 8 discussion/conclusion sections that were favorable for alprazolam, 4, or 50%, had at least one author with an FCOI. Of these 4, 2 articles were also rated as having a high risk of bias.

When looking at favorability for benzodiazepines, there were 7 results sections which were favorable, 7 unfavorable, and 23 mixed. Of the 7 articles with favorable results for benzodiazepines, 4 discussion/conclusion sections were also favorable for benzodiazepines. Of these 4 articles, 1, or 25% had at least one author with an FCOI. This 1 article was also rated as having a high risk of bias.

Based on Open Payments Data, the author who received the most pharmaceutical funding received $170,713.03 in general funding, $228,171.51 in research payments, and $1,861,883.00 in associated research funding between 2013 and 2018 which in total is $2,260,767.54. Of the general funding, $114,005.68 was in consulting fees and $35,573.59 was in travel and lodging. This author published a paper in 1995 concluding that alprazolam is superior to placebo and equivalent to tricyclic antidepressants in treating depression. There are three other authors of this study, one of whom has received $259,337.30 in general payments from pharmaceutical companies between 2013 and 2018, $232,218.75 of which were consulting fees. The two other authors did not have any financial disclosures that we could find. Due to limitations of the databases, we cannot determine if these authors received funding prior to the publication of the study in question. Therefore, we cannot say with certainty if they possessed financial conflicts of interest at the time of publication. The authors wish to stress that it is unknown if any of the above-mentioned payments were related to alprazolam or contributed to its use.

Discussion

Despite finding a significant amount of funding from undisclosed sources, the results did not show a large number of authors with FCOIs in systematic reviews or meta-analyses of alprazolam. Lack of FCOIs in the study sample is possible for different reasons. First, it could be that there is little FCOI in alprazolam studies. Another possibility could be secondary to the limitations discussed below. Of the 8 articles that had favorable conclusions/discussions for alprazolam, 4 of the articles contained at least one author who had FCOIs (50%). Though this does raise some concern, bias may be present as the sample size is small, 8. Again, while this FCOI does not necessarily imply that it was related to alprazolam, with the limited ability to look back past the creation of Open Payments, it provides some insight.

It is most likely that an even higher percentage of authors in this study actually had a financial conflict of interest. It has been shown that 80% of authors of clinical drug trials have an undisclosed financial conflict of interest¹³. We recommend that journals have stricter requirements for authors in regard to disclosing the payments they have received as well as when they received those payments. More transparency could help to ensure that for-profit industries are not skewing medical research in their favor by financially influencing their researchers’ work. This has never been more important considering the current overdose epidemic¹⁴.

The countries with the most authors were the US, Italy, and Canada. In the US, 12.6% of the population reported past-year benzodiazepine use¹⁵ compared to 9% of current users in Italy¹⁶ and 12% of past-year prescription sedative users in Canada¹⁷. The largest consumers per capita of short-acting benzodiazepines are the US, Spain, France, and Italy¹⁸. It is certainly concerning that countries with the most authors were similar to the countries that had large amounts of prescriptions.

Some limitations of this study involve the resources used to determine financial conflicts of interest. Open Payments does not include information before 2013, and many of the papers were published before this time. This is demonstrated by the fact that of the authors with discoverable FCOIs, 81% were included for alprazolam research within the last decade. The scarce amount of resources for discovering FCOIs before this time period makes it difficult to accurately determine the number of authors of older studies with FCOIs. Additionally, Open Payments only includes data on US physicians, and all of the institutions covered by Dollars for Profs are located in either the US or Canada^19,20. Only 25.2% of the authors included are affiliated with the US, so these resources do not return results for the majority of authors regardless of any financial incentives they may have received. 30.9% of the authors with FCOIs are affiliated with the US, and the discrepancies in US affiliation percentages between the included authors and FCOI authors could be due to the wider availability of resources in researching US authors. Therefore, it is imperative that international databases or payment reporting proxies in nations with large research outputs are established to allow for better financial screening of researchers.

Throughout our full text reviews of each included article, it became more evident that another direction of research needs to be addressed. Some articles use unethical reporting techniques that state a treatment group is superior to comparators despite not having statistically significant evidence to support the claim.²¹ This is a growing area of concern and has been coined spin.²¹ Jellison et al. define spin as “the use of specific reporting strategies, from whatever motive, to highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or to distract the reader from statistically nonsignificant results”²¹. This study found evidence of spin in 56% of the randomized controlled trials that met their inclusion criteria²¹. We also found evidence of spin over our full text review of our included articles. One example was alarming. This article stated that the group taking alprazolam had an increased risk of worsening suicidal ideation in patients with depression in their results section. However, it stated that their findings “show no suggestion that use of alprazolam in depressed patients is associated with either the emergence or worsening of suicidal ideation” in their discussion section²². The authors of this article disclosed their affiliation with the original producers of alprazolam, The Upjohn Co. and Pharmacia & Upjohn, Inc²². It is also interesting that, despite this affiliation, neither author had any payments reported on any of the four search engines used in our study. This could be due to the limitations that we previously discussed, or they could be due to other reasons. Regardless, better oversight of spin and payment reporting needs to occur to maintain the validity and the integrity of peer-reviewed research.







References

1. Neuman J, Korenstein D, Ross JS, Keyhani S. Prevalence of financial conflicts of interest among panel members producing clinical practice guidelines in Canada and United States: cross sectional study [Internet]. Vol. 343, BMJ. 2011. p. d5621–d5621. Available from: http://dx.doi.org/10.1136/bmj.d5621

2. Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome: systematic review with meta-analysis [Internet]. Vol. 44, Intensive Care Medicine. 2018. p. 1603–12. Available from: http://dx.doi.org/10.1007/s00134-018-5293-7

3. Van Zee A. The promotion and marketing of oxycontin: commercial triumph, public health tragedy. Am J Public Health [Internet]. 2009 Feb;99(2):221–7. Available from: http://dx.doi.org/10.2105/AJPH.2007.131714

4. Votaw VR, Geyer R, Rieselbach MM, McHugh RK. The epidemiology of benzodiazepine misuse: A systematic review. Drug Alcohol Depend [Internet]. 2019 Jul 1;200:95–114. Available from: http://dx.doi.org/10.1016/j.drugalcdep.2019.02.033

5. Schmitz A. Benzodiazepine use, misuse, and abuse: A review. Ment Health Clin [Internet]. 2016 Jun;6(3):120–6. Available from: http://dx.doi.org/10.9740/mhc.2016.05.120

6. Ait-Daoud N, Hamby AS, Sharma S, Blevins D. A Review of Alprazolam Use, Misuse, and Withdrawal. J Addict Med [Internet]. 2018;12(1):4–10. Available from: http://dx.doi.org/10.1097/ADM.0000000000000350

7. Grohol J. Top 25 Psychiatric Medication Prescriptions for 2013. Psych Central; 2015. 2015.

8. Hughes A, Williams MR, Lipari RN, Bose J, Copello EAP, Kroutil LA. Prescription drug use and misuse in the United States: Results from the 2015 National Survey on Drug Use and Health. NSDUH Data Review. 2016;A1–24.

9. SAMHSA. Drug Abuse Warning Network, 2011: National Estimates of Drug-Related Emergency Department Visits. 2013; Available from: https://www.samhsa.gov/data/sites/default/files/DAWN2k11ED/DAWN2k11ED/DAWN2k11ED.pdf

10. Office for Human Research Protections (OHRP). 45 CFR 46. 2016 Feb 16 [cited 2020 Jun 9]; Available from: https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html

11. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med [Internet]. 2009 Jul 21;6(7):e1000097. Available from: http://dx.doi.org/10.1371/journal.pmed.1000097

12. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions [Internet]. Wiley; 2008. 672 p. Available from: https://play.google.com/store/books/details?id=RepLNQEACAAJ

13. Tau N, Shochat T, Gafter-Gvili A, Amir E, Shepshelovich D. Undisclosed Financial Conflicts of Interest of Authors of Clinical Drug Trials Published in Influential Medical Journals: A Cohort Study. Mayo Clin Proc [Internet]. 2019 Nov;94(11):2272–6. Available from: http://dx.doi.org/10.1016/j.mayocp.2019.08.025

14. CDC. America’s Drug Overdose Epidemic: Data to Action [Internet]. Centers for Disease Control and Prevention. 2020 [cited 2020 Jun 12]. Available from: https://www.cdc.gov/injury/features/prescription-drug-overdose/index.html

15. Maust DT, Lin LA, Blow FC. Benzodiazepine Use and Misuse Among Adults in the United States. Psychiatr Serv [Internet]. 2019 Feb 1;70(2):97–106. Available from: http://dx.doi.org/10.1176/appi.ps.201800321

16. Magrini N, Vaccheri A, Parma E, D’Alessandro R, Bottoni A, Occhionero M, et al. Use of benzodiazepines in the Italian general population: prevalence, pattern of use and risk factors for use [Internet]. Vol. 50, European Journal of Clinical Pharmacology. 1996. p. 19–25. Available from: http://dx.doi.org/10.1007/s002280050063

17. Health Canada. Canadian Tobacco Alcohol and Drugs (CTADS) Survey: 2017 summary - Canada.ca [Internet]. 2018 [cited 2020 May 25]. Available from: https://www.canada.ca/en/health-canada/services/canadian-tobacco-alcohol-drugs-survey/2017-summary.html

18. Khong TP, de Vries F, Goldenberg JSB, Klungel OH, Robinson NJ, Ibáñez L, et al. Potential Impact of Benzodiazepine Use on the Rate of Hip Fractures in Five Large European Countries and the United States [Internet]. Vol. 91, Calcified Tissue International. 2012. p. 24–31. Available from: http://dx.doi.org/10.1007/s00223-012-9603-8

19. Open Payments Data - CMS | CMS Open Payments Data [Internet]. OpenPaymentsData.CMS.gov. [cited 2020 May 28]. Available from: https://openpaymentsdata.cms.gov/

20. Wei S, Waldman A, Armstrong D. Dollars for Profs - All Institutions | ProPublica [Internet]. ProPublica. [cited 2020 May 28]. Available from: https://projects.propublica.org/dollars-for-profs/organizations

21. Jellison S, Roberts W, Bowers A, Combs T, Beaman J, Wayant C, et al. Evaluation of spin in abstracts of papers in psychiatry and psychology journals. BMJ Evid Based Med [Internet]. 2019 Aug 5; Available from: http://dx.doi.org/10.1136/bmjebm-2019-111176

22. Jonas JM, Hearron AE Jr. Alprazolam and suicidal ideation: a meta-analysis of controlled trials in the treatment of depression. J Clin Psychopharmacol [Internet]. 1996 Jun;16(3):208–11. Available from: http://dx.doi.org/10.1097/00004714-199606000-00003