An Evaluation of Spin Within Abstracts of Systematic Reviews Concerning Treatments for Peyronie’s Disease
David Wenger, B.S., Office of Medical Student Research, Oklahoma State University Center for Health Sciences
Greg Balcerak, B.S., Office of Medical Student Research, Oklahoma State University Center for Health Sciences
Ross Nowlin, B.S., Office of Medical Student Research, Oklahoma State University Center for Health Sciences
Austin Johnson, B.S., Office of Medical Student Research, Oklahoma State University Center for Health Sciences
Brad Johnson, B.S. Office of Medical Student Research, Oklahoma State University Center for Health Sciences
FUNDING:This work was not funded
CONFLICTS OF INTEREST:The authors state no conflict of interest
Abstract:
Background: Nearly 11% of men in the United States have Peyronie’s disease. Here we aim to determine which mechanisms of spin are the most common among those found in PD systematic reviews.
Methods: A search of MEDLINE and Embase databases was conducted and all returns were screened for inclusion criteria. Articles meeting inclusion criteria were evaluated for the 9 most severe types of spin and characteristics including: intervention type, date of the review, funding source, PRISMA adherence of the review, PRISMA adherence for the journals that published the reviews, and the journals’ five-year impact factor, were extracted.
Results: No spin was found in the abstracts of the 20 included systematic reviews. However, 60% (12/20) of the included systematic reviews did not mention a risk of bias assessment.
Conclusions: Systematic reviews pertaining to the treatment of Peyronie’s disease appear to contain factual reporting of study outcomes, however many of these studies are lacking in terms of methodological quality assessing for risk of bias. As a result of this finding, clinicians charged with the treatment of Peyronie’s disease should consume this research critically and cautiously. Systematic reviews pertaining to the treatment of Peyronie’s disease appear to be free of spin, however, we recommend critical consumption of this research as risk of bias assessment appears to be lacking.
Introduction
A 2016 study estimates that nearly 11% of men in the United States have Peyronie’s disease (PD),1which leads to the disruption of sexual function and psychological effects, such as negative sexual self-image, isolation, and social stigmatization 2. PD is a fibrotic disorder of the penis that occurs after minor trauma to the penis in genetically predisposed men. The abnormal healing process causes a build up of collagen plaques on the tunica albuginea resulting in scarring that creates excess penile curvature or pain resulting in severe sexual dysfunction.3 ,4 Thus, it is crucial that there are treatment modalities available to those with PD that have been demonstrated through rigorous study designs and the highest forms of evidence.
Systematic reviews have been considered the “main torch-bearers of evidence based medicine”.5 Knowing this, many treatment guidelines use systematic reviews as their source of evidence, including PD treatment guidelines 6. Systematic reviews must be objectively reported such that study results may be accurately interpreted by readers. One form of inaccurate reporting known as spin, is the intentional or unintentional misrepresentation of a study’s findings 7. Previous investigations have found spin to be present in 31% of systematic review abstracts regarding the treatment of acne vulgaris 8. Spin has the potential to influence the interpretations of systematic reviews, ultimately affecting the care of patients. Due to the evidence of spin found in other fields of medicine, we want to determine the extent that spin can be found in abstracts of systematic reviews for PD treatments. In addition, we want to determine which mechanisms of spin are the most common among those found in PD systematic reviews. To further advance clinical practice, the urology community should understand and be aware of potential spin among SRs.
Methods
A search of MEDLINE and Embase databases was conducted on June of 2020 with the search string available in Figure 1. The search returns were uploaded to Rayyan, a systematic review platform, and screened for inclusion criteria in a masked and duplicated fashion by DW and GB. Duplicates were removed and systematic reviews and meta-analyses that focused on the treatment of Peyronie’s disease, were written in English, have only human subjects, and contain an abstract were ultimately included for further analysis. Any screening disagreements were resolved by an arbitrator. The authors tasked with data extraction, DW and GB, participated in training prior to starting the study. During the training, the nine most severe forms of spin defined by Yavchitz et al.7 were defined and examples of each form of spin were discussed in detail. Study characteristics were extracted using a pilot tested Google form by DW and GB. Study characteristics of interest including intervention type, date of the review, funding source, PRISMA adherence of the review, PRISMA adherence for the journals that published the reviews, and the journals’ five-year impact factor. Following extraction, the overall frequency of spin, as well as its subtypes, were analyzed using descriptive statistics. Since our study did not include human subjects it did not meet the definition of the human subjects research per the US Code of Federal Regulations and was not subject to review by the institutional review board. To promote reproducibility and transparency the study protocol, training material, and extraction forms were uploaded to the Open Science Framework. This study was conducted in tandem with other studies that also evaluated spin in systematic reviews across other medical conditions.
Results
Our original search query returned 122 articles, of which 31 were duplicates and 72 were removed during full text screening, which resulted in the inclusion of 19 systematic reviews. The majority of included studies pertained to surgical intervention (7/19; 36.8%) and were not funded (9/19; 47.4%). A majority of studies (11/19; 57.9%) mentioned PRISMA adherence, and 10 (of 19; 52.6%) were published in a journal that required PRISMA adherence. Overall, no spin was found in the abstracts of the included systematic reviews (Table 1). Risk of bias was not assessed in 12 (12/19; 63.2%) of the systematic reviews, limiting our ability to evaluate one form of spin— whether the results in the abstract were discussed within the context of the risk of bias of the included studies.
Discussion
Reporting in systematic reviews of Peyronie’s disease treatment modalities appears to be free of spin, however this finding may be complicated by a significant absence of risk of bias reporting. The presence of a risk of bias assessment is essential to the determination of spin types 5 and 9 and factors into the evaluation of multiple others, therefore, its absence prevents an in depth and complete evaluation of the prevalence of spin. While on the surface it appears that systematic reviews and meta-analyses pertaining to the treatment of Peyronie’s disease contain factual and impartial reporting, as seen by the absence of spin, we concede that larger scale investigations are needed to better understand the true nature of spin within the field.
The lack of risk of bias assessment among our included studies is a worrisome finding as systematic reviews and meta-analyses are commonly used to generate clinical practice guidelines and direct everyday clinical decision making.9 The strength of systematic reviews and meta-analyses is dependent on the synthesis of a wide body of literature, therefore the use of strong primary studies is critical to the production of reliable evidence. Determining the strength of the primary studies within a systematic review requires the methodological quality of each study be evaluated. Methodological quality often refers to the study’s internal validity which in turn is assessed through a risk of bias assessment.10, 11The following excerpt from a study conducted by Reddy et al. evaluating the presence of spin in erectile dysfunction provides an example of spin type 5: “The abstract [of Wang et al] states, “All included studies were tested for publication bias, and results indicated that there was no significant bias… Traditional Chinese medicine combined with tadalafil has significant efficacy in the treatment of ED with no increase in side effects.” However, while reviewing the full article, the authors failed to mention in their abstract that there was a high risk of detection bias in 10 of the 11 included studies and a high risk of performance bias in all 11.” 12, 13. This example demonstrates the importance of a risk of bias assessment and its role in identification of specific spin types. The absence of a risk of bias assessment hinders the ability to interpret findings, explain heterogeneity, and grade the strength of a body of evidence, ultimately limiting the utility of systematic reviews and meta-analyses. 14
Our findings may not be isolated to systematic reviews and meta-analyses pertaining to Peyronies disease treatment as previous investigations have demonstrated that a significant percentage of systematic reviews of randomized control trials failed to incorporate a risk of bias assessment into their statistical analyses and results.15 Perhaps one cause of the lack of risk of bias assessment is the wide array of methodology available for evaluating the risk of bias, each utilizing different study constructs in the assessments. 14,16,17 Greater standardization of risk of bias evaluations may be warranted.
Conclusion:
Systematic reviews pertaining to the treatment of Peyronie’s disease appear to be free of spin. We recommend critical consumption of this research as risk of bias assessment appears to be lacking.
REFERENCES
1. Stuntz M, Perlaky A, des Vignes F, Kyriakides T, Glass D: The Prevalence of Peyronie’s Disease in the United States: A Population-Based Study. PLoS One. 2016, 11:e0150157. 10.1371/journal.pone.0150157
2. Rosen R, Catania J, Lue T, Althof S, Henne J, Hellstrom W, Levine L: Impact of Peyronie’s disease on sexual and psychosocial functioning: qualitative findings in patients and controls. J Sex Med. 2008, 5:1977–84. 10.1111/j.1743-6109.2008.00883.x
3. Ziegelmann MJ, Bajic P, Levine LA: Peyronie’s disease: Contemporary evaluation and management. Int J Urol. 2020, 27:504–16. 10.1111/iju.14230
4. Miner MM, Seftel AD: Peyronie’s disease: epidemiology, diagnosis, and management. Curr Med Res Opin. 2014, 30:113–20. 10.1185/03007995.2013.842544
5. Lai N: Evidence based medicine series: evidence based medicine: an overview. Malays Fam Physician. 2009, 4:19–22.
6. Peyronie’s Disease Guideline - American Urological Association. Accessed: July 15, 2020. http://auanet.org/guidelines/peyronies-disease-guideline.
7. Yavchitz A, Ravaud P, Altman DG, Moher D, Hrobjartsson A, Lasserson T, Boutron I: A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity. J Clin Epidemiol. 2016, 75:56–65. 10.1016/j.jclinepi.2016.01.020
8. Ottwell R, Rogers TC, Michael Anderson J, Johnson A, Vassar M: Evaluation of Spin in the Abstracts of Systematic Reviews and Meta-Analyses Focused on the Treatment of Acne Vulgaris: Cross-Sectional Analysis. JMIR Dermatology. 2020, 3:e16978. 10.2196/16978
9. Peyronie’s Disease Guideline - American Urological Association. Accessed: August 31, 2020. http://auanet.org/guidelines/peyronies-disease-guideline.
10. Ma L-L, Wang Y-Y, Yang Z-H, Huang D, Weng H, Zeng X-T: Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better? Mil Med Res. 2020, 7:7. 10.1186/s40779-020-00238-8
11. Shuster JJ: Review: Cochrane handbook for systematic reviews for interventions, Version 5.1.0, published 3/2011. Julian P.T. Higgins and Sally Green, Editors. Research Synthesis Methods. 2011, 2:126–30. 10.1002/jrsm.38
12. Reddy AK, Lulkovich K, Ottwell R, et al.: Evaluation of Spin in Abstracts of Systematic Reviews and Meta-analyses Focused on Treatments of Erectile Dysfunction: A Cross-sectional Analysis. Sexual Medicine. 2021, 9:100284. 10.1016/j.esxm.2020.10.012
13. Wang YL, Geng LG, He CB, Yuan SY: Chinese herbal medicine combined with tadalafil for erectile dysfunction: a systematic review and meta-analysis. Andrology. 2020, 8:268–76. 10.1111/andr.12696
14. Viswanathan M, Patnode CD, Berkman ND, et al.: Recommendations for assessing the risk of bias in systematic reviews of health-care interventions. J Clin Epidemiol. 2018, 97:26–34. 10.1016/j.jclinepi.2017.12.004
15. Hopewell S, Boutron I, Altman DG, Ravaud P: Incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional study. BMJ Open. 2013, 3:e003342. 10.1136/bmjopen-2013-003342
16. Whiting P, Savović J, Higgins JPT, et al.: ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016, 69:225–34. 10.1016/j.jclinepi.2015.06.005
17. Shea BJ, Reeves BC, Wells G, et al.: AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017, 358:j4008. 10.1136/bmj.j4008