A Descriptive Analysis of Registered Pragmatic Clinical Trials

Kaleb Veit, DO, Department of Internal Medicine, Oklahoma State University College of Osteopathic Medicine
Matt Vassar, Ph.D., Department of Psychiatry and Behavioral Sciences, Oklahoma State University College of Osteopathic Medicine
Jennifer Thomas, DO, Graduate Medical Education, Oklahoma State University Medical Center



Abstract

Pragmatic trials are designed to determine the effect of an intervention under usual, as opposed to ideal, conditions. A descriptive analysis of all pragmatic clinical trials registered on Clinical trials.gov was performed. A total of 424 pragmatic trials were first posted during the time period searched. The analysis included study type, study design, clinical condition studied, intervention, phase, enrollment, and funding source.

Introduction

Clinical trials can be designed with two approaches: a pragmatic approach focused on choosing between treatments, and an explanatory approach focused on understanding if a difference exists between treatments.1 Pragmatic clinical trials compare clinically relevant treatment options, study a heterogeneous population, and focus on clinically relevant outcomes.2 They investigate intervention effectiveness, the performance of an intervention according to routine clinical use.3 Pragmatic trials are intended to validate clinical benefit under real world conditions, rather than structured settings commonly found with explanatory trials.1,4 In contrast, explanatory trials are conducted to determine the efficacy of an intervention and are performed according to strict conditions.3 Explanatory trials often seek to confirm a clinical or biological theory.5

Placebos are often absent in pragmatic trials because the intent is to compare a proposed treatment with the standard of care. Bias is also treated differently in pragmatic trials. While carefully guarded against in explanatory trials, pragmatic trials accept a certain degree of bias inherent in patient and provider choices. Outcome measures also differ in pragmatic versus explanatory trials. Explanatory trials often measure intermediate outcomes, while pragmatic trials can measure more comprehensive outcomes, such as quality of life or incidence of stroke.3

As the cost of healthcare rises, and as greater focus is placed on relevance of trials to routine clinical practice, pragmatic trials are increasingly used to investigate quality and value of healthcare interventions.4 Previous studies have investigated the influence of pragmatic trials on different areas of medicine, such as oncology.6 However, a more global review of all pragmatic trials has not been conducted. To address this need, we performed a descriptive analysis of registered pragmatic clinical trials.

Methods

We searched ClinicalTrials.gov using the search term “pragmatic” in the “other terms” field. We searched for trials first posted between January 1, 2002 and June 1, 2016. The year 2002 was chosen as it was the first year that a pragmatic trial was listed on ClinicalTrials.gov. The search was not limited to country or specialty, but it was limited to papers in English. The analysis included study type, study design, clinical condition studied, intervention, phase, enrollment, and funding source. Frequency and percentage of study types, designs, conditions, interventions, phases, enrollment, and funding sources were calculated. Search results were exported to Microsoft Excel. All analyses were performed using Excel. The search was performed on May 7, 2018. The search data are available in the supporting information.

Supplemental File - Pragmatic Search Data

Results

A total of 424 pragmatic trials were posted during the time period searched. Outcome measures varied by clinical condition. Each year from 2011 through 2015, the number of pragmatic trials increased. The largest increase was from 2014 (n=70) to 2015 (n=99) when the number published increased by 29.3% (n=29).

Interventional trial phases ranged from early phase 1 through phase 4, but most did not have phases. Enrollment ranged from 0 to 933,789 with a median of 290 and a mean of 6,668. An analysis of study type showed that 92.9% (n=394) were interventional and 7.1% (n=30) were observational. Design in interventional trials included 92.6% randomized (n=365), 3.3% single group (n=13), 3.0% non-randomized (n=12), 0.5% open label (n=2), 0.3% parallel assignment (n=1), and 0.3% crossover assignment (n=1). Design in observational trials included 63.3% cohort (n=19), 6.7% case control (n=2), 6.7% case-crossover (n=2), 6.7% case-only (n=2), 6.7% post-marketing observation (n=2), 3.3% cross-sectional (n=1), 3.3% ecologic or community (n=1), and 3.3% other (n=1).

Clinical conditions or topics studied were varied and included 78.3% medical/surgical (n=332), 9.0% mental health (n=38), 6.4% behavioral (n=27), 4.7% public health (n=20), 1.4% healthcare quality (n=6), and 0.2% medical education (n=1).

Interventions were 28.3% behavioral (n=120), 19.6% drug (n=83), 8.0% device (n=34) 7.8% procedure (n=33), 1.4% radiation (n=6), 0.5% biological (n=2), 0.5% dietary supplement (n=2), and 34.0% other (n=144).

Of the interventional trials (n=394), phases were 0.5% early phase 1 (n=2), 0.3% phase 1 (n=1), 3.6% phase 2 (n=14), 1.3% phase 2 and 3 (n=5), 10.9% phase 3 (n=43), 12.9% phase 4 (n=51), and 70.6% not applicable (n=278).

ClinicalTrials.gov categorizes funding sources into four categories: U.S. National Institutes of Health (NIH), other U.S. Federal agencies, industry, and all others, which includes individuals, universities, and community-based organizations. Funding sources were 17.5% various combinations of industry/all others/NIH/other U.S. Federal agencies (n=74), 5.4% industry (n=23), 0.7% other U.S. Federal agencies (n=3), 0.2% National Institutes of Health (NIH) (n=1), and 76.2% all others (n=323). It is possible that studies in the all others category included studies which were unfunded, as unfunded studies are not excluded from registration on ClinicalTrials.gov. However, the specific funding source, or lack of funding, for each of the studies in the all others category was not indicated.

Discussion

Pragmatic trials may be more representative of clinical medicine, as study conditions are often not present in the lives of patients. Our results suggest that these studies are employed in diverse areas of medicine. Pragmatic trials have become increasingly popular, at least partially due to an increased interest in comparative effectiveness research by policymakers.4

Our analysis found that pragmatic clinical trials often investigate medical/surgical topics. However, a significant proportion (21.7%) investigated mental health, behavioral, public health, healthcare quality, and medical education topics. These topics may have been more easily investigated using a pragmatic design because of the complex nature of interventions or conditions studied.5 In addition, interventions were most often unable to be categorized (other, 34.0%), followed by behavioral (28.3%). This finding lends support to the idea that pragmatic trials may be well suited to investigate complex interventions.5

One limitation of our cross-sectional analysis is that it does not capture all trials with pragmatic elements. Pragmatism is a continuum, and virtually all clinical trials have both explanatory and pragmatic elements.4 In addition, our analysis reviewed only trials reported in English, and it is possible that pragmatic trials are more frequently reported in other languages.

To facilitate the study and usage of pragmatic clinical trials, we propose the creation of a pragmatic clinical trial registry. The creation of a trial registry devoted solely to pragmatic trials would facilitate investigation of pragmatic trials, greater transparency of data, prevent selective reporting, and assist patients in learning about pragmatic clinical trials for which they may be eligible. In addition, the creation of a pragmatic clinical trial registry would promote understanding and interpretation of pragmatic trials by journal editors and consumers of medical literature.7 Use of the PRECIS-2 tool8 could help determine which trials should be registered in a pragmatic clinical trial registry. Although not originally designed for this purpose, the PRECIS-2 tool does quantify the degree of pragmatism present in nine different trial design domains, with a higher score on each domain indicating a greater degree of pragmatism.8 By averaging the score of the nine domains, a threshold value could be specified above which a trial would be required to register with the pragmatic trial registry.

The impact of pragmatism on clinical practice remains to be determined. With the rise of patient-centered care, the concept of applying evidence-based medicine to what patients are actually able to perform is exciting. Pragmatic trials could lead to a greater focus on patient-centered outcomes and impact how clinicians view treatment options. Trials with pragmatic elements may be particularly well-suited to investigate complex interventions or conditions. The creation of a pragmatic clinical trial registry could facilitate greater pragmatic trial investigation, transparency, reporting, public awareness, and understanding by physicians and scientists.

References

1. Schwartz D and Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. Journal of Chronic Diseases. 1967;20(8):637-648.

2. Tunis S, Stryer D, Clancy C. Practical Clinical Trials: Increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290(12).

3. Roland M, Torgerson D. Understanding controlled trials: What are pragmatic trials?. BMJ. 1998;316(7127):285-285.

4. Patsopoulos N. A pragmatic view on pragmatic trials. Dialogues in Clinical Neuroscience. 2011; 13(2):217-224.

5. Ford I and Norrie J. Pragmatic trials. NEJM. 2016; 375(5):454-463.

6. Menezes A, Barnes A, and Scheer, A. Clinical research in surgical oncology: An analysis of ClinicalTrials.gov. Annuals of Surgical Oncology. 2013; 20: 3725-3731.

7. Zarin D, Keselman A. Registering a Clinical Trial in ClinicalTrials.gov. Chest. 2007;131(3):909-912.

8. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015; 350:h2147.