0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letters |

Time to Publication Among Completed Clinical Trials FREE

Joseph S. Ross, MD, MHS; Marian Mocanu, MD; Julianna F. Lampropulos, MD; Tony Tse, PhD; Harlan M. Krumholz, MD, SM
[+] Author Affiliations

Author Affiliations: Section of General Internal Medicine (Dr Ross), Section of Cardiovascular Medicine (Dr Krumholz), and Robert Wood Johnson Foundation Clinical Scholars Program (Drs Ross and Krumholz), Department of Medicine, Yale University School of Medicine, New Haven, Connecticut; Center for Outcomes Research and Evaluation, Yale–New Haven Hospital, New Haven (Drs Ross, Lampropulos, and Krumholz); Section of Cardiovascular Medicine, Department of Medicine, Mayo Clinic, Rochester, Minnesota (Dr Mocanu); Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, Maryland (Dr Tse); and Section of Health Policy and Administration, Yale University School of Epidemiology and Public Health, New Haven (Dr Krumholz).


JAMA Intern Med. 2013;173(9):825-828. doi:10.1001/jamainternmed.2013.136.
Text Size: A A A
Published online

Prior studies have shown that 25% to 50% of clinical trials are never published.14 However, among those published, we know little about the length of time required for publication in the peer-reviewed biomedical literature after study completion. Ioannidis5 previously demonstrated that a sample of randomized phase 2 and 3 trials conducted between 1986 and 1996 required nearly 2.5 years for publication, while our more recent study of National Institutes of Health (NIH)-funded trials found that the average time to publication was almost 2 years.4 We sought to determine time to publication for a recent and representative sample of trials published in 2009.

We examined clinical trials published during the 2009 calendar year that were indexed via MEDLINE and linked to a ClinicalTrials.gov identifier (National Clinical Trial [NCT] number).6 We excluded articles that were not the first publication reporting on the clinical trial by reviewing article text for citations to and searching MEDLINE for earlier trial publications. We imputed completion dates based on the subject enrollment dates and the longest period of follow-up required for any outcome measure reported in the article. For trials without enrollment dates, we used the completion date provided on ClinicalTrials.gov; this date was not used uniformly because our prior work demonstrated its unreliability.3 We determined publication dates using MEDLINE, using online publication dates for articles made available ahead of print. Because ClinicalTrials.gov reports completion dates in month/year format, publication dates were similarly recorded for purposes of calculating time to publication.

We obtained additional information through the National Library of Medicine, using the ClinicalTrials.gov identifier, including data elements reporting lead sponsor, study design, and condition studied (Table). We obtained journal impact factors from Web of Knowledge (Thomson Reuters). We used Kruskal-Wallis tests to compare median times to publication between trial characteristic categories using a type I error rate of 0.003 to account for multiple comparisons (n = 16). Statistical analysis was performed using JMP 7.0.1 software (SAS Institute Inc).

Table Graphic Jump LocationTable. Time to Publication Among Completed Clinical Trials Registered in ClinicalTrials.gov and Published in the Biomedical Literature (Cited in MEDLINE), Stratified by Trial Characteristics

There were 1336 clinical trials published during the 2009 calendar year, indexed via MEDLINE and linked to a ClinicalTrials.gov identifier. The majority of trials were funded by nonprofit organizations and enrolled more than 100 subjects (Table). Nearly half were published in journals with an impact factor less than 5.

Median time to publication was 21 months, with an interquartile range of 13 to 32 months (eFigure) and modest differences across types of trials (Table). For instance, median time to publication was longer among trials funded by industry compared with trials funded by government and nonprofit organizations (24 vs 20 months; P < .001), but was shorter among trials enrolling 1000 subjects or more compared with trials enrolling 100 subjects or fewer and trials enrolling between 100 and 1000 subjects (18, 20, and 23 months, respectively; P < .001) and was shorter among trials published in journals with an impact factor greater than 10 compared with trials published in journals with an impact factor less than 10 (17 vs 23 months; P < .001).

We found, on average, that nearly 2 years had passed between completion and publication of clinical trials, across all trial funders. Moreover, given that our study was necessarily limited to examining time to publication among completed trials that were eventually published, this estimate is conservative. First, we studied trials registered in and linked to ClinicalTrials.gov, a select group of studies. Because of policies in place as of 2005, many may have been registered to ensure compliance with International Committee of Medical Journal Editors requirements for publication7 and thus more likely to publish in a timely manner. Second, we only studied trials that were published (and indexed via MEDLINE and linked to ClinicalTrials.gov identifiers). Between 50% and 70% of studies registered in ClinicalTrials.gov are eventually published many years after trial completion3,4; 30% to 50% are never published.

Differences across trial types were generally modest and indicate that timely dissemination of research needs to be uniformly prioritized to enhance science. Many steps must be successfully taken between trial completion and publication, including data management, statistical analysis, and writing, along with processes out of investigators' control, such as peer review. The substantial variation in publication time, from months to years, suggests that there may be opportunities to improve the speed and efficiency with which investigators publish completed studies.

We cannot rule out that study findings may have been disseminated through means other than publication, including scientific meeting presentations. However, with the exception of public results reporting, these alternative dissemination strategies lead to limited public awareness of the research. Given the time required to publish results from these clinical trials, our findings support current federal initiatives requiring results reporting of clinical studies within 12 months of trial completion8 to ensure the timely dissemination of clinical science.

Correspondence: Dr Ross, Section of General Internal Medicine, Yale University School of Medicine, PO Box 208093, New Haven, CT 06520 (joseph.ross@yale.edu).

Published Online: March 4, 2013. doi:10.1001/jamainternmed.2013.136

Author Contributions: All authors had full access to all the data in the study and Dr Ross takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Ross, Tse, and Krumholz. Acquisition of data: Ross, Mocanu, Lampropulos, and Tse. Analysis and interpretation of data: Ross, Mocanu, Lampropulos, and Krumholz. Drafting of the manuscript: Ross. Critical revision of the manuscript for important intellectual content: Ross, Mocanu, Lampropulos, Tse, and Krumholz. Obtained funding: Krumholz. Administrative, technical, and material support: Mocanu and Tse.

Conflict of Interest Disclosures: Dr Ross reports that he is a member of a scientific advisory board for FAIR Health Inc. Dr Krumholz reports that he chairs a scientific advisory board for UnitedHealthcare. Dr Tse reports that he is a Program Analyst for ClinicalTrials.gov

Funding/Support: The analyses on which this publication is based were performed under Contract No. HHSN276201000788P, entitled “Evaluating Time-to-Publication Following Completion of Clinical Trials andRelated Factors,” funded by the National Library of Medicine, Department of Health and Human Services. Drs Ross and Krumholz receive support from Medtronic Inc and from the Centers of Medicare & Medicaid Services to develop and maintain performance measures that are used for public reporting. Dr Ross is supported by the National Institute on Aging (K08 AG032886) and by the American Federation for Aging Research through the Paul B. Beeson Career Development Award Program. Dr Tse is supported by the Intramural Research Program of the National Institutes of Health, National Library of Medicine. Dr Krumholz is supported by a National Heart Lung Blood Institute Cardiovascular Outcomes Center Award (1U01HL105270-02).

Disclaimer: The ideas and opinions expressed are the authors' own. The content of this publication does not necessarily reflect the views or policies of the US National Institutes of Health, Public Health Service, or Department of Health and Human Services. The authors assume full responsibility for the accuracy and completeness of the ideas presented.

Additional Contributions: Deborah A. Zarin, MD, of the Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, provided comments on an early version of the manuscript. Dr Zarin received no compensation for her review.

Dwan K, Altman DG, Arnaiz JA,  et al.  Systematic review of the empirical evidence of study publication bias and outcome reporting bias.  PLoS One. 2008;3(8):e3081
PubMed   |  Link to Article
Lee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug applications: a literature analysis.  PLoS Med. 2008;5(9):e191
PubMed   |  Link to Article
Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis.  PLoS Med. 2009;6(9):e1000144
PubMed   |  Link to Article
Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis.  BMJ. 2012;344:d7292
PubMed   |  Link to Article
Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials.  JAMA. 1998;279(4):281-286
PubMed   |  Link to Article
US National Institutes of Health.  ClinicalTrials.gov website. 2009. http://www.clinicaltrials.gov/. Accessed February 5, 2012
DeAngelis CD, Drazen JM, Frizelle FA,  et al; International Committee of Medical Journal Editors.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors.  JAMA. 2004;292(11):1363-1364
PubMed   |  Link to Article
Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database—update and key issues.  N Engl J Med. 2011;364(9):852-860
PubMed   |  Link to Article

Figures

Tables

Table Graphic Jump LocationTable. Time to Publication Among Completed Clinical Trials Registered in ClinicalTrials.gov and Published in the Biomedical Literature (Cited in MEDLINE), Stratified by Trial Characteristics

References

Dwan K, Altman DG, Arnaiz JA,  et al.  Systematic review of the empirical evidence of study publication bias and outcome reporting bias.  PLoS One. 2008;3(8):e3081
PubMed   |  Link to Article
Lee K, Bacchetti P, Sim I. Publication of clinical trials supporting successful new drug applications: a literature analysis.  PLoS Med. 2008;5(9):e191
PubMed   |  Link to Article
Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis.  PLoS Med. 2009;6(9):e1000144
PubMed   |  Link to Article
Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis.  BMJ. 2012;344:d7292
PubMed   |  Link to Article
Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials.  JAMA. 1998;279(4):281-286
PubMed   |  Link to Article
US National Institutes of Health.  ClinicalTrials.gov website. 2009. http://www.clinicaltrials.gov/. Accessed February 5, 2012
DeAngelis CD, Drazen JM, Frizelle FA,  et al; International Committee of Medical Journal Editors.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors.  JAMA. 2004;292(11):1363-1364
PubMed   |  Link to Article
Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database—update and key issues.  N Engl J Med. 2011;364(9):852-860
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment

Multimedia

Supplemental Content

Ross JS, Mocanu M, Lampropulos JF, Tse T, Krumholz HM. Time to publication among completed clinical trials. JAMA Intern Med. Published online March 4, 2013. doi:10.1001/jamainternmed.2013.136

eFigure. Time to publication after completion among clinical trials registered in ClinicalTrials.gov and published in the biomedical literature (cited in MEDLINE)

Supplemental Content

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 6

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles