Author Affiliations: Masspro, Waltham, Massachusetts (Dr LaBresh and Ms Albright); Department of Epidemiology, Michigan State University, East Lansing (Dr Reeves); School of Medicine, Emory University, Atlanta, Georgia (Dr Frankel); and Stroke Service, Massachusetts General Hospital, Boston (Dr Schwamm).
Adherence to evidence-based interventions for hospitalized patients who have experienced a stroke is suboptimal. We examined the association of process improvement and Internet-based data collection and decision support with stroke care.
A 1-year intervention study to assess performance measure adherence in hospitals using the “Get With The Guidelines–Stroke” program. The program included 18 410 patients with ischemic stroke or transient ischemic attack admitted to 99 volunteer community and teaching hospitals. Data from eligible patients in the preintervention baseline period were compared with data from 4 subsequent quarters for 12 acute care or secondary prevention measures and an all-or-none measure of care based on 7 prespecified measures.
Significant improvements from baseline to the fourth quarter were seen in 11 of 13 measures: use of thrombolytic medications for patients with ischemic stroke presenting within 2 hours of onset, 23.5% vs 40.8% (P < .001); early use of antithrombotic medications, 88.2% vs 95.2% (P < .001); antithrombotic medications prescribed at discharge, 91.0% vs 97.9% (P < .001); anticoagulation agents for atrial fibrillation, 81.4% vs 96.5% (P < .001); smoking cessation counseling, 38.3% vs 54.5% (P < .001); lipid treatment for low-density lipoprotein levels 100 mg/dL or greater, 58.7% vs 77.0% (P < .001); diabetes mellitus treatment, 48.5% vs 83.5% (P = .001); and weight reduction counseling 32.5% vs 43.4% (P < .001). The all-or-none measure increased from 50.2% to 58.0% (P < .001). Complications from thrombolytic medications and prophylaxis for deep venous thrombosis did not change.
Statistically and clinically significant improvement in 11of 13 quality improvement measures for the treatment of patients hospitalized for cerebrovascular disease was seen in hospitals participating in the Get With The Guidelines program.
Cerebrovascular disease is the third leading cause of death in the United States and a leading cause of serious long-term disability.1 Despite the wide dissemination of evidence-based guidelines, recommended interventions are frequently not initiated during hospitalization for acute events.2- 7
Barriers cited for this gap between efficacy (demonstrated usefulness in clinical trials) and effectiveness (demonstrated usefulness in clinical practice) include lack of knowledge, lack of acceptance of the concept of guidelines, lack of systems for implementation, and lack of resources.8 Typical educational responses to this treatment gap include didactic presentations, such as grand rounds, journal articles, and other efforts to disseminate the guidelines. Recognizing that knowledge and acceptance of guidelines, although necessary, do not notably change guideline adherence,8,9 the American Heart Association/American Stroke Association (AHA/ASA) launched the “Get With The Guidelines–Stroke” program (GWTG-Stroke), an initiative focused on the redesign of hospital systems of care.10
This program is based on a collaborative model and the Internet-based Patient Management Tool (PMT) (Outcome Inc, Cambridge, Massachusetts).11,12 The collaborative learning model13,14 includes interactive learning sessions, teleconference, and electronic communication between multidisciplinary teams from hospitals in a variety of settings to facilitate the transfer of the “how-to” information necessary to produce system change.15 The PMT supports concurrent data collection and provides clinical decision support and real-time online reporting. Using this model in 1738 hospitalized patients with cardiovascular disease over a 1-year period, a diverse group of 24 Massachusetts hospitals produced statistically and clinically significant changes in smoking cessation counseling, lipid measurement and treatment, blood pressure treatment and control, and referral to cardiac rehabilitation while maintaining high baseline levels of aspirin, β-blocker, and angiotensin-converting enzyme inhibitor use.16
The purpose of this evaluation was to assess the quality of care of cerebrovascular disease patients in 99 hospitals across the United States participating in GWTG-Stroke.
The components of GWTG-Stroke include organizational stakeholder and opinion leader meetings, hospital recruitment, collaborative workshops for hospital teams, hospital tool kits, local clinical opinion leaders, and hospital recognition.13 Data collection, decision support, and hospital data feedback via multiple on-demand reports of performance on all key measures, are performed with the Internet-based PMT.
Hospitals in a given region are recruited to participate in GWTG-Stroke by local ASA staff and volunteers, working with state hospital associations and departments of health. All participating hospitals volunteer to participate based on their level of interest in quality improvement in stroke care and capacity to fulfill the requirements of the program. Workshops include didactic presentation of clinical trial evidence and the ASA guidelines for acute treatment and secondary prevention of stroke and other major vascular events, followed by examples of successful hospital implementation. These workshops are given quarterly by ASA staff and volunteers with expertise in clinical science and quality improvement. Standardized quality improvement methods based on the Model for Improvement13,14 are presented at each session. Simultaneous, facilitated breakout sessions, a key part of the GWTG-Stroke workshops, allow multidisciplinary teams from 6 to 8 hospitals to discuss barriers and potential solutions, share tools and ideas each has developed, and share results of their small tests of change. Typical team members include a stroke nurse, a quality improvement nurse, and a physician leader. Each team then develops a plan for testing and implementation and presents their plans to other participants to create a sense of purpose, urgency, and accountability to their peers. Between workshops, hospital interactions are continued via conference calls and e-mail exchanges. Guest faculty presentations are included in the calls to address emerging issues around new clinical data and guidelines.
This analysis includes data from all of the 18 410 patients from 99 participating hospitals, entered into the PMT for 12 months (April 2003 through March 2004), after the initial baseline collection. The baseline collection was done prior to the quality improvement intervention. All hospitals in 8 selected geographic regions (Georgia, Massachusetts, Michigan, Ohio, Florida, Arizona, northern California, southeastern Pennsylvania) were invited to participate. The 99 participating hospitals were characterized based on self-reported status as teaching vs nonteaching (based on the presence of at least 1 residency training program) and large vs small (based on number of beds and number of stroke admissions) status. Data were collected by participating hospitals without financial compensation; all hospitals that submitted baseline data continued to participate for the 1-year intervention.
Case ascertainment of admissions for ischemic stroke or transient ischemic attack (TIA) was conducted by prospective clinical identification, retrospective identification using International Classification of Diseases, Ninth Revision (ICD-9)17 discharge codes (followed by medical chart review to confirm case eligibility), or a combination of both approaches. Retrospective identification of stroke admissions was performed using ICD-9 codes 433, 434, or 436, whereas ICD-9 codes beginning with 435 were used to identify TIA admissions. To ensure that patients entered in the registry as having TIA truly had deficits caused by ischemia rather than vague symptoms classified as TIA, hospitals were instructed to include patients with transient symptoms as TIA only if symptoms were still persistent at the time of hospital arrival. A final diagnosis of TIA was subsequently confirmed if symptoms lasted less than 24 hours and no alternative diagnosis was identified. Data from the first 30 patients entered in the PMT at each hospital prior to the start of the intervention were used as baseline data; the duration of baseline data collection thus varied as a function of the hospitals’ admission volume (mean duration, 2 months).
Measures assessed in this quality improvement program are listed in detail in Table 1. Indicator-specific inclusion and exclusion criteria were applied to create a denominator of patients eligible for each specific indicator. Patients were excluded if there was insufficient information to assess eligibility or if a contraindication was documented as the reason for withholding the intervention. The patients remaining in the denominator reflect those who should with certainty be candidates for the intervention in question.
An all-or-none measure was constructed based on the following 7 prespecified performance measures: use of thrombolytic medications for patients with ischemic stroke presenting within 2 hours of symptom onset, antithrombotic medications administered within the first 48 hours of admission (early administration), deep venous thrombosis (DVT) prophylaxis, antithrombotic medications at discharge, anticoagulation agents for atrial fibrillation, lipid treatment for low-density lipoprotein (LDL) cholesterol levels of 100 mg/dL or higher, and smoking cessation counseling (to convert LDL cholesterol to millimoles per liter, multiply by 0.0259). These 7 measures are used as the core set of measures based on the strength of evidence in clinical trials and published guidelines. The all-or-none measure was defined as the percentage of patients receiving all of the 7 performance measures for which they were eligible. Thus, any patient failing to receive 1 or more of the performance measures for which they were eligible has not passed the all-or-none care measure.
Data were collected for each hospitalization, including patient demographics, medical history, findings from initial head computed tomography, in-hospital treatment and events, discharge treatment and counseling, and discharge destination. The data collection tool supports concurrent data collection as well as retrospective data entry; concurrent collection was encouraged as a process improvement goal in each hospital. The GWTG-Stroke data collection tool included predefined logic features and user alerts to identify potentially invalid format or values entry (Outcome Inc). Required fields were structured so that valid data must be entered before the form can be saved as a complete record and the data entered into the database. Range checks were used for inconsistent or out-of-range data and prompted the user to correct or review data entries that were outside a predefined range. All hospital personnel using the tool received individual passwords to create an audit trail for data entered or changed. Training in the use of the tool was provided online and via telephone for all users. This Web-based system used coding, deidentification, and secure transmission techniques that maintain patient confidentiality, in compliance with current federal privacy standards. Data collected by hospitals were not independently audited by external medical chart review.
All analyses were performed using SAS statistical software (SAS Institute, Research Triangle, North Carolina). Data are presented as the proportion of eligible patients who met criteria for each measure. When data elements required for a specific measure were missing, that patient was excluded from the analysis for the measure. Changes in adherence rates for the measured outcomes over the study period were assessed using the Mantel-Haenszel χ2 test for time trend analysis. All P values are 2-sided, with P < .05 considered to be statistically significant. The Fisher exact test was used to compute 95% confidence intervals for adherence rates. The distribution of individual hospital performance was determined as a simple proportion of hospitals performing below the mean value of performance for each measure.
All participating hospitals received either human research approval to enroll patients in GWTG-Stroke or received a waiver of authorization and consent, and exemption from subsequent review from their institutional review board, based on the use of deidentified records and absence of direct patient contact in this quality improvement activity.
Data from 18 410 patients at 99 hospitals were included in this analysis. The demographic distribution and clinical characteristics of the patient group are shown in Table 2. The population was predominantly white, with more men than women and a mean (SD) age of 75 (14.3) years. The medical history included a high prevalence of hypertension, diabetes mellitus, hypercholesterolemia, and smoking. Acute ischemic stroke accounted for 75.1% of the admissions.
The 99 hospitals in the study represent a diverse group by size, ischemic stroke volume, and geographic distribution by census region. Most of the hospitals were classified as nonteaching because they did not have residency programs (Table 3).
The use of thrombolysis (tissue plasminogen activator [rt-PA]) in those patients with ischemic stroke presenting within the first 2 hours after symptom onset increased by 17.3% (P < .001) (23.5% vs 40.8%), documentation of contraindications for receiving rt-PA rose from 64.0% to 86.2% (P < .001), whereas the rate of complications did not increase and remained below the National Institute of Neurologic Disorders Study18 rate of 6.4%. Early antithrombotic use increased substantially by 7.0%, whereas prophylaxis for DVT did not change (Table 4 and Figure 1).
All-or-none and performance measure adherence over 4 quarters (Q) for 18 410 patients. AF indicates atrial fibrillation; DC, discharge; DVT, deep venous thrombosis; LDL100, lipid-lowering treatment if the low-density lipoprotein (LDL) cholesterol level is 100 mg/dL or higher or the patient is receiving treatment on admission; smoking, smoking cessation counseling. To convert LDL cholesterol to millimoles per liter, multiply by 0.0259. *P < .001. †P = .10.
Baseline and quarterly data demonstrated significant increases (P < .001) in all 8 individual secondary prevention measures over the intervention period (Table 4 and Figure 1). Substantial improvements were seen in hypoglycemic treatment with oral agents or insulin for patients with diabetes mellitus (19.6%), lipid measurement (14.9%), lipid treatment overall (21.4%), lipid treatment for LDL cholesterol level of 100 mg/dL or higher (18.3%), smoking cessation counseling (15.7%), and anticoagulation for atrial fibrillation (15.1%), with more modest improvement in weight-loss counseling for patients with a body mass index (calculated as weight in kilograms divided by height in meters squared) greater than 25, and in patients receiving antithrombotic agents at discharge.
The all-or-none care measure, a composite of 7 measures that included lytic therapy, early and discharge antithrombotic therapy, lipid-lowering therapy for those with an LDL cholesterol level of 100 mg/dL or higher or those receiving therapy on admission, DVT prophylaxis, and anticoagulation agents for atrial fibrillation, increased by 7.8% (P < .001).
Measure performance across hospitals varied widely, even in the fourth-quarter data, as shown in Figure 2 for select measures. Variability between hospitals was low in the high-performing measures, such as early prescribing of antithrombotic agents, whereas more hospital-hospital variability was observed in the lower-performing measures; the percentage of hospitals that had performance levels below the mean was 68% for rt-PA use, 52% for LDL cholesterol therapy, 59% for smoking cessation counseling, and 46% for all-or-none measure performance.
Fourth-quarter performance by hospital for selected measures. A, Patients receiving all-or-none care; B, patients who received intravenous tissue plasminogen activator (IV rt-PA) within 3 hours of onset of stroke symptoms if they arrived at the hospital within 2 hours; C, patients receiving antithrombotic medication within 48 hours of hospitalization; D, patients discharged while receiving antithrombotic medication; E, smokers who received smoking cessation counseling; and F, patients with a low-density lipoprotein (LDL) cholesterol level higher than 100 mg/dL or undocumented LDL cholesterol who were discharged while receiving cholesterol-lowering drugs. Each hospital performance is indicated as a bar for each measure. Hospitals are arranged by relative performance by measure from highest performance (as high as 100%) on the left to the lowest performance (as low as 0%) on the right. To convert LDL cholesterol to millimoles per liter, multiply by 0.0259.
This GWTG-Stroke quality improvement project represents the first large-scale effort to translate efficacy into effectiveness in stroke care for hospitalized patients. There have been only a handful of previous quality improvement studies that have reported on the treatment of hospitalized patients with stroke. A recent collaborative study19 conducted in 13 Michigan hospitals also using the GWTG-Stroke PMT found clinically and statistically significant improvements in smoking cessation, dysphagia screening, use of the National Institutes of Health stroke scale, and documentation of the nonuse of rt-PA and a medical history of dyslipidemia. Other stroke-related quality improvement projects include an initiative,20 sponsored by the Centers for Medicare and Medicaid Services, that found improvements in the use of anticoagulants for atrial fibrillation, antithrombotics agents at discharge, and avoidance of sublingual nifedipine among Medicare beneficiaries admitted with ischemic stroke or TIA. A similar study21 among Michigan Medicare beneficiaries examined additional indicators and found improvements in the use of computed tomography or magnetic resonance imaging during hospitalization and in DVT prophylaxis. The Preventing Recurrence of Thromboembolic Events through Coordinated Treatment study22 was a single-institution project that used tools, protocols, and physician education to improve the provision of secondary prevention therapies, counseling, and education prior to hospital discharge. Other reports23,24 focusing on the quality of hospital care for patients with stroke include the positive effects of the use of written care orders.
Evaluation of 99 participating hospitals during the time frame of this analysis of the program shows a rapid and marked improvement in the provision of 11 of 12 evidence-based acute care and secondary prevention interventions, and the all-or-none measure of defect-free care. In less than 12 months, the hospital group, representing a range of size, teaching status, and geographic location, was able to improve the quality of acute stroke care. The all-or-none measure of care at the patient level for the 7 measures used for its construction represents a much higher level of reliability for hospitals by its all-or-none nature.25 There was an absolute increase in the percentage of patients receiving defect-free care in each quarter of intervention, reaching 8% by the fourth quarter; this represents an additional 891 patients receiving all of the 7 measures of care for which they were eligible during the 1-year intervention compared with the number of patients receiving such care at baseline. For those not receiving this level of care, many received improved care as indicated by hospital progress in individual measures.
This study specifically looked at a balancing measure and the number of complications from rt-PA as well as the documentation of contraindications to this therapy. With the emphasis on increased use of rt-PA therapy for ischemic stroke presenting within 2 hours of symptom onset, it was important to monitor the potential for inappropriate use of rt-PA, as manifested by increased complications. Although this attention to documentation may have produced some reduction in the denominator of eligible patients in the rt-PA treatment measure, it seems likely that the development of systems to assess and document contraindications played a role in maintaining a low complication rate. This is an illustration of the important role of increased documentation in improving the quality of care.26 The improvement in stroke care quality seen in this project may be related to multiple supportive systems, including a collaborative environment and the PMT that facilitated concurrent feedback of guideline information at the point of care. The intuitive layout, embedded data definitions, and data entry instructions available online as drop-down screens for each data element simplified training and supported rapid assimilation into hospital data entry routines. The fact that not all of these hospitals were able to achieve the high levels of performance indicates that the process of changing hospital systems and culture is both complex and challenging. Even when individual measure performance is high, the reliability of these systems in delivering appropriate care to every patient remains a challenge, with about half of patients not receiving all of a group of accepted, evidence-based care interventions that have been demonstrated to improve outcomes.
This analysis has limitations. We utilized a prevention and postintervention design to assess the effect of this quality improvement program without the use of randomization or a control group. Because input into the database is part of the intervention, it is not possible to assess the performance of nonparticipating hospitals as part of a control sample. Hospitals were invited to participate and were accepted based on self-selection and not randomly selected. This makes it likely that these hospitals are more committed to quality improvement in stroke care or that they had low performance and a greater motivation to improve care. In either case, their level of performance may not be generalizable. The change in these indicators observed across the time period of the study could, in part, represent a general improvement in care provided by the hospital as a whole. In the absence of a randomized, concurrent, control group of hospitals and patients, we cannot fully exclude secular trends and/or the impact of other efforts accounting for some of the quality improvement observed. However, given the magnitude of improvements in care for most indicators over the 1-year period of the intervention, secular trends are unlikely to be the entire explanation for the changes in care observed.
Similarly, we cannot confirm that the cases entered into the GWTG-Stroke PMT represent a consecutive or unbiased patient sample. The purpose of data collection was to identify gaps in care that were not explained by individual patient characteristics and that could be corrected with protocols, reminders, and other system change elements. Hospitals were instructed to include all consecutive cases, but the extent to which that occurred was not independently audited. Thus, the potential exists for selection bias. However, there is continuous improvement in adherence over time observed in this cohort rather than a discrete and sudden change in performance, which suggests that case selection bias is not the driving force behind the change in adherence over time. Although concurrent data collection enables the use of embedded decision support, hospitals in this project may have used retrospective medical chart abstraction or concurrent data entry, and many were assisted in a transition to concurrent collection during the project. Thus, the role of data collection methods cannot be fully discerned in this analysis.
In conclusion, hospital participation in the AHA/ASA GWTG-Stroke program was associated with rapid and marked improvements in the quality of care during hospitalization for acute stroke and TIA. Despite this important finding, there remains much to learn about this process and more work to be done to optimize cerebrovascular care.
Correspondence: Kenneth A. LaBresh, MD, Masspro, 245 Winter St, Waltham, MA 02451 (firstname.lastname@example.org).
Accepted for Publication: September 16, 2007.
Author Contributions: Dr LaBresh had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: LaBresh, Frankel, and Schwamm. Acquisition of data: LaBresh, Frankel, and Schwamm. Analysis and interpretation of data: LaBresh, Reeves, Frankel, Albright, and Schwamm. Drafting of the manuscript: LaBresh, Frankel, and Schwamm. Critical revision of the manuscript for important intellectual content: LaBresh, Reeves, Frankel, Albright, and Schwamm. Statistical analysis: Reeves and Albright. Obtained funding: LaBresh. Administrative, technical, and material support: LaBresh and Schwamm. Study supervision: LaBresh, Frankel, and Schwamm.
Financial Disclosure: Dr Schwamm has received an unrestricted education grant from Genentech and honoraria from Alpha Medical, Discovery Institute of Medical Education, and is on the medical advisory board of Cordis.
Funding/Support: This study was supported in part by an unrestricted educational grant to the American Heart Association by Pfizer Inc.
Role of the Sponsor: Pfizer Inc did not participate in the design, analysis, or preparation of the manuscript.
Thank you for submitting a comment on this article. It will be reviewed by JAMA Internal Medicine editors. You will be notified when your comment has been published. Comments should not exceed 500 words of text and 10 references.
Do not submit personal medical questions or information that could identify a specific patient, questions about a particular case, or general inquiries to an author. Only content that has not been published, posted, or submitted elsewhere should be submitted. By submitting this Comment, you and any coauthors transfer copyright to the journal if your Comment is posted.
* = Required Field
Disclosure of Any Conflicts of Interest*
Indicate all relevant conflicts of interest of each author below, including all relevant financial interests, activities, and relationships within the past 3 years including, but not limited to, employment, affiliation, grants or funding, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. If all authors have none, check "No potential conflicts or relevant financial interests" in the box below. Please also indicate any funding received in support of this work. The information will be posted with your response.
Some tools below are only available to our subscribers or users with an online account.
Download citation file:
Web of Science® Times Cited: 79
Customize your page view by dragging & repositioning the boxes below.
The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
Updated Literature Search
The Rational Clinical Examination: Evidence-Based Clinical Diagnosis
Evidence to Support the Update
All results at
Enter your username and email address. We'll send you a link to reset your password.
Enter your username and email address. We'll send instructions on how to reset your password to the email address we have on record.
Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.