The Impact of Standardized Order Sets and Intensive Clinical Case Management on Outcomes in Community-Acquired Pneumonia FREE

Community-acquired pneumonia (CAP) results in approximately 600 000 annual admissions to acute care hospitals in the United States, resulting in greater than 4 200 000 inpatient days.¹ Annual spending for CAP hospital admissions is greater than $8.4 billion.² An important determinant of hospital costs and efficiency is the length of hospital stay (LOS). Furthermore, excessive LOS may adversely affect patient outcomes by increasing exposure to nosocomial infections, deep venous thrombosis, deconditioning, and other medical complications. Previous studies have found remarkable variability between hospitals in CAP LOS.^3- 5 Importantly, McCormick et al⁶ found that shorter LOS did not adversely affect quality outcomes. This suggests that opportunities exist to safely reduce hospital LOS for CAP while maintaining good patient outcomes.

One approach to improve LOS in CAP is the use of clinical guidelines and pathways. Studies in CAP have generally found improved processes and outcomes, including mortality, with adherence to guidelines.^7- 10 This may be less true for LOS. Halm et al¹¹ tested a multifaceted intervention in which a key aspect was guideline use and found no improvement. However, in that study, no particular effort was made to support use of the guideline. The intensity of implementation of a guideline may greatly impact its effect on processes of care, as found in a study of CAP by Yealy et al.¹⁰ We hypothesized that the addition of tracking of critical events and detection of variances (hereinafter, variance tracking) and concurrent clinical feedback and reminders would result in improved efficiency of care for CAP. We term this combined intervention intensive clinical case management (ICCM). The purpose of the current study, therefore, was to rigorously evaluate the effectiveness of ICCM combined with guidelines to reduce LOS while maintaining quality in the treatment of CAP.

Winthrop University Hospital is a 591-bed teaching hospital located in Mineola, Long Island, New York. All admissions for CAP between November 2002 and February 2005 were potentially eligible for study. The criteria for a CAP diagnosis were chest radiograph findings consistent with pneumonia and at least 2 of the following: (1) temperature higher than 38°C, (2) productive cough, (3) chest pain, (4) dyspnea, and (5) crackles on auscultation. Patients were excluded from entry if there was a hospitalization in the previous 30 days, initial intensive care unit admission, active immunosuppression (any immunosuppressive medications, including steroids), AIDS, active malignancy (undergoing treatment, except skin cancer), cystic fibrosis, or if the patient refused active treatment (antibiotics).

The purpose of the study was to compare CAP treatment using conventional methods (no intervention) with treatment guided by standardized order sets (SOSs) (guidelines) with or without ICCM. We used a sequential course of study, with 3 consecutive blocks (sequences). Each block remained open until approximately 110 patients were enrolled.

Block 1 patients underwent treatment not guided by SOSs or ICCM. Clinicians wrote orders using the hospital's usual blank order form. Data were collected on a daily basis by a research assistant not involved in patient care. There was no tracking of variances or feedback to clinicians.

For block 2 patients, clinicians were reminded to use the SOS every hospital day. The ICCM review began with a daily check of the patient's critical events. When a critical event order was not completed as specified by the SOS, or if the care process was not completed on a timely basis, the clinician was called to discuss the necessary care process. The ICCM case manager intervened as appropriate to facilitate processes, to help with delays in testing, and to identify other issues that could hinder quality or efficiency; in most cases this required daily contact with the physician in person or by telephone. The case manager placed particular emphasis on prompting for timely conversion to oral antibiotics and discharge.

For block 3 patients, the clinician was reminded to write orders on a daily basis using the CAP SOS. There was no ICCM intervention.

The study protocol was approved by the Winthrop University Hospital institutional review board without the need for informed consent.

Three different ICCM case managers participated in block 2; all were medical residents trained in the goals and evidence supporting the SOS. These were volunteers who were not paid for this task. All performed the daily tasks of ICCM including assessment of clinical and critical events, detection of variances, and reminders and prompting to treating clinicians. All of the residents performed as ICCM case managers during clinical rotations of their own, but in no case were their own cases involved. The daily time involvement was 1 to 2 hours. All data collected were concurrent from the medical record; some data elements such as mental status required interaction with nursing staff or treating physicians.

The Winthrop University Hospital CAP SOS is a preprinted order form that covers all anticipated orders for all hospital days for patients hospitalized with CAP. The orders are paper based, printed off the hospital's computer system at the time of admission, and formatted in the “ADC VAN DISSEL” format familiar to physicians. They are designed to be all encompassing, potentially covering all orders required for the patient. Clinicians are encouraged, however, to adapt the orders to the needs of the individual patient. The SOS is designed to promote quality and efficiency of care and to reduce errors of omission. Critical events on the SOS included correct choice of appropriate antibiotics, thromboembolism prophylaxis, adult vaccination, change from parenteral to oral antibiotics, progressive ambulation, smoking cessation counseling, and hospital discharge order. The SOS is updated every year based on changes in evidence and practice.

During the block 3 sequence, the hospital initiated performance improvement projects involving adult in-hospital vaccination and smoking cessation counseling. Since there was the potential to affect 3 of the secondary outcome measures, block 3 data for influenza and pneumococcal vaccination and smoking cessation were not analyzed.

Patients were considered to have reached clinical stability when daily temperatures were 37.2°C or lower, respiratory rates were lower than 24 breaths per minute, arterial oxygen saturation was higher than 92% (adapted from Halm et al¹²), and dyspnea index was 1 (no shortness of breath at rest). Patients who died were excluded from analysis for clinical stability.

We estimated that the LOS in block 2 would differ from that in blocks 1 and 3 by 1.5 days (a 20% difference). The within-group standard deviation was estimated at 4 days. A 2-tailed 1-way analysis of variance at the .05 level of significance would have a power of 80% with 105 patients per block. These parameters translate into a Cohen f² effect size of 0.032. This proved to be conservative: the actual effect size after the conclusion of the study was 0.13.

The predefined primary outcome measure was mean LOS. A secondary analysis was conducted with LOS outliers excluded (>2 SDs above the mean; ie, >14 days), since an extremely long stay is likely to reflect medical complications rather than care practices.

Analyses were performed on the raw data and also using nonparametric ranked data. Analysis of variance models were run both including and excluding covariates representing age and Mini-Mental State Examination scores as these were correlated with outcome. Because results were robust and did not depend on the model or transformations of the data, only the parametric results on the raw data are reported, without adjusting for age or Mini-Mental State Examination score.

All results for continuous variables are presented as means ± SDs. The Pneumonia Severity Index was calculated as described by Fine et al.¹³ Differences in group results for continuous variables were analyzed with 1-way analysis of variance. Mortality rate refers to in-hospital mortality; readmissions are within 30 days to the same hospital for any diagnosis. Progressive ambulation was defined as a change in orders for activity and documentation in the medical record that increased activity was achieved. Post hoc differences between individual groups were performed using the Tukey-Kramer multiple comparisons test. For discrete variables, differences were analyzed using the χ² test. P values less than .05 were considered to be statistically significant.

Block 1 ran from November 1, 2002, through March 31, 2003; block 2 from April 1, 2003, through March 31, 2004; and block 3 from June 1, 2004, through February 28, 2005. Baseline demographic and clinical data are reported in Table 1. The only statistically significant difference between the groups at baseline was a lower mean oxygen pressure in block 1 than in either block 2 or 3, but with no significant difference in arterial oxygen saturation. The mean pneumonia severity index was 88.9, not statistically different between the blocks.⁵ Mean mental status was not statistically different at baseline.

The mean ± SD LOS for all patients was 7.1 ± 4.0 days. The distribution of LOS is displayed in Figure 1. The mean LOS was 5.3 ± 3.5 days in block 2, significantly lower than the 8.8 ± 4.4 days in block 1 (P<.001) and 7.3 ± 3.9 days in block 3 (P<.01) (Table 2). The mean LOS in block 3 was significantly shorter than in block 1 (P = .01). The distribution of LOS classified by block is displayed in Figure 2. When outliers with LOS longer than 14 days (2 SDs above mean) are removed, the mean LOS for block 2 was still significantly lower than the LOS in blocks 1 and 3 (5.2 ± 2.0 vs 7.5 ± 2.8 and 6.1 ± 2.6 days for block 2 vs blocks 1 and 3 respectively) (block 2 vs block 1, P<.001; block 2 vs block 3, P<.05).

In addition to block of study, only age and mental status level on admission were significant predictors of LOS (for age, r = 0.19 and P<.003; for Mini-Mental State Examination score, r = −0.30 and P<.001). Including age and Mini-Mental State Examination score in the analysis of variance model did not appreciably or significantly change the results. In addition, the ICCM plus SOS intervention (block 2) was highly effective for reduction of LOS at all 4 quartiles of age and mental status, with no difference in magnitude of effect between quartiles (P = .32).

An important determinant of LOS for CAP is change from parenteral to oral antibiotics, which generally occurs after the patient achieves clinical stability (Figure 3). Clinical stability was reached at mean ± SD hospital day 3.3 ± 1.3. There was no significant difference in the day of clinical stability for the different blocks: block 1, day 3.3 ± 1.4; block 2, day 3.2 ± 1.2; and block 3, day 3.4 ± 1.3 (P = .72).

The change to oral antibiotics was significantly earlier in block 2 (3.7 ± 0.9 days) than in blocks 1 and 3 (5.7 ± 2.4 and 5.0 ± 1.9 days, respectively) (P<.001 for both). The mean ± SD time from clinical stability to change to oral antibiotics was significantly longer for block 1 (2.4 ± 1.4 days) than for block 2 (0.5 ± 1.3 days) and block 3 (1.6 ± 1.3 days) (P = .01 for block 1 vs block 2 and block 3). The mean time from the change to oral antibiotics to hospital discharge was 3.1 ± 2.2 days, 1.6 ± 1.6 days, and 2.3 ± 2.0 days in blocks 1, 2, and 3, respectively (P = .01 for block 2 vs block 1 and block 3). The mean time from clinical stability to hospital discharge was significantly shorter for block 2 (2.1 ± 2.2 days) than for block 1 (5.3 ± 4.4 days) (P<.001) and block 3 (4.9 ± 4.2 days) (P<.001) (Figure 3).

A greater proportion of the patients in block 2 had progressive ambulation during the hospital stay (P<.001). A significantly greater proportion of eligible patients in block 2 than in block 1 received pneumococcal (P<.001) or influenza vaccination (P = .01), and a greater proportion of smokers received counseling for smoking cessation (P = .01). In blocks 2 and 3, the rate of deep venous thrombosis prophylaxis was significantly greater than in block 1 (P<.001) (Table 2). There was no significant difference between the blocks in mean time to administration of initial antibiotics, hospital mortality rate, number of chest radiographs, correct American Thoracic Society antibiotic selection, or 30-day hospital readmission rate.

We found a significant reduction in LOS with the use of SOSs combined with ICCM. The magnitude of reduction was substantial: 3.5 days (39.8%) vs conventional therapy. There was no difference in pneumonia severity or time to clinical stability between the groups, and thus the reduction in LOS appears to be mainly due to care processes as a result of the SOS plus ICCM intervention.

Previous studies have failed to demonstrate that guidelines and SOSs result in reduced LOS.¹¹ Weingarten and colleagues¹⁴ used a different design of an alternate month application of a guideline for CAP and observed no benefit compared with months when the guideline was not used. However, the intensity of implementation appeared to be different from the efforts made in block 2 of the present study, and the focus of their guideline was less broad than in our study. Similarly, we found only a marginal reduction in LOS in block 3 (SOS alone) compared with controls. This is in contrast to the significantly reduced LOS in block 2 (SOS plus ICCM) compared with either controls or SOS alone. Therefore, ICCM may be central to the ability to reduce LOS, and SOSs and guidelines used without ICCM are probably considerably less effective. Guidelines and SOSs can encourage certain treatment processes but lack the ability to reinforce and integrate real-time changes in clinical status with appropriate changes in care. The ICCM intervention leverages the skills of a clinically trained individual who can monitor clinical status, measure progress and achieved critical events, and map these against the SOS and expected processes and events. These findings are similar to those of Yealy et al,¹⁰ who found that increased intensity of guideline implementation was critical for their success.

The time from hospital admission to clinical stability was not significantly different among the 3 groups. Despite this, patients in the SOS plus ICCM group were changed to oral antibiotics a mean of 2 days earlier than those in the conventional treatment group. This reflects both daily prompts on the SOS to consider change to oral administration as well as queries from the ICCM manager. Earlier switch to oral antibiotics is an important driver of reduced LOS in CAP.^15- 18 After changing to oral antibiotics, the clinically stable patient probably requires little additional in-hospital observation.^9,19- 20 We found that with conventional treatment, most patients remained in the hospital an additional 2 to 3 days after change to oral antibiotics. Patients in the SOS plus ICCM group were not only discharged significantly earlier after the switch than control patients, but significantly earlier than patients cared for with SOS alone. This is further evidence of the need for greater implementation intensity, provided by the addition of ICCM to SOS.

Reduction in LOS could potentially adversely affect patient outcomes if patients were discharged prematurely. Importantly, we found that there was no increase in the rate of hospital readmission despite the dramatic reduction in LOS in block 2. In addition, the mortality rate was not significantly different among block 2 patients. Therefore, reduced LOS for CAP in block 2 did not adversely affect patient outcomes. However, the study may not have been adequately powered to exclude differences in these outcomes. The improved LOS in the SOS plus ICCM group was accompanied by improvements in process quality measures as well. In particular, there was increased use of appropriate deep venous thrombosis prophylaxis, smoking cessation counseling, and both pneumococcal and influenza vaccination. These findings are similar to the observations made by Yealy et al¹⁰ that with more intensive application of guidelines, processes of care improve.

To study the effectiveness of our combined intervention, we believed that none of the prior study designs of guideline implementation were suitable. A parallel-group, randomized controlled trial was not an optimal approach because the intervention could unduly affect and contaminate the control group. For example, a physician could admit 2 patients with pneumonia in a single night, with one randomized to the control group and the other to an intervention group. In this case, it might be impossible to avoid having the clinical prompts and reminders of SOS and ICCM not affect and contaminate the treatment processes of the control patient. A multihospital approach, with an intervention used at some hospitals and not used at others, has been employed in similar studies. We believe that this method is suboptimal because of the great differences between hospitals, leading to excessive heterogeneity. Use of retrospective controls in a single hospital is problematic because of changes in patterns of care that occur over time.

Therefore, the design we selected was a sequential 3-phase study at a single hospital site. By progressing in sequence from controls to the combined intervention to guidelines alone, we accomplished 3 important goals: (1) controls were tested prior to any learning effect; (2) the most intensive treatment block, SOS plus ICCM, was tested immediately after controls to avoid a learning effect; and (3) by placing SOSs last in the sequence, any learning effect would be concentrated in this block, positively biasing this block while negatively biasing the primary tested intervention in block 2. This further strengthens the finding of improved LOS in block 2.

The passage of time with associated sequential improvements in care processes did not appear to explain the improved LOS with the SOS plus ICCM intervention. During the time of this study, the overall hospital LOS decreased by 3.1%. This is far less than the 39.8% reduction in LOS with the combined intervention compared with controls. Furthermore, at other hospital members of the Long Island Health Network (Winthrop is the largest of 10 member hospitals), mean LOS for CAP fell by only 2.6% during the study period.

In conclusion, the combined use of ICCM and SOSs led to a substantial reduction in LOS while maintaining and improving quality of care for CAP. This was true for SOS plus ICCM compared with either conventional treatment or SOS alone, the latter demonstrating the primary importance of the ICCM interventions. This supports recent findings made by Yealy et al¹⁰ that guidelines used in isolation may be suboptimal and that intensive strategies to facilitate implementation may be necessary. Application of this intervention has the potential to improve patient satisfaction and outcomes as well as organizational efficiency. The cost-effectiveness of this approach as well as its suitability for other medical diagnoses is uncertain and worthy of study.

Correspondence: Steven Fishbane, MD, 200 Old Country Rd, No. 135, Mineola, NY 11501 (sfishbane@winthrop.org).

Accepted for Publication: March 30, 2007.

Author Contributions:Study concept and design: Fishbane, Niederman, Brody, Gaffney, and Parker. Acquisition of data: Fishbane, Daly, Magin, Kawabata, de Corla-Souza, Choudhery, Gaffney, and Parker. Analysis and interpretation of data: Fishbane, Niederman, Daly, Brody, Pollack, and Parker. Drafting of the manuscript: Fishbane, Niederman, Kawabata, de Corla-Souza, Gaffney, and Pollack. Critical revision of the manuscript for important intellectual content: Fishbane, Niederman, Daly, Magin, Kawabata, Choudhery, Brody, and Parker. Statistical analysis: Fishbane, Kawabata, and Pollack. Obtained funding: Gaffney. Administrative, technical, and material support: Fishbane, Niederman, Daly, Kawabata, de Corla-Souza, Choudhery, Brody, Gaffney, and Parker. Study supervision: Fishbane and Magin.

Financial Disclosure: None reported.