Improving the Quality of Care for Patients With Pneumonia in Very Small Hospitals FREE

STUDIES^1- 4 HAVE shown that interventions aimed at improving processes of care for patients with pneumonia, including performance of blood cultures, timely administration of antibiotics, and selection of initial empirical antibiotics, have resulted in improvements in risk-adjusted mortality and length of stay and a reduction in total charges. In addition, the American Thoracic Society, the Infectious Diseases Society of America, the Canadian Thoracic Society, and the Canadian Infectious Diseases Society have published guidelines that provide recommendations for the initial evaluation and management of community-acquired pneumonia.^5- 10 Yet, various studies^11- 16 have demonstrated wide variability in the delivery of processes of care and average length of stay. This creates many opportunities to intervene in the care of patients hospitalized with pneumonia, which may, in fact, lead to improved quality of care manifested by improved patient outcomes.

This study was performed as part of the Health Care Quality Improvement Program sponsored by the Centers for Medicare & Medicaid Services. The goal of the Health Care Quality Improvement Program is to improve the processes of care and medical outcomes for Medicare beneficiaries through the performance of cooperative projects.^17- 18 Quality improvement organizations (QIOs) are external change agents charged with the task of motivating changes in physician and hospital performance of certain quality indicators to improve patient outcomes. Although there is some evidence that suggests that external feedback may be effective in stimulating quality improvement activities for pneumonia, there have been few controlled studies and virtually no studies limited to small hospitals.^{2- 4,19- 27}

We initiated 2 cooperative projects to evaluate the care of Medicare beneficiaries with pneumonia who were admitted to small hospitals. The objective of this study was to demonstrate that a project implemented by a QIO as an external change agent would lead to an improvement in care that could not be accounted for by secular trends in the management of pneumonia. To our knowledge, this is one of the first studies to examine quality improvement exclusively in small hospitals.

The study design was a cohort control study through 2 separate Health Care Quality Improvement Program projects. Thirty-six participating hospitals in Oklahoma underwent a retrospective baseline measurement of quality indicators. Hospitals were divided into 2 groups: those that underwent the intervention (n = 20) and those that did not (control group) (n = 16). Hospitals in the intervention group received feedback of information on their processes of care and subsequent remeasurement of performance of the quality indicators. The control hospitals served as a control during the baseline measurement and remeasurement period. The control hospitals were "crossed over," and underwent the same intervention and a second measurement of performance of the quality indicators. Informed consent and institutional review board approval were not required because the data were collected as a part of the Health Care Quality Improvement Program, not for research, and access to these data is given to the Medicare program by law. Feedback of performance information occurred immediately after baseline measurements in the intervention and control hospitals.

The hospitals were primarily rural community hospitals, with fewer than 200 licensed beds per hospital. In the intervention group, 4 (20%) of the hospitals were accredited by the Joint Commission on Accreditation of Healthcare Organizations; the median number of licensed beds was 40 (interquartile range, 33-58), and the median average daily census was 10 (interquartile range, 6-20). In the control group, 4 (25%) of the hospitals were accredited by the Joint Commission on Accreditation of Healthcare Organizations; the median number of licensed beds was 40 (interquartile range, 34-55), and the median average daily census was 8 (interquartile range, 7-15). All hospitals involved in the study were required to submit hospital medical records for data abstraction. All improvement activities after receiving feedback information regarding performance of the quality indicators were left to the discretion of the staff at each intervention hospital. Hospitals were chosen to represent the various geographic areas of the state and because they had not previously or were not currently involved in any QIO-directed quality improvement projects. Hospitals in the control group were chosen because of their demographic and geographic similarities to the intervention hospitals, and because they had not previously or were not currently involved in any QIO-directed quality improvement projects.

Eligible patients were Medicare beneficiaries who had a principal diagnosis of pneumonia (International Classification of Diseases, Ninth Revision, Clinical Modification codes 480.0-487.0) in the Medicare part A fee-for-service claims database. Exclusion criteria included transfer from another acute-care facility, infection with the human immunodeficiency virus, cytotoxic treatment within 1 month of hospital admission, history of organ transplantation, death within 4 hours of admission, and no documentation of pneumonia in the medical record.

Case selection occurred at 3 time points during the study. The first selection included a 50% random sample of eligible medical records that fit the previously described criteria from the intervention hospitals with a discharge date between October 1, 1992, and September 30, 1993. The intervention occurred at these hospitals between April 1 and June 30, 1995. The second selection included a 10% random sample of eligible medical records from the control hospitals with a discharge date between October 1, 1992, and September 30, 1993, and a 100% sample of eligible medical records from the intervention and control hospitals with a discharge date between June 1 and December 31, 1995. A similar intervention was then performed with the control hospitals between November 1, 1996, and March 31, 1997. A third selection included a 100% sample of eligible medical records of patients who were discharged from the control hospitals between October 1, 1997, and February 28, 1998.

Quality indicators for use in this study were developed by the staff of the Oklahoma Foundation for Medical Quality (Oklahoma QIO) and members of a study group, which included 4 family practice physicians, 2 internists (D.W.B. and one other), and 1 infectious disease specialist, and were believed to be in concordance with the American Thoracic Society, the Canadian Thoracic Society, and the Canadian Infectious Diseases Society guidelines for the treatment and diagnosis of community-acquired pneumonia and supported by the data available.^2,5- 6 The quality indicators that were selected for measurement were the proportion of patients admitted with pneumonia who: (1) had sputum cultures ordered within 4 hours of arrival, (2) had at least 1 blood culture obtained within 4 hours of arrival, and (3) received their first dose of empirical antimicrobial agents within 4 hours of arrival.^2,5- 6

Data were abstracted from the medical records using a structured data collection form. Documentation that the attending physician was treating the patient for pneumonia was required for further review to occur. Demographic data collected included age, sex, race, and skilled nursing facility residence. Information regarding the presence of at least 1 of a list of comorbid conditions, such as chronic obstructive pulmonary disease, chronic liver disease, chronic renal failure, diabetes mellitus, congestive heart failure, and hospitalization within the past year, was collected. The date and time of arrival, arrival location, date of discharge, and discharge disposition were abstracted. Severity indicators were recorded, including respiratory rate, blood pressure, pulse oximetry reading, PaO₂, PaCO₂, serum urea nitrogen level, evidence of bilobar or multilobar involvement, need for mechanical ventilation, need for vasopressors, and presence of oliguria or renal failure, as specified by the American Thoracic Society guidelines.⁶ Information regarding initial diagnostic testing was recorded, including the results of a sputum gram stain, a sputum culture, a blood culture, and serologic tests for atypical pathogens. The results of an initial chest radiograph and thoracentesis, if done, were noted. The timing of the first dose of antimicrobial agent and the choice of agent were recorded.

To ensure reliability of the information abstracted from the medical records, 100% of cases were rereviewed by a second abstractor regarding the data items related to timing of the first dose of antibiotic and performance and timing of the initial diagnostic studies.

The intervention performed by the QIO was external feedback that consisted of a face-to-face meeting with the medical staff, usually during regularly scheduled medical staff meetings. A personalized feedback packet of information was compiled for each hospital, which included tables characterizing the hospital's performance of the quality indicators compared with the other participating hospitals, a review of the literature, and a sample quality improvement plan. Quality improvement plans were requested from all the participants. The content of these was left to the discretion of the individual hospitals. Any hospitals that requested additional assistance in the form of training techniques of quality improvement, additional site visits, or teleconferences were accommodated. Reassessment of a hospital's performance of the quality indicators was performed approximately 6 months after the face-to-face feedback meeting.

After the study was completed, the intervention hospitals were administered a questionnaire regarding the types of quality improvement activities that were instituted (September 1996).

Patient characteristics were compared using the Fisher exact test or the χ² test for proportions, and 2-tailed t tests and an analysis of variance were used to compare means. P≤.05 was considered statistically significant.

Performance of the quality indicators was analyzed using χ² tests and generalized estimating equations. The generalized estimating equation models with a logit-link function were implemented using SAS statistical software (SAS PROC GENMOD, version 6.12; SAS Institute Inc, Cary, NC), to account for the repeated-measures design and clustering of responses within a hospital. For each outcome being considered, the basic model included direct effects for the intervention group and for the timing of the measurement (ie, preintervention vs postintervention). Because of the double-control nature of the study design, consisting of a concurrent parallel group of hospitals and preintervention measurements on the intervention group of hospitals, the effect of the intervention on the outcomes of interest is measured by the regression coefficient for an interaction term between group assignment and measurement time. An exchangeable covariance matrix structure was assumed. Regression coefficients and their associated SEs were converted to odds ratios (ORs) and 95% confidence intervals (CIs). Interaction terms between the timing of each measurement and the presence of a clinical pathway or a standing order were used to determine the effect of the intervention for each outcome in the intervention hospitals.

A total of 2154 medical records were reviewed during the entire study. Sixty-seven of these (3.1%) met one of the exclusion criteria (transfer from another facility, human immunodeficiency virus infection, cytotoxic treatment within the past month, organ transplantation, died within 4 hours of hospital admission, or no documentation of pneumonia), and were excluded from the analysis. The patient characteristics during the different measurement periods are summarized in Table 1 and Table 2. The mean age of the patients was 74 years, and 93.9% of the patients were white. Patients admitted to the control hospitals during the baseline period were significantly older than those admitted during the baseline 2 and postintervention periods. Most patients (65.4%) were admitted from home. Of the patients, 80.5% had at least 1 comorbid condition other than being 65 years or older. More patients admitted to the intervention hospitals during the postintervention period had comorbid conditions, compared with the baseline measurement period. Antibiotic therapy before hospital admission was documented in 29.1% of the patients.

More than half of the patients (66.3%) admitted during this study initially presented to the emergency department, and 94.4% were admitted to the general ward. Fewer patients admitted to control hospitals during the baseline period were admitted to a general ward compared with patients in the baseline 2 and postintervention periods. More than three quarters (79.1%) of the patients had at least 1 indicator of severe pneumonia. Of the patients, 25.2% had a pleural effusion on a chest radiograph. Patients admitted to control hospitals during the postintervention period were less likely to have a pleural effusion on a chest x-ray film compared with patients admitted during the baseline and baseline 2 periods.

Limited information about patient sputum and blood cultures was collected. Most sputum cultures had no growth or had normal flora. The most common organisms cultured from the sputum were Haemophilus influenzae and Streptococcus pneumoniae. These organisms were present in the sputum cultures of 22.8% of the patients at baseline and 22.1% of the patients in the remeasurement sample of the intervention hospitals. Similarly, these organisms were present in the sputum cultures of 17.4% of the patients at baseline and 20.3% of the patients in the remeasurement sample of the control hospitals. There were no significant differences in the frequency of culture of any other organisms in the sputum across the various periods of the study. Because there were few blood cultures that were positive for organisms (21 [5.1%] of 412) in our baseline assessment, this information was not captured in subsequent medical record reviews. Of the 21 blood cultures that were positive for organisms at baseline, Staphylococcus epidermidis grew in 9, S pneumoniae in 4, and Bacteroides species in 2. There was 1 blood culture each positive for Pseudomonas aeruginosa, Proteus mirabilis, Nocardia species, and Staphylococcus aureus. Two cultures had unknown organisms. Viral pneumonia was diagnosed in fewer than 1% of the cases based on principal diagnoses across all periods studied.

We profiled use of antibiotics for the patients. There was little difference between the intervention and control hospitals during the 2 comparative periods in the prescription of antibiotics during the first 24 hours of the hospital stay. The use of either a second- or a third-generation cephalosporin increased from 50.8% to 68.6% of the cases from the intervention hospitals and from 50.9% to 67.3% in the control hospitals during the same period (October 1, 1992–September 30, 1993). The most striking difference in prescribing patterns was a reduction in the use of the first-generation cephalosporins in the intervention hospitals (14.8% down to 5.5%) and an increase in the prescription of macrolide antibiotics (5.3% to 18.4%) after feedback. There was no change in the use of first-generation cephalosporins (12.3% to 12.4%) or macrolide antibiotics (0% to 3.5%) in the control hospitals during the same period.

Performance of quality measures in the initial management of pneumonia is shown in Table 3. There were statistically significant improvements in the performance of all quality indicators measured after the intervention, notably the percentage of patients who received antibiotics in the emergency department, the percentage of patients who received antibiotics within 4 hours of arrival, and the percentage of patients who had sputum cultures ordered and blood cultures obtained within 4 hours of arrival. The intervention hospitals were more likely to demonstrate a statistically significant improvement in their performance of all the quality indicators compared with the control hospitals.

Patient outcomes improved after the intervention. The unadjusted mortality for patients admitted with pneumonia improved from 12.2% (95% CI, 9.9%-14.7%) to 8.4% (95% CI, 5.8%-11.7%) in the intervention hospitals (P = .05). In the control hospitals, the unadjusted mortality decreased from 12% (95% CI, 6.6%-19.7%) to 9.8% (95% CI, 7.2%-12.9%) (P = .66). The difference in mortality between the intervention and the control hospitals was not statistically significant (P = .39). The length of stay decreased from a mean of 7.79 to 6.31 days after the intervention (P<.001). However, in that same period, the length of stay in the control hospitals decreased from 7.69 to 6.51 days (P = .006). The differences in the length of stay between the intervention and control hospitals were not statistically significant (P = .47).

Improvements in the performance of quality measures in the intervention compared with the control hospitals are as follows: antibiotic administration within 4 hours (OR, 2.17; 95% CI, 0.94-4.99), antibiotics given in the emergency department (OR, 10.72; 95% CI, 3.56-32.30), sputum culture ordered within 4 hours (OR, 1.54; 95% CI, 0.90-2.64), and blood culture obtained within 4 hours (OR, 2.48; 95% CI, 1.17-5.25).

The results of the subsequent intervention in the control hospitals are shown in Table 4. There were statistically significant improvements after the intervention in the performance of all of the quality indicators, except ordering of sputum cultures within 4 hours of arrival to the hospital. The unadjusted mortality decreased from 9.8% to 5.6% after the intervention. The length of stay decreased from 6.51 to 5.69 days (P<.001) after the intervention.

In the by-hospital analysis, patients admitted to the control hospitals after the intervention were significantly more likely to have antibiotics administered within 4 hours of admission (OR, 1.81; 95% CI, 1.21-2.73), antibiotics administered in the emergency department (OR, 5.45; 95% CI, 2.00-14.80), and blood cultures collected (OR, 1.88; 95% CI, 1.07-3.30). The difference for sputum cultures ordered within 4 hours of arrival was not significant (OR, 1.37; 95% CI, 0.94-2.00). The unadjusted mortality in the control hospitals was significantly less after the intervention (P = .04).

The effects of the interventions conducted at the individual hospital level on the performance of each quality indicator are shown in Table 5. The institution of clinical pathways was associated with an improvement in the timing of antibiotic administration in the by-patient analysis. The institution of standing orders was associated with an improvement in the performance of blood cultures in the by-patient and the by-hospital analyses.

We showed that interventions made by the QIO at the hospital level were associated with changes in the performance of processes of care for the treatment of pneumonia, which can potentially lead to improved patient outcomes. We found statistically significant improvements in the intervention hospitals compared with the control hospitals in the performance of sputum cultures and blood cultures, the administration of antibiotics within 4 hours of hospital admission, and the provision of the first dose of antibiotics in the emergency department when the data were analyzed using the patient as the unit of analysis. The lack of change in the control group of hospitals supports the theory that the improvements seen in the performance of these quality measures in the intervention hospitals were, in fact, due to the intervention rather than secular trends in the care of pneumonia. Further evidence to support this theory is the fact that the control hospitals similarly demonstrated an improvement in their performance of quality indicators after the same intervention. Simply measuring performance of quality indicators without provision of feedback did not stimulate changes in care. Moreover, this may be the first study showing that improvements in care can be associated with quality improvement activities in very small hospitals.

Since our study, Meehan et al¹ have shown a decrease in mortality in the same Medicare population with performance of similar quality indicators (ie, the administration of antibiotics within 8 hours of hospital arrival and blood culture collection within 24 hours of hospital arrival). Dean et al⁴ have shown similar decreases in mortality in Medicare-insured patients with community-acquired pneumonia after the implementation of a pneumonia guideline in small rural and large urban hospitals, compared with patients hospitalized during the same period in hospitals without such a guideline. Recent data from the Medicare National Pneumonia Project²⁸ have demonstrated that the administration of antibiotics within 4 hours of hospital arrival is associated with improved in-hospital and 30-day mortality.

Other similar hospital-level intervention studies^{19- 20,22,26} have been performed and have shown improvements in the process-based quality indicators as well. Two of these studies^22,26 looked at mortality and length of stay, and demonstrated decreases in these outcome measures. However, none of these studies^{19- 20,22,26} had a control group of hospitals that allowed a comparison to differentiate between improvements due to secular trends in the management of pneumonia and improvements due to the intervention, and none were primarily performed in small hospitals.

The QIOs are charged by the Centers for Medicare & Medicaid Services with promoting improvement in health care quality. This study shows that this is a realistic goal and that change can occur in small hospitals. Marciniak et al²⁹ have also demonstrated that meaningful improvements in the care of patients with acute myocardial infarction can be made when hospitals are stimulated by performance feedback from the QIOs from 4 selected states compared with control states in the Cooperative Cardiovascular Project.

Hospitals were selected for participation based on the fact that they were not involved in any QIO-directed quality improvement projects and for their demographic qualities. This eliminates some selection bias, because hospitals that volunteer for quality improvement projects may be more likely to show improvement rather than those that do not.

There are several limitations of this study. Although it seems that there was a trend toward improvement in the performance of all of the quality indicators in the by-hospital analysis, only improvements in the administration of antibiotics in the emergency department and blood culture collection were statistically significant. This may have been because of the limited power and sample size. Because the intervention in the study was implemented at the hospital level, and expected to generally influence the "culture" surrounding the treatment of patients with pneumonia, it was important to allow for the effect of clustering of cases within each hospital. This correlation or similarity of treatment among patients within an institution is measured by the intraclass correlation coefficient. Unfortunately, no data were available to allow an estimation of the intraclass correlation coefficient in this setting at baseline for the end points being considered. This made it impossible to perform a power calculation ahead of time for the by-hospital analysis. Because of this, we elected to present the results for the by-hospital analysis using ORs and their associated 95% CIs. For example, the OR for the increase in the percentage of patients receiving antibiotics within 4 hours of arrival is 2.17 (95% CI, 0.94-4.99). While it is true that this CI includes 1 and, thus, we have not unequivocally demonstrated an intervention effect on this outcome, the CI is centered around 2.17 and barely includes 1, consistent with an intervention effect. A similar comment can be made regarding the end point of having a sputum culture ordered within 4 hours, although the apparent effect size was of a lower magnitude.

In addition, changes in antibiotic prescribing practices after feedback may have had an impact on patient outcomes. However, our projects were focused on improving processes of care, and were not designed to detect differences in measures of outcomes, such as mortality and length of stay, between intervention and control hospitals. Also, differences in feedback and other assistance that the intervention hospitals may have received from the QIO were not controlled for, and may have affected the amount of change in process measures seen. Only the medical records of Medicare-insured patients were included in the review. This may limit generalizability to parts of the general population (ie, younger patients and those in the managed care setting). Also, this study was performed in small hospitals, so the results may not be relevant to larger institutions. Another limitation is that because the postintervention data abstractions represented a cross section in time, we were not able to demonstrate a sustained benefit over time. Finally, hospitals were not randomized to the intervention or control groups. This study grew out of quality improvement work that we were doing with small rural hospitals. The Centers for Medicare & Medicaid Services policy for QIOs prevented us from randomizing hospitals to a control group. It was only after we had initiated our pneumonia project with the first group of 20 hospitals (intervention group) that we decided to select a group of demographically similar hospitals to participate in a second project.

In conclusion, we have demonstrated that the improvements in processes of care achieved by the intervention hospitals resulted from activities initiated because of participation in a QIO-directed project. This is one of the first studies to demonstrate the effectiveness of quality improvement activities in very small hospitals.

Corresponding author and reprints: Dale W. Bratzler, DO, MPH, Oklahoma Foundation for Medical Quality, 14000 Quail Springs Pkwy, Suite 400, Oklahoma City, OK 73134 (e-mail: okpro.dbratzler@sdps.org).

Accepted for publication June 13, 2002.

The analyses on which this article is based were performed under contract 500-99-P619, entitled "Utilization and Quality Control Peer Review Organization for the State of Oklahoma," sponsored by the Centers for Medicare & Medicaid Services (CMS), Department of Health and Human Services. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the US government. The authors assume full responsibility for the accuracy and completeness of the ideas presented. This article is a direct result of the Health Care Quality Improvement Program initiated by the CMS, which has encouraged identification of quality improvement projects derived from analysis of patterns of care and, therefore, required no special funding on the part of this contractor. Ideas and contributions to the authors concerning experience in engaging with issues presented are welcomed.