Processes of Care, Illness Severity, and Outcomes in the Management of Community-Acquired Pneumonia at Academic Hospitals FREE

PNEUMONIA IS the sixth leading cause of death in the United States,¹ accounting for 88 383 deaths in 1997.² It was the primary discharge diagnosis for more than 1.3 million hospital admissions in 1997, and was listed among discharge diagnoses for more than 2 million hospitalizations.³ Average annual costs for pneumonia treatment from 1988 to 1994 were $8.4 billion, $7.5 billion of which was for inpatient care.⁴

Pneumonia-related mortality, expenditures, and variation in hospital care have driven efforts to identify markers of quality inpatient pneumonia management.^5- 7 Several process-of-care quality indicators have been proposed by expert consensus, including prompt antibiotic administration and performance of blood cultures, blood culture performance before antibiotic administration, and timely oxygenation measurement.⁸ Although these process markers have strong intuitive appeal, few studies have examined their independent relationship to clinical outcomes. One retrospective study⁹ of 14 000 Medicare-insured patients found that antibiotic administration within 8 hours and blood cultures within 24 hours of hospital arrival were associated with a lower 30-day all-cause mortality. These findings have not been replicated.

Initiatives to improve the quality of treatment for pneumonia have focused on these process-of-care markers.^10- 12 In particular, the Health Care Financing Administration has begun a program to increase the proportion of patients who receive antibiotics within 8 hours of hospital arrival, and who have blood cultures performed before antibiotics are given.¹³ Achievement of such process markers may be used to gauge the quality of pneumonia care hospitals provide. Research is needed to assess the link between process-of-care markers and important clinical outcomes.

We studied the relationship between achieving pneumonia process-of-care quality markers and the clinical outcomes of inpatient mortality, the proportion of patients remaining unstable at 48 hours after hospital arrival, and length of hospital stay. We focused on a cohort of patients with community-acquired pneumonia (CAP) admitted to a network of 38 academic hospitals.

Patients hospitalized with CAP were retrospectively identified from 38 US academic hospitals that participated in a University HealthSystem Consortium–sponsored pneumonia benchmarking project. Each hospital enrolled up to 40 consecutive patients discharged between December 1, 1997, and February 28, 1998, with a primary International Classification of Diseases, Ninth Revision, Clinical Modification diagnosis of pneumonia (code 003.22, 21.2, 39.1, 052.1, 055.1, 073.0, 112.0, 114.0, 115.05, 115.15, 115.95, 130.4, 510.0, 510.9, 511.1, 480-480.2, 480.8, 480.9, 481, 482-482.4, 482.8-483, 484.1, 484.3, 484.5-484.8, 485, or 486) or a secondary International Classification of Diseases, Ninth Revision, Clinical Modification pneumonia diagnosis, where the primary diagnosis was respiratory in nature, septicemia, or dehydration (code 038.0-038.9, 276.5, 490, 512.0-512.9, 518.81-518.82, or 786.0-786.9). A medical record review was performed at each site by University HealthSystem Consortium–trained personnel using a standard abstraction form in either a paper or an electronic format.

Patients were excluded if they were younger than 18 years, the initial chest radiograph was performed more than 24 hours before or 48 hours following hospital arrival, no infiltrate was found on the chest x-ray film, the antibiotic administration time was not identified, or antibiotics were not administered within 48 hours of arrival or were known to have been given before hospital arrival (medical record review of hospital records may underestimate prehospitalization antibiotic use). Other exclusion criteria included discharge from an acute-care hospital within 10 days of admission; transfer from another acute-care hospital; active immunosuppressive therapy, including prednisone, 15 mg/d or greater; known human immunodeficiency virus seropositivity; active chemotherapy; and a diagnosis of cystic fibrosis or tuberculosis.

We used 4 process markers of quality pneumonia care that were previously developed by expert consensus.⁹ These markers were as follows: (1) time from arrival at the hospital to antibiotic administration (a cut point of 8 hours was used); (2) a blood culture within 24 hours of arrival; (3) performing blood cultures before antibiotic administration; and (4) oxygenation measurement within 24 hours of arrival. The arrival time for patients admitted via the emergency department was taken from the initial nurse triage note. For patients admitted directly to a medical floor, the admission time was used. Antibiotic administration time was taken from the nursing medication flow sheets, and test performance time came from laboratory reports.

Pneumonia severity on hospital admission was estimated using the Pneumonia Severity Index (PSI), a validated predictor of 30-day all-cause mortality for patients with CAP.¹⁴ The PSI incorporates demographic information (age, sex, and nursing home status), the presence of 6 comorbid conditions, 5 features of the initial physical examination, and 7 laboratory measurements and chest x-ray findings. The results of laboratory tests that are not performed are considered to be normal. Patients were assigned to risk classes 1 (low) to 5 (high) based on a score computed by adding points for each predictor variable.¹⁴ Our outcome variables were inpatient death, failure to reach clinical stability within 48 hours of admission, and length of hospital stay. Clinical stability was achieved when the patient's heart rate, oxygenation status, respiratory rate, and temperature had met certain preset criteria within the same day; the patient was eating; and the patient's mental status was stable.⁶

Outcomes were expressed as dichotomous variables: inpatient death and clinical instability were coded as occurred or not, and length of hospital stay was coded as greater than the overall median of 4 days or not. For analyses of length of stay, 66 patients who died in the hospital, 12 who left against medical advice, and 11 who were transferred to another acute-care facility were excluded. Pneumonia severity on admission was categorized according to the 5 PSI classes (I-V). Patients who had blood cultures performed or oxygenation measured within 24 hours of arrival were compared with all others. This latter group included patients who achieved the process marker (blood culture performance or oxygenation measurement) beyond 24 hours and those who did not achieve the process marker at all. Patients who had blood cultures before antibiotics were given were compared with those who had blood cultures after antibiotics were given. The median length of hospital stay, the proportion of inpatient deaths, and the proportion of patients remaining clinically unstable at 48 hours after arrival were calculated across all patients and by hospital.

We calculated the median time to process marker achievement, the proportion of all patients achieving each process-of-care marker, and the range of process marker achievement across hospitals. We evaluated process marker achievement according to severity at admission as estimated by the PSI, and compared the mean time to marker achievement in class I vs class V patients. The association between process marker achievement and pneumonia severity on admission was tested using logistic regression.

Our primary analysis examined the univariate and multivariate association between process marker achievement and clinical outcomes. Multiple regression models controlled for the presence of all other process markers and pneumonia severity using the PSI. Because the 2 blood culture process-of-care markers were expected to be highly colinear, each was included in a separate multivariate model. Generalized estimating equations were used to control for potential correlations between patients treated at the same hospital.¹⁵

We also categorized patients into high- and low-risk groups according to PSI class (classes I-III vs classes IV-V) to assess whether the impact of process marker achievement on outcome differed according to pneumonia severity on admission. Using logistic regression modeling, we calculated the relative odds of outcomes in both severity strata, and included terms reflecting the interaction between process marker achievement and risk status to test for effect modification by pneumonia severity.

For the dichotomous process markers "blood culture within 24 hours of arrival" and "oxygenation measurement within 24 hours," the respective comparison group contained 2 subgroups: those undergoing the diagnostic test more than 24 hours after arrival and those who did not undergo the test at all during the hospitalization. Separate logistic regression models were constructed using the different comparison groups. We also compared the number of comorbidities (including diabetes, ischemic heart disease, and chronic lung disease), age, PSI class, and outcomes according to whether the process marker was performed within 24 hours of hospital admission, was performed after 24 hours, or was not performed. We used 2-tailed t tests or the Wilcoxon rank sum test and adjusted for multiple group comparisons with the Bonferroni correction. All analyses were performed using SAS statistical software, version 7.0 (SAS Institute Inc, Cary, NC). The pneumonia database was created using Microsoft Access 95, version 4 (Microsoft Corp, Redmond, Wash).

The 38 participating hospitals enrolled 1457 patients with CAP. Of these patients, 1062 (73%) met eligibility criteria for the present study. The primary reasons for exclusion were lack of evidence of pneumonia on the admission chest x-ray film (n = 224) and transfer from another acute-care hospital (n = 111). The median number of patients from each hospital included in this study was 28 (range, 16-38; interquartile range [IQR], 24-32).

The median patient age was 64 years (range, 18-98 years) (Table 1). Half the patients were women, and approximately 40% were nonwhite. Fifty-nine percent had Medicare and/or Medicaid insurance. Most patients were admitted through the emergency department into the care of an internist. Forty-nine percent had at least 1 of the listed chronic comorbid conditions. On admission, 52% of patients were high risk (PSI class IV or V), and 10% had "do not resuscitate" orders.

Six percent of the patients died in the hospital, and approximately 70% were not clinically stable by 48 hours after arrival (Table 2). The median length of hospital stay was 4 days (IQR, 3-7 days), and the range was from less than 1 day to 67 days (mean, 5.9 days). These outcomes varied widely across hospitals: inpatient mortality ranged from 0% to 15%, not achieving clinical stability by 48 hours ranged from 0% to 77%, and the median length of hospital stay ranged from 3 to 8 days. All outcomes were significantly associated with increasing pneumonia severity on admission (Table 2).

All patients received antibiotics. Eighty-six percent had blood cultures performed and 97% underwent oxygenation measurement at some point during their hospitalization. The median times to receiving antibiotics, performing blood cultures, and measuring oxygenation were 4.2 hours (IQR, 2.4-7.8 hours), 2.6 hours (IQR, 1.1-5.8 hours), and 0.4 hours (IQR, 0.1-1.4 hours), respectively.

Process marker achievement ranged from 72.3% for a blood culture before antibiotics were given to 94.5% for oxygenation measurement within 24 hours of arrival (Table 3). The proportion of patients achieving process markers varied greatly across hospitals. For example, the proportion receiving antibiotics within 8 hours of arrival ranged from 53.8% to 100.0% across the 38 hospitals.

The percentage of patients having blood cultures performed within 24 hours or antibiotics administered within 8 hours of arrival increased with pneumonia severity on admission (Table 4). The mean time to antibiotic administration and first blood culture was also significantly shorter in PSI class V patients than in PSI class I patients. Patients in PSI class V received antibiotics an average of 2 hours earlier than PSI class I patients (P = .008), and had blood cultures performed an average of 3.1 hours earlier (P = .02). However, oxygenation assessment and performing blood cultures before antibiotic administration were not associated with pneumonia severity on admission. For oxygenation assessment, the process marker was achieved in nearly all patients.

The crude and adjusted relationships between achieving the 4 process-of-care quality markers and the 3 clinical outcomes are presented in Table 5. Antibiotic administration within 8 hours of arrival and performing a blood culture before antibiotic administration were associated with an excess adjusted risk of inpatient death (P = .18 and .40, respectively), while performing a blood culture within 24 hours of arrival carried a relative risk of 0.86 (P = .74). None of these relative risks was statistically significant, reflecting the small number of patients who died (n = 66). Clinical instability at 48 hours after arrival was significantly related to having had a blood culture performed within 24 hours (P = .008). The early administration of antibiotics and the performance of a blood culture before antibiotic administration were associated with a decreased length of hospital stay, but these associations were not statistically significant (P = .47 and .89, respectively). These relationships did not differ significantly in high vs low-risk patients (ie, P >.11 for all interaction terms).

The reference group for a blood culture and an oxygenation measurement within 24 hours of arrival included 2 subgroups: those who underwent testing after 24 hours and those who did not undergo testing at all. The baseline characteristics of patients having a blood culture performed within 24 hours, having a blood culture performed after 24 hours, or not having a blood culture performed at all are shown in Table 6. There was no difference in median age or mean number of comorbidities among the 3 groups. However, those not having a blood culture performed had better clinical outcomes in general than either of the other 2 groups. When patients who did not undergo a blood culture were used as the reference group (data not shown), undergoing a blood culture within 24 hours was associated with a higher adjusted relative odds of poor outcome (1.19-1.66 among the 3 outcome measures) than when the original composite reference group was used. Similar findings were seen for oxygenation measurement within 24 hours of arrival.

Rapid antibiotic therapy, blood culturing to identify the causative organism, and surveillance for compromised oxygenation are accepted hallmarks of high-quality care for patients hospitalized with CAP.⁸ However, the empirical evidence linking these processes to improved outcomes is limited.^9,16- 17 We examined the relationship among proposed quality-of-care markers, the severity of illness on admission, and various outcomes for patients hospitalized with CAP at academic centers.

We found no consistent relationship between process marker achievement and improvement in the clinical outcomes of inpatient death, clinical stability within 48 hours of arrival, and length of hospital stay. We did observe a strong relationship between pneumonia severity on admission and achieving process markers. Specifically, earlier antibiotic administration and blood culture performance were significantly more likely with higher pneumonia severity as measured by the PSI. Physicians appeared to respond to severely ill patients with more aggressive management.

Previous observational studies have reported a relationship between clinical outcomes and pneumonia process-of-care markers. Kahn et al¹⁶ reported a lower 30-day mortality in Medicare-insured patients with the highest scores on a composite process marker scale that included antibiotic administration within 4 hours of arrival and early oxygenation measurement. Similarly, data from a single Pennsylvania hospital showed that, in combination with other interventions, antibiotic administration within 4 hours of admission, initial blood cultures, and oxygenation measurement were associated with reduced mortality, reduced length of stay, and lower hospital expenses.¹⁷ The largest study⁹ of this question retrospectively reviewed 14 000 Medicare-insured patients with a discharge diagnosis of pneumonia. The researchers found that antibiotic administration within 8 hours and blood culture performance within 24 hours of hospital arrival were associated with a lower relative risk of 30-day all-cause mortality (for antibiotic administration: odds ratio, 0.85 [P<.001]; for blood culture performance: odds ratio, 0.90 [P = .07]).

In contrast to the Kahn et al and Medicare studies, we considered the more proximate end point of inpatient death. The difference in results between these studies and ours may also relate to differences in patient and/or hospital characteristics. Our study had many low-risk patients (29% were in PSI classes I and II) and, unlike the Medicare study, did not exclude PSI class I patients. Furthermore, we observed that low-risk patients were less likely to achieve process markers, yet still did well. As a result, cohorts enriched with lower-severity patients may demonstrate an inverse relationship between process marker achievement and favorable outcomes. In addition, our patients were treated at academic hospitals, whereas the Medicare study sampled teaching and nonteaching hospitals. If the quality of pneumonia care is higher at teaching vs community hospitals, as some suggest,^18- 19 any beneficial effect of achieving process markers may be blunted in academic settings. Higher rates of achieving process markers in our study compared with the national Medicare cohort support this assertion.⁹

Issues of confounding by patient and hospital characteristics, such as illness severity, academic status, and hospital culture, underscore how difficult it is to validly assess the impact of quality care markers on outcome. In our data, it appeared that the PSI did not fully account for the link between illness severity and aggressive management ("confounding by indication"). Similar confounding by indication in the Medicare study⁹ may explain why the relative odds of death in patients who received antibiotics within 1 hour of hospital arrival was 1.2 (P>.05) compared with those receiving antibiotics later, even after adjusting for the PSI.

Our study has several strengths. Like the national Medicare study, we examined the relationship between process marker achievement and mortality while controlling for pneumonia severity on admission and the performance of other process markers. However, we also included more proximate and perhaps more pneumonia-specific outcome measures, including length of hospital stay and clinical stability within 48 hours of arrival. Our patient population was heterogeneous for age, severity of illness, ethnicity, and insurance status. The cohort was assembled recently and reflected relatively current management approaches. Finally, all patients were treated at academic institutions, which may reduce institutional differences such as the presence of house staff.

Like prior studies of this question, limitations of our study include its retrospective study design based on discharge diagnoses and medical record review. Data on pathogens and appropriateness of antibiotic choice were not included in this analysis. However, a pathogen is typically identified in only 30% to 50% of the patients hospitalized with pneumonia, and analysis of antibiotic appropriateness can be complex in this subgroup.²⁰ Furthermore, given the relatively high rate of process marker achievement among our patients, the low rate of inpatient death, and our moderate sample size, there was limited power to detect small but potentially important differences in the risk of outcome attributable to process marker achievement. For our more robust outcomes, prolonged length of hospital stay and clinical instability, there was at least 80% power to detect an odds ratio ranging from 1.48 to 1.74 for antibiotic administration within 8 hours of arrival, a blood culture within 24 hours of arrival, and a blood culture before antibiotic administration. The equivalent values expressed as maximum detectable odds ratios below 1 range from 0.57 to 0.68.

In summary, we observed no consistent relationship between process marker achievement and improved clinical outcomes in patients hospitalized for CAP. Efforts to promote early appropriate antibiotic therapy and diagnostic tests or to decrease inappropriate variability in the care of patients hospitalized with CAP make good clinical sense. But our results emphasize how difficult it is to demonstrate a beneficial impact of process-of-care marker achievement on pneumonia-related outcomes using observational study designs. Prospective randomized studies are needed to elucidate the relation between achieving process markers and pneumonia quality of care. Crafting such studies will be complicated by the relatively high degree of process marker achievement observed in our study.

Sound clinical judgment clearly supports timely blood cultures, oxygenation measurement, and antibiotic administration in the treatment of patients hospitalized with CAP. However, without randomized trial evidence or more powerful risk adjustment techniques, it would be premature for administrative or reimbursement agencies to use these process markers as validated measures of high-quality care.

Accepted for publication February 22, 2001.

Presented as an abstract at the Society for General Internal Medicine Annual Meeting, Boston, Mass, May 5, 2000.

We thank Celeste Wasz, BSc, for her insights about the data collection process, and the clinician coordinating committee members and site participants for their hard work in executing this project.

Corresponding author and reprints: Julien Dedier, MD, MPH, Section of General Internal Medicine Research Unit, Boston Medical Center, 91 E Concord St, Suite 200, Boston, MA 02118-2393 (e-mail: jdedier@bu.edu).