Influenza Vaccine Effectiveness in Patients on Hemodialysis:  An Analysis of a Natural Experiment FREE

Influenza causes substantial morbidity and mortality in the general population, with approximately 39 000 people dying each year.¹ Patients with end-stage renal disease (ESRD) may be at higher risk of illness and death from influenza relative to healthy adults. For more than 40 years, trivalent inactivated influenza vaccine has been recommended by the Advisory Committee on Immunization Practices for patients with ESRD.² Seasonal influenza vaccination has become routine practice at most dialysis clinics during the past 2 decades. Although patients undergoing hemodialysis have lower response rates to influenza vaccine compared with healthy adults, immunogenicity studies show that 50% to 93% of patients on dialysis develop antibody titers after vaccination.^3- 4 However, it is currently unclear how much morbidity and mortality is prevented by the influenza vaccine in patients with ESRD.⁵ To date, 1 study among patients on hemodialysis has estimated a 12% to 14% vaccine effectiveness (VE) for influenza/pneumonia hospitalizations and a 25% VE for all-cause mortality.⁶ Recent studies in the elderly population who are not undergoing dialysis have suggested that large VE effects (≤50% reduction of all-cause mortality in some studies^7- 9) obtained from standard epidemiologic studies may be the result of confounding by unmeasured prognostic variables, and the true effect may be small to negligible.^10- 14

One potential way to avoid confounding by patient-level differences is to exploit the natural experiment that is caused by strong year-to-year variation in the match of the vaccine to the circulating strain. The influenza virus that predominates in a season can undergo antigenic drift after the vaccine strain has been chosen, resulting in a vaccine that provides reduced immunity. In seasons with a well-matched vaccine (hereinafter referred to as matched years), vaccination is expected to be effective in preventing influenza-related outcomes, whereas in mismatched seasons (hereinafter referred to as mismatched or placebo years), vaccination is expected to have a minimal effect. It has been documented that the 1997-1998 influenza vaccine strain (A/Wuhan/359/95) did not match the circulating strain (A/Sydney/5/97),¹⁵ and outbreak investigations suggested that the vaccine provided limited protection.¹⁶ A randomized controlled trial confirmed that the vaccine did not prevent clinically relevant outcomes during this season among healthy adults younger than 65 years; vaccinated patients had more influenza-like illnesses (ILIs) and upper respiratory tract infections than patients receiving placebo.¹⁷ In 3 of the following 4 years, the same strain of virus circulated in the community, and the vaccine was well matched.^18- 20

We evaluated the difference in VE between years in which the vaccine was well matched and the 1997-1998 placebo year, in which the vaccine was poorly matched and was shown to have provided little benefit. By studying this natural experiment, we sought to reduce confounding bias due to frailty and unmeasured health behaviors to obtain a more accurate measure of VE.

We used Medicare claims from the US Renal Data System, a population-based national system that collects information on all patients with ESRD in the United States. Claims include information on physician services; codes from the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM), assigned to hospitalizations and outpatient care; and information on dialysis care, medication, and immunization use. This information is captured for all patients with Medicare as a primary payer (ie, no health maintenance organization insurance as a primary payer or Medicare as a secondary payer).

Our cohorts consisted of all adult patients with ESRD who had Medicare as a primary payer and underwent continuous hemodialysis use. Each yearly cohort consisted of patients who had initiated dialysis before October 1 of the preceding year. An 8-month window from January 1 through August 31 of each year was used to identify insurance status and comorbidities. A 3-month window from June 1 through August 31 was used to identify continuous dialysis status. For example, the cohort identified for the 1997-1998 season would have initiated dialysis before October 1, 1996, and would have been receiving continuous hemodialysis from June 1 through August 31, 1997, and had Medicare as a primary payer from January 1 through August 31, 1997. Vaccination and outcome status was assessed beginning on September 1 of each year. Cohort members were followed up each year until they experienced 1 of the 3 study outcomes (discussed in the “Outcomes” subsection), death for nonmortality outcomes, transplant, loss to follow-up, or administrative censoring on August 31 of the following year (eg, August 31, 1998, for the 1997 influenza season).

We chose to analyze specific years based on the characteristics of each influenza season: years with similar influenza severity and in temporal proximity to the mismatched season. We used years before paying out of pocket at grocery stores or pharmacies became common to limit exposure misclassification. Cohorts were created for the following influenza seasons: 1997, 1998, 1999, and 2001. Seasons were defined by the year in which vaccination began for that influenza season (eg, the 1997-1998 season was defined as 1997). These 4 seasons were used because of their similar severity and strain of influenza but with various levels of vaccine match.^15,18- 20 We excluded the 2000 season to limit differences between seasons due to influenza severity; the predominant strain in the community in 2000 was a less severe strain (A/H1N1).²¹ We estimated the start of each influenza season by using national influenza surveillance data from the Centers for Disease Control and Prevention. We defined the start of the season as the midpoint of the first week during which more than 10% of the isolates were positive for influenza. A sensitivity analysis examined the effect of a less restrictive definition, with the start of the season defined as the week with 5% of isolates positive for influenza.

Medicare part A hospital/outpatient files and part B physician/supplier files were searched for Current Procedural Terminology codes 90724, 90656, and 90658-60 and Health Care Financing Administration Common Procedure Coding System codes G0008 and G8482. Because our study population is often hospitalized, we also searched for ICD-9-CM procedure code 99.52.

We examined the following 3 outcomes: all-cause mortality, hospitalization due to influenza or pneumonia, or ILI. Mortality was identified by the Centers for Medicare and Medicaid Services form 2746, the ESRD Death Notification Form. We searched the principal discharge diagnoses in the Medicare part A inpatient hospitalization files for the first instance of ICD-9-CM codes 480.xx through 487.xx to identify influenza/pneumonia hospitalizations. Inpatient and outpatient codes were searched to identify the first instance of ILI as classified by Lindsay et al²² (eTable 1). In a sensitivity analysis, we limited ILI to more specific codes by removing ICD-9-CM codes 465, 466, and 490.

All confounders were identified using the existing evidence base, including the investigative team's knowledge and the published literature. We used the Centers for Medicare and Medicaid Services form 2827, the Medical Evidence Form, to ascertain age, race, sex, first service date with ESRD, and cause of kidney failure. Parts A and B claims were searched during the 8-month window from January 1 to August 31 for oxygen use and the following comorbidities as identified by Liu et al²³: atherosclerotic heart disease, congestive heart failure, cerebrovascular accident/transient ischemic attack, peripheral vascular disease, other cardiac disease, chronic obstructive pulmonary disease, gastrointestinal tract bleeding, liver disease, dysrhythmia, cancer, and diabetes mellitus. Comorbidities were modeled as dichotomous variables in the final models. Adherence to dialysis was calculated by summing the number of dialysis sessions during the 8-month baseline period; patients were considered adherent if they had 95 sessions or more. Patients with no dialysis sessions during the baseline period were dropped from the analysis. We also included the number of hospital days during the baseline period. Use of mobility aids was ascertained by searching parts A and B claims for Health Care Financing Administration Common Procedure Coding System equipment codes for wheelchairs, walkers, canes, and bathroom assistance equipment during the baseline period (eTable 1).

We used Cox proportional hazards models to estimate hazard ratios (HRs) comparing vaccinated with unvaccinated cohorts within each year.²⁴ Vaccination was modeled as a time-varying covariate, with all cohort members entering the analysis on September 1 as unvaccinated. Once vaccinated, patients remained in the vaccinated category until the end of that influenza year (August 31). To quantify bias in these estimates, we ran the same models during the preinfluenza period (September 1 through the day before the influenza season started). When limited to the period before the start of influenza season, when VE should be biologically negligible, we would expect the HR estimate to be close to 1.00 if no confounding were present. This method identifies whether the conventional analysis remains biased even after adjustment.

To estimate effects between seasons, we ran proportional hazards models with an interaction between vaccination status and year, with vaccination status treated as described in the preceding paragraph. Kaplan-Meier survival curves are reported for the comparison of different years among vaccinated patients. We report the antilog of the β coefficient for the interaction term, which represents the ratio of 2 HRs comparing the vaccinated cohort in a matched year with the vaccinated cohort in the mismatched year divided by the comparison of the unvaccinated cohort in a matched year with the unvaccinated cohort in the mismatched year. We calculated VE as 1 − effect measure. Because patients could be in multiple cohorts, robust variance was used initially to account for the possibility of having multiple events in the analysis of events other than mortality. Using robust variance did not change the variance estimate; thus, we report standard variances.

Adjusted models in all analyses controlled for age, race, sex, cause of ESRD, length of time with ESRD (vintage), adherence to dialysis, number of mobility aids as a proxy for functional status, oxygen use, hospital days, ESRD network, and comorbidities. The proportional hazards assumption was checked graphically. To examine the effect of nonproportional hazards, we limited our final model to run only through the end of the influenza season, which is approximately the time when the curves crossed. The analysis was conducted using commercially available software (SAS, version 9.2; SAS Institute, Inc) and Efron's method²⁵ for tied event times. This study was considered exempt from human subjects review by the institutional review board at the University of North Carolina.

More than 100 000 patients met the inclusion criteria in each influenza season cohort, and vaccination rates were approximately 48% each year, similar to previously reported estimates (Table 1).^6,26 Patients who received the influenza vaccine were older, had fewer years with ESRD, were more likely to be white, and had better adherence to the dialysis regimen. These differences persisted during the study period. In addition, the mean age of the vaccinated cohorts increased, and the proportion who had diabetes mellitus as the cause of ESRD increased during the study period.

The A(H3N2) strain predominated in all the influenza seasons, and all were severe influenza seasons. The start of the influenza seasons ranged from late November to early January (Table 2).

Conventional analysis comparing vaccinated with unvaccinated patients resulted in average, adjusted VE estimates of 13%, 16%, and 30% for ILI, influenza/pneumonia hospitalization, and death, respectively (Table 3). Adjustment for measured confounders increased all VE estimates slightly. However, when limited to the period before the start of influenza season, the estimates were similar or stronger, which strongly suggests that confounding bias is present. The adjusted HR for death in the preinfluenza period ranged from 0.36 to 0.51, indicating that there is severe bias in the comparison between vaccinated and unvaccinated cohorts for all-cause mortality. Defining the start of influenza season with an earlier date (with 5% of isolates positive) resulted in even more biased estimates (Table 3).

Vaccinated patients in all matched years had more events than did vaccinated patients in the mismatched year, and there was little difference in the survival curves for each outcome (Figure). The models for 1998 vs 1997 and 1999 vs 1997 produced similar results, showing no benefit for any of the 3 outcomes. The comparison between 2001 and 1997 produced a small beneficial effect. The pooled ratio of HRs comparing matched seasons with a placebo season resulted in a VE of 0% (95% CI, −3% to 2%) for ILI, 2% (95% CI, −2% to 5%) for influenza/pneumonia hospitalization, and 0% (95% CI, −3% to 3%) for death (Table 4). Neither limiting the model to run only through the end of the influenza season (data not shown) nor restricting the ILI definition (eTable 2) appreciably changed the estimates. Starting follow-up on December 1 resulted in slightly stronger estimates, with the CIs for ILI and hospitalization excluding the null (eTable 2).

In this population-based study, we analyzed the natural experiment created by year-to-year variation in the match of the influenza vaccine to the circulating virus. We used the vaccine during a mismatched year as a working placebo and compared its effectiveness with that of well-matched vaccines in subsequent years. We found little evidence that the well-matched vaccines were more effective than the mismatched vaccine for the prevention of ILI, influenza/pneumonia hospitalization, and all-cause mortality among patients undergoing hemodialysis.

We also conducted traditional analyses comparing vaccinated and unvaccinated patients. These analyses revealed strong evidence of unobserved confounding. In all years, we found that the vaccinated patients were at decreased risk for all outcomes even before influenza began circulating in the community. Despite adjusting for many clinical factors, these analyses remained biased. Comparing patients who are vaccinated in one year with patients who are vaccinated in another year implicitly controlled for unmeasured aspects of health, functional status, and health behaviors that may differ between the vaccinated and unvaccinated cohorts.²⁷

Patients with ESRD have some level of immune dysfunction that may limit their ability to respond adequately to the influenza vaccine. Specifically, these patients have fewer B cells because of apoptosis and inflammatory cytokines pushing immune cell differentiation toward the T-cell pathway.^28- 29 Although immunogenicity studies have shown that patients with ESRD can produce antibodies, antibody production may not be sufficient to provide protection from influenza infection.

Because patients with ESRD may have levels of immune deficiency similar to those of elderly individuals, our results are consistent with recent work in the elderly population. Fireman et al³⁰ reported an estimate of VE for all-cause mortality of 5% (95% CI, 1% to 8%), whereas Baxter et al¹⁴ reported estimates for influenza/pneumonia hospitalizations of 12% (95% CI, 2% to 22%) in persons aged 50 to 64 years and 9% (95% CI, 3% to 14%) in those 65 years or older. Jackson et al¹² estimated VE for community-acquired pneumonia among elderly individuals as 8% (95% CI, −10% to 23%). Caution is needed, however, in comparing patients who have ESRD with the general elderly population. Patients with ESRD are seen at medical facilities 2 to 3 times per week for dialysis; therefore, the reasons for being vaccinated may be different.

Our results comparing different influenza seasons differed from a previous observational study of influenza VE in ESRD patients.⁶ The previous study compared vaccinated with unvaccinated patients and reported VE estimates during the 1998-1999 matched season of 14% (95% CI, 8%-23%) for influenza/pneumonia hospitalizations and 23% (95% CI, 19%-27%) for all-cause mortality.⁶ These results were similar to our conventional adjusted estimates. By limiting our conventional analysis to the preinfluenza period, we showed that the traditional epidemiologic approach may exaggerate the benefits of vaccination.

There are limitations to this study. First, we assumed that the vaccine was ineffective in preventing clinical outcomes in the 1997 season. If the vaccine provided some benefit, the difference in effectiveness between the matched and mismatched years would be narrowed and thus our estimate would be closer to the null than the true estimate. However, evidence from a randomized controlled trial showed that the vaccine did not protect against clinical outcomes among younger healthier people.¹⁷ Moreover, the vaccine is even less likely to have provided protection to an immune-compromised population. Second, because we used administrative claims data, we may have not adequately captured all the important confounders, particularly variables that changed between years, such as quality of care, temperature variations, or other circulating viruses. We did, however, adjust for a variety of clinical characteristics, and this is the first study, to our knowledge, to account for adherence to dialysis, which may be an important predictor of exposure to preventive health care services. In addition, we limited the comparisons to a 5-year period to limit temporal changes. Third, it is likely that the ILI outcome was underascertained. Unless physicians were making their diagnosis in part on the basis of the patient's vaccination status during the visit, this misclassification would be nondifferential. If a true effect did exist, we would expect the estimate to be stronger for a more specific influenza outcome, such as ILI, compared with a less specific outcome, such as mortality. Our estimates did not reflect this trend; therefore, it is possible that our estimate for ILI may be biased toward the null. Finally, we may have missed some vaccinations if patients received a vaccine that was paid out of pocket. Because influenza vaccine is covered by Medicare for our study population and because patients undergoing hemodialysis have health care encounters 2 to 3 times per week, we expect that the number of people who paid out of pocket would be low. These limitations cannot rule out a protective effect of the vaccine; however, we believe our findings suggest that the effect may be smaller than previously suggested.

The findings of this study should not be interpreted to mean that the practice of influenza vaccination be discouraged. Rather, they suggest that current strategies for vaccination, which rely on single dosing with a trivalent inactivated influenza virus, should be reevaluated. Alternative vaccine formulations exist and may be more suitable for the dialysis population. For example, adjuvants such as AS03_A and MF59 can act as a delivery system for the virus and potentiate the immunogenic response. One recent study demonstrated a significantly higher antibody response in patients undergoing dialysis who use the AS03_A adjuvant vaccine compared with the standard vaccine.³¹ A high-dose vaccine that contains 3 times the amount of virus compared with standard vaccine also offers an alternative strategy. Future studies should examine the clinical effectiveness of these alternate vaccination strategies.

In summary, our analysis suggests that the potential health benefits of the current influenza vaccine may be small to negligible in the dialysis population. Conventional analyses comparing vaccinated with unvaccinated groups are prone to bias. Although it is premature to discontinue vaccinating high-risk patients, alternate vaccination strategies should be investigated in patients with ESRD to achieve better health outcomes.