The Role of Adherence on the Effectiveness of Nonpharmacologic Interventions:  Evidence From the Delirium Prevention Trial FREE

ADHERENCE IS defined as the extent to which medical recommendations are followed and implemented, typically representing the degree of the patient's compliance with a prescribed drug regimen. As such, adherence has received widespread recognition as a fundamental contributor to the effectiveness of pharmacotherapy for a variety of conditions, notably human immunodeficiency virus therapy,^1- 3 alcohol dependence,^4- 5 coronary artery disease,^6- 7 hypertension,^8- 9 antibiotic prophylaxis,¹⁰ depression,^11- 13 and schizophrenia.^14- 15 However, the importance of adherence for the effectiveness of nonpharmacologic treatment strategies has not been examined systematically. Specifically, issues such as the existence of a "dose"-response relationship for nonpharmacologic interventions and the impact of influences that are beyond the control of the patient (eg, interventions requiring adherence by nurses, physicians, and other health care providers) have not been investigated previously. Examination of these issues is crucial to better understand the effectiveness of the broad domain of nonpharmacologic interventions, including educational and behavioral strategies. Moreover, in this era of cost containment, the issue of adherence assumes particular importance for the cost of care because considerable resources and staff time are required to ensure its achievement.

The data for this study of adherence were obtained during the performance of the Delirium Prevention Trial,¹⁶ a controlled clinical trial of a multicomponent intervention for the prevention of delirium in hospitalized older patients. This trial, already published, showed that a multicomponent nonpharmacologic intervention strategy could significantly reduce the incidence of delirium. The intervention incorporated protocols that involved patients and staff to address 6 delirium risk factors. The data collected for the present study were designed to examine how much adherence to a complex intervention strategy by patients and staff influenced the outcome of the nonpharmacologic approaches.

The specific aims of this study were to examine the impact of level of adherence on effectiveness of the interventions in the Delirium Prevention Trial. Our hypothesis was that the extent of adherence with the intervention strategy would be a critical determinant of intervention effectiveness, and that the more of the intervention a patient received, the less likely the patient would be to develop delirium.

This study included the patients in the intervention group from the Delirium Prevention Trial, a prospective cohort of patients, described in detail elsewhere.¹⁶ Briefly, potential participants were consecutive patients admitted to one general medicine floor (non–intensive care) at Yale New Haven Hospital (New Haven, Conn) from March 25, 1995, through March 18, 1998. Yale New Haven Hospital is a 900-bed urban teaching hospital with 200 medical beds, serving a large community and referral population. A total of 871 patients met the inclusion criteria of 70 years or older, no delirium at admission, and at least intermediate risk of delirium at baseline as determined by our previously developed risk stratification system.¹⁷ Of these, 335 patients were excluded because of inability to participate in interviews for reasons such as profound aphasia or intubation (n = 117), coma or terminal illness (n = 34), hospital stay of less than 48 hours (n = 89), or other reasons including unavailability of interviewer or patient (n = 95). The excluded patients (n = 335) did not differ significantly from the enrolled patients (n = 422) in terms of age, sex, or baseline delirium risk. Finally, there were 114 refusals by patients, families, or physicians. Thus, the final study sample included 422 patients.

Informed consent for participation was obtained verbally from the study patients or, for those with substantial cognitive impairment, from a proxy (relative or legal guardian), according to procedures approved by the institutional review board of Yale University School of Medicine, New Haven.

The intervention protocols, described in Table 1, were implemented by specially trained hospital staff members called "Elder Life Specialists," assisted by trained volunteers and overseen by a geriatric clinical nurse specialist and geriatrician. The intervention protocols were designed to be practical and feasible approaches to address well-documented risk factors for delirium.¹⁶ The protocols included orientation, therapeutic activities, mobility, sleep, hearing or vision, and volume repletion. These protocols were targeted either to all patients or to the appropriate subgroup of patients with the identified risk factor. Table 1 describes the standardized intervention protocols and the targeted patients for each protocol.

On admission, eligible patients were enrolled and assigned to 3 or more appropriate intervention protocols by the Elder Life Specialists.^16,18 All patients were assigned to receive an orientation, mobility, and therapeutic activities protocol; other protocols were assigned according to risk factors present at screening. Patients were reassessed daily for changes in risk factors, such as level of orientation, insomnia, or dehydration, that might necessitate changes in their protocol assignments. All intervention staff and volunteers underwent quarterly standardization with completion of competency-based checklists to ensure consistent application of all intervention protocols.

On a daily basis, the Elder Life Specialists recorded the level of adherence to each assigned intervention protocol for each patient by reviewing forms filled out by the interventionist after completing each protocol. The data recorded included the intervention assignment and the degree of completion of each protocol assigned (ie, number of times and duration if applicable, full or curtailed receipt of all parts of intervention). If any intervention protocol was not completed in full, detailed information on reasons for nonadherence were recorded, including reasons such as patient refusal, lack of availability of the patient because of medical procedures, lack of availability of intervention staff members, or severe medical symptoms preventing participation.

Adherence was considered complete if the patient received all parts of the assigned protocol for the total number of times it was to be given that day. Partial adherence indicated that the patient either received some but not all parts of the protocol on at least one occasion or did not receive the protocol for the required number of times that day. Nonadherence indicated that none of the parts of the assigned protocol was received that day. An adherence rate was calculated for each protocol for each patient-day. A summary adherence measure for each protocol was computed for the entire hospital stay as the average of all the daily adherence rates for that patient. To accomplish this, the adherence rates for each patient-day were summed and divided by the number of assigned days, yielding an average adherence score for each intervention protocol. These average adherence scores were further categorized as follows. The goal was to categorize the average adherence scores into 3 groups defined as low, intermediate, and high adherence groups. Although initially attempted, the use of the same definition to categorize each intervention into the 3 adherence groupings was not feasible, since the distributions were highly skewed for some protocols. Thus, categorizations were selected that resulted in as balanced a distribution of patients across the 3 groupings as possible. For vision-hearing and volume repletion protocols, only 2 categorizations could be created. It is important to note that all of these categorizations were selected a priori solely on the basis of the distribution of patients in the groupings, without reference to delirium rates.

On the basis of the 3 intervention protocols that were assigned to all study subjects (orientation, mobility, and therapeutic activities), a composite overall adherence score was computed. The other 3 intervention protocols (sleep, vision-hearing, and volume repletion) were excluded from the composite score because (1) they were received by only a proportion of the sample and (2) they were targeted toward patients with specific risk factors, making it difficult to assess intervention effectiveness (and, thus, the effect of adherence on effectiveness) across all of these protocols. The composite scores were calculated by assigning 0 points for low adherence, 1 point for intermediate adherence, and 2 points for high adherence on the orientation, mobility, and therapeutic activities protocols. These scores were summed across the 3 protocols, resulting in a composite score with a range from 0 to 6.

All other study variables, including the delirium outcome, were collected by a separate research team who had no role in the intervention and who were blinded to the research question, the interventional nature of the study, and the patients' group assignment. The research team was composed of experienced clinical research staff who underwent intensive training, standardization, and interrater reliability assessment in all interview and data collection procedures.¹⁶ The screening and baseline interviews, completed within 48 hours of admission, included demographic information, current living situation, self-reported Katz activities of daily living¹⁹ and instrumental activities of daily living²⁰ 2 weeks before admission, shortened Geriatric Depression Scale,²¹ standard near-vision (Jaeger type) and hearing (Whisper)²² tests, Mini-Mental State Examination,²³ Digit Span Test,²⁴ and the Confusion Assessment Method.²⁵ Baseline delirium risk was defined according to a previously validated predictive model for delirium, based on 4 delirium risk factors: visual impairment, severe illness, cognitive impairment, and high blood urea nitrogen–creatinine ratio.¹⁷ Intermediate risk was defined as the presence of 1 or 2 risk factors at baseline, and high risk as the presence of 3 or 4 risk factors. A nurse was interviewed on admission to provide an overall subjective rating of illness severity.²⁶ A family member was interviewed at the time of admission to describe the patient's cognitive functioning before admission and to complete the modified Blessed Dementia Rating Scale.^27- 28

Thereafter, patients were examined daily until discharge with a structured interview consisting of the Mini-Mental State Examination, Digit Span Test, and the Confusion Assessment Method. After discharge, medical records were reviewed for medical diagnoses, laboratory results, vital signs, and pertinent clinical data needed to rate the Acute Physiology and Chronic Health Evaluation (APACHE II) score²⁹ and Charlson Comorbidity Index.³⁰ All medical record abstractors were blinded to the intervention group assignment, adherence status, and delirium outcome status of each patient.

The primary outcome was new-onset delirium during hospitalization, defined according to the Confusion Assessment Method criteria,²⁵ which required the presence of acute onset and fluctuating course, inattention, and either disorganized thinking or altered level of consciousness. Each of these features was rated by the blinded research staff on the basis of cognitive assessment and observations made during the daily interviews. The Confusion Assessment Method criteria provide a standardized rating of delirium, which has been previously validated against geropsychiatric diagnoses, with a sensitivity of 94% to 100%, a specificity of 90% to 95%, and high interrater reliability.²⁵ For the present study, delirium was considered as present or absent according to its earliest occurrence, and only one episode of delirium per patient was counted. High interrater reliability for the delirium rating was confirmed in 16 paired observations that involved all research staff in the present study (κ = 1.0). Although delirium was the primary outcome measure, adherence was the independent variable against which the outcome was measured.

The characteristics of the study group were described with appropriate descriptive statistics, including means and SDs for continuous variables and percentages for categorical variables. We described assignment to intervention protocols by the number and proportion of patients assigned to each protocol at any time during their hospital stay, and among those assigned a given protocol, adherence rates by patient-day. For each intervention protocol, the delirium rate was computed for each level of adherence and the trend in delirium rate with increasing adherence was assessed via the 2-sided Cochran-Armitage test for trend. To account for the small sizes of some cells, the exact method of calculating the P value was used.

The proportion of patients developing delirium at each level of the overall composite adherence measure was calculated. To assess the relationship between the overall composite adherence score and the incidence of delirium, adjusting for other factors likely to be associated with adherence and risk of delirium, we conducted logistic regression analysis. In these models, the composite adherence measure was treated as a continuous predictor, and the resulting odds ratios estimated the reduction in the odds of delirium for a 1-point increase in adherence. By comparing the fit of these models with a model with a separate indicator variable for each level of adherence, we tested whether the association between our adherence measure and delirium could be considered a linear relationship. Comparison of these models demonstrated that the fit of the logistic models treating the adherence measure as categorical was no better than the models in which adherence was kept linear; thus, adherence was retained as a continuous measure in the final model.

All statistical tests were 2-tailed, and a P value of less than .05 was considered to indicate statistical significance. All analyses were performed with the SAS statistical program (SAS Versions 6.12 and 8.01, SAS Institute Inc, Cary, NC).

Characteristics of the patients in the study are shown in Table 2. The cohort represents a frail older population with substantial degrees of functional and cognitive impairment (86.7% with impairment of instrumental activities of daily living, 34.4% with impairment of activities of daily living, and 41.0% with Mini-Mental State Examination score <24), as well as high indexes of illness burden (mean APACHE II score of 15.5 and mean Charlson index score of 3.1).

Patients assigned to the intervention protocols and the adherence rates with each protocol by patient-days are indicated in Table 3. The orientation protocol had the highest adherence rates, and the sleep protocol had the lowest rates. The overall rate of complete adherence with all intervention protocols across all patient-days was 57% (5610/9882). Combined partial and complete adherence with all intervention protocols across all patient-days was 87% (8593/9882). No adverse effects were associated with the intervention protocols. The most common reasons for nonadherence in 1289 (13%) of 9882 patient-days were lack of availability of intervention staff members (52%), patient refusal (27%), lack of availability of the patient because of medical procedures (10%), and severe medical symptoms preventing participation or medical contraindication (7%).

Delirium rates by adherence groupings for each intervention protocol are shown in Table 4. Delirium rates were lower when adherence was higher for orientation, mobility, and therapeutic activities—demonstrating the beneficial effect of higher levels of adherence with these protocols on the outcome. While trends were demonstrated for sleep and vision-hearing protocols, these values did not achieve statistical significance.

Using the composite adherence score, calculated from the orientation, mobility, and therapeutic activities protocols (See "Statistical Analysis" subsection of the "Methods" section), Figure 1 demonstrates a significant decrease in incidence of delirium with higher levels of adherence (P_trend = .002). Even after stratification by baseline delirium risk group (intermediate vs high), the relationship of lower incidence of delirium with higher levels of adherence persisted (P_trend = .04 for each delirium risk group).

Table 5 shows the impact of adherence on the risk of delirium, and the effect of adding important covariables that were selected because of their potential impact on either adherence or incident delirium in multivariable analyses. The unadjusted odds ratio of 0.67 indicated a substantial reduction in risk of delirium associated with each 1-point increase in adherence score. The fully adjusted model—controlling for age, sex, education, Charlson score, depression, impairment in activities of daily living, illness severity (APACHE II and nurse's illness severity rating), Mini-Mental State Examination score, blood urea nitrogen–creatinine ratio, and visual impairment—yielded an adjusted odds ratio of 0.69. Thus, the fully adjusted model demonstrated a consistently strong and significant protective effect of adherence on incident delirium, and the relationship demonstrated little confounding by the multiple covariables included in the model.

To translate the odds reduction of 0.69 for each 1-point change in adherence level, we calculated the corresponding odds reduction for a patient in the highest level of adherence (adherence score of 6), compared with the lowest level (score of 0). For a patient in the highest adherence level, for example, the corresponding odds ratio would be 0.111—that is, the odds of delirium are 11.1% of those of a patient in the lowest adherence group, which translated into an 89% reduction in delirium risk. In our sample, patients in the highest adherence group had a delirium rate of 2.9%, compared with 38.1% in the lowest adherence group.

Our observations show that the effectiveness of a nonpharmacologic multicomponent intervention strategy for a serious condition can be critically dependent on the level of adherence with the interventions. In the Delirium Prevention Trial, the degree of adherence correlated with the effectiveness of the protocol. Higher levels of adherence resulted in reduced rates of incident delirium in a graded fashion. In the fully adjusted model, the risk of delirium of a patient in the highest adherence group was 89% lower than the risk in the lowest adherence group. In the highest adherence group, the rate of delirium was less than 3%. The relationship of adherence and delirium persisted even after controlling for baseline delirium risk, severity of illness, and comorbidity in stratified and multivariable analyses. Thus, adherence was not merely a reflection of severity of illness, with sicker patients being more likely to be nonadherent. In fact, adherence demonstrated an independent, statistically significant effect on delirium even in the face of multiple covariables. Multivariable analyses showed that these covariables had relatively little confounding effect on the relationship of adherence and incident delirium.

The results demonstrate convincingly that the intervention strategy works, and the more of the intervention received by the patients, the better the response. Moreover, one component alone did not account for the effect, but rather, several components worked together to produce the effect, lending strong support for the multicomponent approach and for the underlying mechanism of multiple risk factor reduction. These results were neither obvious nor predictable. It was equally likely that one component might have driven most of the effect, or that effectiveness might have been demonstrated only with complete adherence, rather than a consistently graded response across adherence levels. Thus, these findings suggest that the multicomponent intervention strategy should be conducted as designed, and that adherence should be ensured to result in an effective intervention.

Previous studies have demonstrated that patients with high adherence to pharmacologic interventions may have better outcomes than patients with low adherence, whether they receive active intervention or placebo^31- 33; however, in these studies, the patients themselves were the primary determinants of adherence. In the Delirium Prevention Trial, adherence was only partly determined by patient factors. In fact, the main reason for nonadherence—lack of availability of intervention staff—was beyond the control of the patients. This factor may explain in part the relative lack of influence of patient-related covariables on the relationship between adherence and incident delirium.

Strengths of this study include the detailed daily tracking of adherence with each intervention protocol, the daily assessment of patients for the outcome of delirium by means of a standardized validated instrument by blinded research staff with no losses to follow-up, and the comprehensive patient-related data on severity of illness, comorbidity, and other potential confounding variables. Nonetheless, we should point out that only 3 intervention components were included as a part of the composite overall adherence measure. Three components were excluded because they were received by only a proportion of the sample; moreover, adherence with these 3 protocols (sleep, vision-hearing, and volume repletion) did not demonstrate a significant trend with the delirium outcome variable. In addition, because the latter 3 intervention protocols were targeted toward patients with specific risk factors present (rather than the entire cohort), it was difficult to compare intervention effectiveness—and consequently, the related effect of adherence on effectiveness—across all protocols.

Our results hold substantial implications. Methods to monitor and to ensure adherence appear to be as critical to the effectiveness of nonpharmacologic intervention strategies as they are to those that involve drugs. The direct relationship between adherence and outcome demonstrated in this study lends strong support for the potency of nonpharmacologic interventions in influencing important medical outcomes, and for the need to apply such interventions completely and consistently. The refinement of nonpharmacologic multicomponent intervention studies, based on the results of adherence-outcomes analyses, may contribute to improved medical outcomes by identifying the most effective intervention protocols. Such refinement has important economic implications during replication and dissemination of these multicomponent intervention strategies. For example, a recent meta-analysis of all hospital-based fall-prevention programs³⁴ commented that the effectiveness of a variety of multicomponent intervention strategies was difficult to establish, as no studies analyzed compliance with their published interventions. Thus, adherence-outcomes evaluations provide crucial information for clinicians, administrators, and policymakers to formulate recommendations about multicomponent intervention strategies. Moreover, future work is needed to examine further the factors influencing adherence in the Delirium Prevention Trial, including both staff- and patient-related factors, and to determine strategies to improve adherence with nonpharmacologic intervention protocols.

Corresponding author: Sharon K. Inouye, MD, MPH, Yale University School of Medicine, 20 York St, Tompkins 15, New Haven, CT 06504.

Accepted for publication July 11, 2002.

This study was supported in part by grant R01AG12551 from the National Institute on Aging, Bethesda, Md, and by in-kind support from the Claude D. Pepper Older Americans Independence Center (P60AG10469), given by the National Institute on Aging. Dr Inouye is a recipient of a Midcareer Award from the National Institute on Aging (K24AG00949) and a Donaghue Investigator Award from the Patrick and Catherine Weldon Donaghue Medical Research Foundation (DF98-105), West Hartford, Conn. Dr Agostini is a Robert Wood Johnson Clinical Scholar at Yale University, New Haven.

We are indebted to the patients, families, nurses, and volunteers at Yale New Haven Hospital who participated in the study; to the Project Recovery research staff; and to the Elder Life Program intervention staff. This work is dedicated to the memory of Alvan R. Feinstein, MD.