Nurses' Recognition of Delirium and Its Symptoms:  Comparison of Nurse and Researcher Ratings FREE

DELIRIUM IS a common and serious problem for hospitalized older patients and is associated with substantial morbidity and mortality rates.¹ Occurring in 14% to 56% of hospitalized older patients, delirium represents the most frequent complication of hospitalization for this group.^2- 3 Despite its importance, delirium is often underrecognized in the hospital setting. Previous studies^4- 7 have shown that clinicians caring for the patient do not recognize delirium in up to two thirds of cases. Whereas many barriers to the recognition of delirium have been hypothesized,¹ patient-related factors contributing to underrecognition have not been directly examined.

Nurses, who spend more time at the bedside than physicians, play a crucial role in the recognition of delirium. Because nurses have frequent and continuous contact with patients, they can better observe fluctuations in attention, level of consciousness, and cognitive functioning.^8- 9 As a result, the observations made by nurses are critical for the early detection of delirium symptoms^10- 11 and for the continuous monitoring of these symptoms that is essential to follow the patient's clinical course.¹² With training and supervision, delirium symptoms can be monitored effectively by nurses.¹²

The Confusion Assessment Method (CAM) is a simple, standardized instrument designed to assist with detection of delirium.¹³ The CAM is widely used in both clinical and research settings and has been translated into at least 5 languages. In the initial validation study, the CAM instrument was scored by geriatricians and trained research nurses after cognitive assessment with the Mini-Mental State Examination (MMSE). When validated against the ratings of geriatric psychiatrists, the CAM had sensitivity of 94% to 100%, specificity of 90% to 95%, and high interobserver reliability. However, the question remained of how the CAM would perform when rated by nurses during routine clinical care—without formal cognitive assessment. Such evaluation would offer important insight into the process of recognition of delirium in the real-world hospital setting. This evaluation provides an opportunity to examine factors that might contribute to underrecognition of delirium by nurses.

The overall goal of this study was to compare recognition of delirium by nurses based on nursing observations made during routine clinical care (without formal cognitive testing) when interviewed by trained researchers using the CAM compared with concurrent ratings made by the researchers after formal cognitive assessment. The specific objectives were to compare nurse and researcher ratings for delirium (overall) and for specific delirium features using the CAM and to identify risk factors associated with underrecognition of delirium by nurses.

This study was performed as part of a larger clinical epidemiological investigation of hospitalized older patients.^14- 16 Potential participants were 1587 consecutive patients 70 years and older admitted to the medicine and surgery (non–intensive care) floors at Yale–New Haven Hospital, New Haven, Conn, between November 6, 1989, and July 31, 1991. Yale–New Haven Hospital is an 800-bed urban teaching facility with 200 medical and 190 surgical beds serving a large community and a referral population. The community population served represents a culturally diverse group in terms of ethnicity, educational level, socioeconomic status, and country of origin. Patients were excluded if they could not be interviewed for any reason, ie, intubation, coma, severe aphasia, or terminal condition (n = 265 [17%]); if they were discharged within 48 hours (n = 232 [15%]); if they or their physicians declined participation (n = 134 [8%]); or if they had been enrolled in the study during a previous hospital admission (n = 142 [9%]). Of 814 enrolled participants, 7 (1%) were excluded from the present study because of incomplete interviews and 10 (1%) because of incomplete CAM information. Thus, the final sample included 797 participants.

Trained clinical researchers conducted standardized interviews with the participants and their primary nurses from study entry until hospital discharge. The baseline patient interview, completed within 48 hours of admission, included demographic information (eg, age, sex, race, educational level, and marital status), current living situation, self-reported activities of daily living,¹⁷ MMSE score,¹⁸ standard near-vision (Jaeger type) and hearing (whisper) tests,¹⁹ and the CAM rating for delirium.¹³ The baseline nurse interview included the nurse's rating of the patient's overall mental status (confused vs not confused), any acute change or fluctuation in mental status (yes vs no), any evidence of delirium (present vs absent), and any evidence (present vs absent) of each of the individual items of the CAM for delirium (ie, inattention, disorganized thinking, altered level of consciousness, disorientation, memory problems, or inappropriate behavior). Examples and prompts for each delirium feature were provided to assist the nurses. To avoid potential bias, the order of nurse and patient interviews was alternated, ie, nurses were interviewed first in half of the cases and patients were interviewed first in the other half. The medical record was abstracted at baseline for data required to complete an APACHE (Acute Physiology and Chronic Health Evaluation) II score.²⁰ A family member or caregiver was interviewed at baseline to complete the modified Blessed Dementia Rating Scale.^21- 22

Thereafter, the clinical researchers interviewed the patients and their nurses every other day until hospital discharge. Patient interviews included MMSE and CAM ratings. Nurse interviews included ratings of overall mental status and evidence of delirium or specific delirium features.

All data were obtained using standardized instruments. The clinical researchers were masked to the research questions and hypotheses. Informed consent was obtained from participants or, for those with substantial cognitive impairment, from the closest relative or legal guardian. The study was approved by the institutional review board of Yale University School of Medicine, New Haven.

Nurse ratings of delirium (overall and individual CAM features), based on their observations during routine clinical care without formal cognitive testing, were compared with clinical researcher ratings. For determination of sensitivity and specificity, researcher ratings, which were based on formal cognitive assessment including the MMSE and which were validated in a previous study,¹³ were used as the reference standard. Because multiple paired observations between nurse and researcher were available for each patient, the "best case" comparison was selected as the pair with the best observed agreement between researcher and nurse ratings to provide 1 rating per patient. Conversely, the "worst case" comparison was selected as the pair with the worst observed agreement between researcher and nurse ratings for that patient.

The primary outcomes were sensitivity, specificity, and concordance (κ statistic) comparing nurse and researcher ratings of delirium. Underrecognition of delirium by nurses was used as a secondary outcome for risk factor analyses.

Delirium was defined by fulfillment of the CAM criteria,¹³ which consisted of acute onset and a fluctuating course of symptoms, inattention, and either disorganized thinking or altered level of consciousness. Inattention was defined as the patient's having difficulty focusing attention, such as being distracted easily, or having difficulty keeping track of what was being said. Disorganized thinking was defined as the patient's speech being disorganized or incoherent, such as rambling or irrelevant conversation, unclear or illogical flow of ideas, or unpredictable switching from subject to subject. Altered level of consciousness was defined as a level of consciousness other than normal (alert), such as vigilant (hyperalert or overly sensitive to environmental stimuli), lethargic (drowsy but easily aroused), or stupor (difficult to arouse). Disorientation was defined as the patient's demonstrating poor orientation to time or place, such as thinking that he or she was somewhere other than the hospital, using the wrong bed, or misjudging the time of day. Memory impairment was defined as the patient's demonstrating memory problems, such as the inability to remember events in the hospital or difficulty remembering instructions. Inappropriate behavior was defined as actions considered unsafe or disruptive, such as loud vocalizations, wandering, climbing over bedrails, and physical or verbal aggressiveness.

Patient-related variables associated with underrecognition of delirium by nurses were evaluated. These investigations were performed to determine whether characteristics of the patients, or of the type of delirium, might have made identification of delirium more difficult. Baseline variables examined included patient characteristics such as age, sex, race, educational level, hearing and/or vision impairment, functional disability, cognitive impairment, and illness severity. Cutoff points for variables were selected a priori and based on clinically relevant values or previously published studies. Any activity of daily living impairment was defined as needing physical assistance with at least 1 of 7 basic care skills (ie, feeding, bathing, grooming, dressing, toileting, transferring, or walking) by patient self-report referent to 2 weeks before hospital admission; this cutoff point has been used previously.^14- 15 Hearing impairment was defined as correctly hearing 6 or fewer of 12 numbers with both ears on the whisper test.²³ Vision impairment was defined as corrected binocular near vision worse than 20/70 on the standard bedside Jaeger test.²³ Delirium risk group was defined using a previously developed predictive model¹⁴ based on 4 risk factors for delirium: vision impairment, severe illness, cognitive impairment, and a high serum urea nitrogen to creatinine ratio. High delirium risk was defined as the presence of 3 or 4 of these risk factors at baseline. Dementia was categorized according to the definition used in previous studies^16,24 as (1) a modified Blessed Dementia Rating Scale score greater than 4 or (2) a modified Blessed Dementia Rating Scale score greater than 2 and an MMSE score less than 20 and duration of cognitive symptoms of at least 6 months. To indicate high severity of illness, the cutoff point of greater than 16 was used for the APACHE II index.^14,25 One postbaseline variable, the presence of hypoactive delirium, was examined for its association with underrecognition by nurses. Hypoactive delirium was defined as the presence of delirium with psychomotor retardation, characterized by an unusually decreased level of motor activity, such as sluggishness, staring into space, staying in one position for a long time, or moving extremely slowly. Although different pathogenic mechanisms for delirium (eg, infection, medications, and metabolic derangements) may have the potential to affect recognition of delirium, exploration of these myriad factors was beyond the scope of the present study.

Sensitivity, specificity, and 95% confidence intervals for sensitivity and specificity were calculated using standard formulas.^26- 27 The κ coefficient for concordance was used as the index of agreement exceeding chance, and the 95% confidence interval for κ was calculated using standard approaches. Kappa values greater than 0.75 indicate excellent agreement beyond chance, values from 0.40 to 0.75 indicate fair to good agreement beyond chance, and values less than 0.40 indicate poor agreement beyond chance.^28- 29 For these analyses, nurse ratings were compared with researcher ratings, the reference standard. First, nurse ratings for delirium were compared with researcher ratings for all observations. Subsequently, the best case and worst case (see the "Comparison of Nurse and Researcher Ratings" subsection) observations were compared to provide 1 observation per patient. Finally, the individual CAM delirium symptoms were compared to evaluate whether specific criteria were consistently rated differently by nurses compared with researchers.

In bivariate analyses, rates of delirium not being recognized by nurses were calculated for the best case comparison when each risk factor was present or absent. Crude odds ratios and associated 95% confidence intervals were calculated. Subsequently, variables with odds ratios greater than 2.0 and clinical relevance were selected for evaluation in multiple logistic regression analysis. The final risk factors were selected using a stepwise algorithm with backward elimination (P<.10 to remove a variable). To avoid redundancy, vision impairment was selected for inclusion from among several sensory impairment variables because it had the strongest bivariate association with underrecognition by nurses. Adjusted odds ratios and 95% confidence intervals were calculated for the final independent risk factors from the parameter estimates and SEs.

A risk stratification system was developed by adding 1 point for each of the final risk factors present for each patient. The Mantel-Haenszel χ² trend test was used to compare rates of underrecognition by nurses between risk groups.

All statistical tests were 2-tailed, and P<.05 was considered statistically significant. All analyses were performed using the SAS statistical program (version 6.10; SAS Institute Inc, Cary, NC).

Baseline characteristics of the study population of 797 patients are shown in Table 1. Delirium occurred in 239 (9%) of 2721 observations or 131 (16%) of 797 patients. This population represents an acutely ill older population (mean age, 78.4 years) of hospitalized medical and surgical patients. The degree of cognitive and functional impairment at baseline was substantial, with 40% of patients having an MMSE score less than 24 and 60% having at least 1 activity of daily living impairment at baseline. Overall, 2721 paired observations by nurses and researchers were made in the 797 patients, for a mean ± SD of 3.4 ± 2.1 observations per patient. The specific nurse often varied between observations on the same patient.

Table 2 presents the comparison of nurse and researcher delirium ratings for all paired observations and for best and worst case comparisons. Researchers found evidence of delirium in 239 of 2721 assessments; nurses identified delirium in only 46 of these observations (19.3%). Of 131 patients who developed delirium, nurses identified this condition in only 40 (30.5%) using best case comparisons. Thus, the sensitivity of nurse ratings for delirium using the CAM criteria was generally low compared with that of researcher ratings (19.3% overall, 30.5% for best case, and 13.7% for worst case), indicating that most cases of delirium were not identified by nurses. The specificity of nurse ratings for delirium, however, was high compared with that of researcher ratings (95.8% overall, 99.4% for best case, and 90.8% for worst case), indicating that nurses did not overidentify delirium (ie, identify a patient as delirious when he or she was not). Agreement beyond chance was generally poor (κ = 0.18 overall, 0.40 for best case, and 0.05 for worst case).

Table 3 presents the best case comparison of nurse and researcher ratings of individual CAM features. For the 3 key delirium features used in the CAM algorithm—inattention, disorganized thinking, and altered level of consciousness—the sensitivities ranged from 15% to 26%, indicating that these features were often missed. For example, nurses identified inattention only 25 (15%) of the 163 times that this feature was present. Disorganized thinking was noted by nurses in only 20 (26%) of 76 observations where present, and altered level of consciousness was noted in only 18 (15%) of 121 positive observations. However, the high specificities (99% for all) indicate that these features were rarely identified by nurses when absent. Kappa statistics ranged from 0.22 to 0.37, indicating poor agreement exceeding chance. Disorientation and memory impairment were recognized in only 30 (7%) of 436 and 27 (4%) of 607 cases, respectively, when present, with κ statistics ranging from 0.02 to 0.06, indicating extremely poor agreement. Inappropriate behavior was better recognized by nurses, with a sensitivity of 47% and κ of 0.36. Notably, specificity rates remained high for all features. Nearly all disagreements in ratings between nurses and researchers were because of underrecognition of delirium and its symptoms by nurses when they were present.

Potential variables associated with underrecognition of delirium by nurses are presented in Table 4. Of 14 variables considered, 6 were selected on the basis of clinical relevance and quantitative significance (odds ratio >2.0): age 80 years and older, any impairment in activities of daily living, vision impairment, high baseline delirium risk, dementia, and presence of the hypoactive form of delirium. After multivariable analysis, 4 final risk factors were selected (Table 5): hypoactive delirium, age 80 years and older, vision impairment, and dementia. Patients with hypoactive delirium demonstrated a 7-fold risk of underrecognition by nurses.

To demonstrate the cumulative effect of these risk factors, a risk stratification system was developed by summing the number of risk factors present in each patient (Figure 1). With no risk factors present, the rate of not being recognized by nurses as having delirium was 2%, increasing to 44% when 3 or 4 risk factors were present. As shown in Figure 1, patients who were older, visually impaired, demented, and with hypoactive delirium were 20-fold times less likely to be recognized by nurses as having delirium than those who did not have these risk factors.

Nurses are key to the early recognition of delirium, yet this study showed that when the CAM was rated by untrained nurses as part of routine clinical care and without any formal cognitive assessment, delirium was often unrecognized. Nurses were asked to rate the presence of delirium, as well as specific delirium features, using the CAM. Despite prompting by trained researchers, nurses properly identified delirium in only 19% of observations overall and in only 31% of patients where present. In addition, nurses were able to identify key features of delirium (ie, inattention, disorganized thinking, and altered level of consciousness) in only 15% to 26% of patients. Thus, compared with ratings by trained researchers after cognitive assessment including the MMSE, nurses had generally low sensitivity for detection of delirium or specific delirium features, and delirium was usually unrecognized. However, the specificity of nurse ratings was high for detection of delirium or specific delirium features; thus, delirium was rarely identified when it was not present. Nearly all disagreements between nurse and researcher ratings were because of underrecognition of delirium or its features by nurses.

The study findings confirm those of the original CAM validation study¹³ that the proficiency and thoroughness of the primary observations on which the ratings are based substantially influence the effectiveness of the CAM for delirium screening. These findings are comparable to those of previous studies. Rockwood et al⁴ found a substantial false-negative rate (32%), but few false-positives (3%) when ratings by nurses were compared with those by study physicians using delirium criteria from the Diagnostic and Statistical Manual of Mental Disorders, Revised Third Edition.³⁰ In another study,¹² the false-positive rate was somewhat higher, 10 (18%) of 55 patients, and false-negative ratings were not assessed.

Four independent risk factors for underrecognition of delirium by nurses have been identified: presence of the hypoactive form of delirium, age 80 years and older, vision impairment, and dementia. When all 4 of these risk factors were present, the risk of underrecognition was increased more than 20-fold. Twenty years ago, Wolanin and Phillips³¹ cited the failure of nurses to identify the hypoactive form of delirium because such patients tended to cooperate with their care. Nurses tended to label patients as delirious when their behavior made them difficult to care for. Palmeteer and McCartney³² reported that nurses did not recognize cognitive impairment in a substantial proportion of older hospitalized patients compared with researchers using a standardized tool. Nurses tended to use patient behavior as an indication of cognitive function, and they mistook compliance as an indication of intact cognition. Thus, cooperative patients with hypoactive delirium were consistently not identified. In addition, a recent study³³ noted that nurses tended to overlook symptoms of delirium in patients with dementia. The recognition of risk factors associated with poor recognition offers the opportunity to heighten the awareness of health care providers to situations in which delirium is likely to be missed. Exploration of the effect of different pathogenic mechanisms on the recognition of delirium is an important area for future research.

The poor recognition of disorientation and memory impairment by nurses was likely due to the lack of assessment of these important domains. Previous studies have found that nurses tend to rely considerably on the presence of disorientation for detecting delirium; yet, disorientation alone—particularly in hospitalized older patients—tends to lack sensitivity and specificity for delirium.³⁴ Palmateer and McCartney³² demonstrated that nurses were not knowledgeable about cognitive assessment, dementia, or delirium. Only 35% of nurses passed a knowledge test, and most did not consider cognitive testing to be a high priority in the clinical care of older patients.

The strengths of the present study include the quality and consistency of the reference standard ratings for delirium, which have been previously validated.^13- 14 In addition, the large sample size and wealth of patient-related risk factor data to examine as potential predictors of underrecognition of delirium by nurses are important advantages. Finally, the multiple paired ratings per patients enabled us to examine the full range of performance across pairs and to select best case and worst case comparisons—providing a sensitivity analysis of our results. Best case comparisons were used for later analyses to provide the optimal assessment of nursing performance.

Several important caveats deserve comment. First, nurses often varied between observations on the same patient. In addition, nurses differed widely in how well they knew the individual patients and their level of clinical experience with older patients. These factors, however, are likely to reflect real-world practice in the general hospital setting. Moreover, this study was performed at a large academic teaching hospital, where nursing turnover is relatively high. Although internal validity should not be compromised, the study results may not generalize to all settings, particularly to centers where nursing turnover is low or where geriatric nursing expertise is high. Furthermore, the nursing environment has changed since the time this study was completed. Although nurses continue to have the most frequent and continuous contact with patients, the quantity and quality of that contact have been compromised by contemporary acute care delivery models, with decreased nursing staff ratios and substitution of unskilled patient care assistants.³⁵

This study holds substantial implications for care of hospitalized older patients. Although physicians often do not identify delirium in their hospitalized older patients,^4- 7 they typically see patients for only brief periods and rely heavily on nursing observations for issues such as mental status changes. The inability of nurses, who have 24-hour contact with patients, to identify delirium is of perhaps greater concern. Nurses are at the front line in the process of delirium recognition, and improvements in this process will be essential to make any inroads to addressing the problem of delirium.^36- 38 Education and training are needed for detection of delirium and its key features by nurses during routine clinical care. Instruction in use of brief cognitive assessment is required or these key features will be missed.³⁹ Such education should ideally start in nursing school and should be an important component of continuing education programs for the nursing profession.

Strategies to enhance training and education for recognition of delirium will likely require substantial effort on the part of all health care providers—nurses and physicians alike—and changes in current health care practices. A widespread, brief nursing mental status assessment of hospitalized older persons would represent a tangible start to this process. Although imposing some burden on staff time, the costs should be more than offset by savings resulting from early recognition and intervention for delirium, which is associated with substantial morbidity and mortality rates and poor hospital outcomes. Without the ability to recognize delirium promptly, intervention strategies to decrease its impact will have limited success, and evaluation of the effectiveness of prevention programs will not be possible. Future studies are needed to examine whether implementation of the recommended strategies, such as education and training of nursing staff in delirium recognition, routine cognitive assessment of older patients, and heightened awareness to high-risk patients, would result in improved recognition of delirium and reduction in its adverse outcomes.

Accepted for publication March 13, 2001.

This study was supported in part by grant RO1AG12551 and Midcareer Award K24AG00949 (Dr Inouye) from the National Institute on Aging, Bethesda, Md; by in-kind support from the Claude D. Pepper Older Americans Independence Center (P60AG10469) from the National Institute on Aging; and by Donaghue Investigator Award DF98-105 from the Patrick and Catherine Weldon Donaghue Medical Research Foundation, West Hartford, Conn (Dr Inouye). This article was completed while Dr Inouye was working on research travel grant 1001225 from the Burroughs Wellcome Fund, Morrisville, NC.

We thank the nurses, patients, and families from Yale–New Haven Hospital who participated in this study and research staff at the Yale Program on Aging. This work is dedicated to Benjamin, Joshua, and Jordan Helfand.

Corresponding author and reprints: Sharon K. Inouye, MD, MPH, Yale University School of Medicine, Yale–New Haven Hospital, 20 York St, Tompkins 15, New Haven, CT 06504.