From the Department of Medicine, St Joseph's Hospital and Medical Center, Phoenix, Ariz (Drs Heffner, Fracica, and Brown and Ms Barbieri), and University of Arizona Health Sciences Center, Tucson (Drs Heffner, Fracica, and Brown).
Do-not-resuscitate (DNR) orders for critically ill patients are frequently miscommunicated between attending physicians, house staff, and nurses. A computer-based system was developed to improve the communication of a procedure-specific DNR order form.
Concordance of understanding of patients' DNR status was measured with the use of unstructured DNR orders (period 1), procedure-specific DNR order forms (period 2), and procedure-specific DNR order forms administered with a computer-based communication system (period 3). The 3 components of the DNR order assessed were (1) the clinical events to which the DNR order applied, (2) whether the DNR order withheld all elements of cardiopulmonary resuscitation, and (3) whether other treatments were to be withheld.
For the 147 patients, the computer-based system in period 3 (n=71) improved concordance for attending physicians and nurses or residents for all 3 of the DNR components compared with period 1 (n=40) and some of the DNR components compared with period 2 (n=36). Concordance was "substantial" or "almost perfect" as measured by the κ statistic during period 3. The proportion of agreement for the composite of all 3 components of the DNR order increased during each period (P<.001, period 3 vs period 1). Overall agreement between all caregivers for the composite DNR order also improved from period 1 (22.2%) to period 2 (47.8%) and period 3 (61.9%; P<.001 vs period 1). Errors in order entry were detected by physicians because of the computer system and corrected in 9.9% of DNR orders in period 3. Progress note documentation of DNR status did not improve during period 3. The procedures of period 3 were considered acceptable by the physician and nursing staff.
A computer-based system combined with a procedure-specific DNR order form improves communication of patients' DNR status in a critical care setting.
THE MANAGEMENT of death and dying in the intensive care unit (ICU) has become increasingly dependent on the accurate communication of treatment plans for limiting life-support care. Recent studies indicate that up to 90% of patients who die in the ICU in some medical centers undergo withholding or terminal withdrawal of life support under the direction of do-not-resuscitate (DNR) orders.1 Procedures to ensure the accurate communication of treatment limitations gain added importance in ICU settings wherein multiple physicians, nurses, and house staff with rotating work schedules contribute to patient care but have limited time to directly interact. Moreover, DNR orders have become increasingly intricate, often containing instructions for withholding some but not all components of cardiopulmonary resuscitation.2 In the complex environment of the ICU, proper implementation of DNR orders and preservation of patient autonomy rely on the accurate transfer of patient and family wishes through the attending physician to bedside caregivers (house staff and nurses)who have the responsibility for correctly carrying out treatment limitation plans.
Unfortunately, recent studies indicate that poor communication, misinterpretations, misunderstandings, confusion, and incomplete documentation commonly distort the intent of DNR orders and other treatment limitation plans.2- 5 Bedside caregivers often misunderstand the definition of cardiopulmonary resuscitation and the operational meaning of a patient's resuscitative status.3 Inaccurate communication of DNR status risks the inappropriate initiation or withholding of life-support interventions counter to patient and family wishes. Inappropriate resuscitation may restore cardiovascular function but leave patients with painful and debilitating complications and injuries.6,7 Inappropriate withholding of life support is a grave error because up to 60% of patients with a DNR order survive to hospital discharge.8
Procedure-specific DNR order forms attempt to promote accurate communication and more comprehensive documentation of DNR status.2,5 In a validation study of a procedure-specific DNR order form,5 we observed improved communication among caregivers, but total congruence of understanding of all elements of the DNR order between attending physicians, residents, and nurses existed for only 48% of patients. In the present study, we measured the effect on communication of supplementing a procedure-specific DNR order form with a computer-based communication system and a caregiver educational program.
Study patients were selected from the 32-bed medical and surgical ICUs at St Joseph's Hospital and Medical Center, an urban tertiary care referral center in Phoenix, Ariz. The ICUs allowed any medical staff physician with appropriate hospital credentialing to care for patients with medical, surgical, trauma, cardiac, neurologic, or neurosurgical conditions. Multiple house staff teams worked in the units and included a 6-resident medical/cardiac critical care team that only cared for ICU patients and neurosurgical, neurologic, surgical, and family practice residents who had additional patient care responsibilities outside the ICU. A study nurse who performed rounds on patients daily entered adult (>18 years of age) patients into the study who had a DNR order placed on their medical charts while in the ICU.
The study was conducted in 3 consecutive periods. Patients managed from periods 1 and 2 were previously described5 and served as the comparison groups for patients managed with the computer-based system during period 3. Period 1 (July 1993 through January 1994) comprised patients who were managed with DNR orders that were placed in the medical record on unstructured, standard physician order sheets. Period 2 (March 1994 through January 1995) patients were managed with a previously described5 procedure-specific DNR order form that contained (1) a synopsis of the medical center's DNR policy encouraging chart documentation of a patient's DNR status and (2) a checklist of the specific life-support and other patient care procedures intended to be withheld. Patients in period 3 (August 1995 through October 1996) were managed with the procedure-specific DNR order form in addition to a computer-based system that generated a DNR communication sheet. Residents and nurses in the ICU received at the outset of period 3 and monthly thereafter an educational inservice on DNR policies and procedures.
During all 3 periods, the decision to place a DNR order on the medical record was reserved for attending physicians. Attending physicians could write and sign the DNR orders or give verbal orders to nurses or residents for entering the DNR orders. Verbal orders required a cosignature by the attending physician within 24 hours. The hospital policy for withholding and withdrawing life support instructed attending physicians or their resident surrogates to complete a progress note with the following elements: (1) a description of the patient's physical condition corroborating the prognosis, (2) a reference to any consultations corroborating the prognosis, (3) a description of any discussions with the patient (or the surrogate for incompetent patients) regarding the patient's or surrogate's understanding of the patient's prognosis and the consequences of the decision to limit care, and (4) a statement regarding the existence of advance directives.
A DNR communication program was developed using Supercard software (Allegiant Technologies, San Diego, Calif) for a Macintosh 6100 computer (Apple Inc, Cupertino, Calif) stationed in the ICU. The program reproduced the procedure-specific DNR order form on the computer screen. The life-support interventions that were ordered by the attending physician to be withheld on the DNR order form could be selected onscreen by a nurse or unit clerk. Appropriate operator prompts were provided to guide data entry. As a quality control of the correctness of the physicians' completion of the procedure-specific DNR order form, alert windows opened if mutually exclusive combinations of orders were entered into the computer (eg, "the patient is a full DNR," and "the patient is a partial DNR"). The alert windows provided prompts for the clerk or nurse to notify the attending physician immediately of incorrectly completed procedure-specific DNR order forms.
After completion of data entry, the computer printed a DNR communication sheet (Figure 1). This sheet translated the DNR orders into easily understandable declarative statements that communicated (1) whether all or only some components of cardiopulmonary resuscitation were to be withheld for a full cardiopulmonary arrest; (2) whether all, some, or no life-support interventions were to be withheld for life-threatening clinical deterioration other than a full cardiopulmonary arrest; (3) the specific interventions that were permitted or intended to be withheld in the setting of a cardiopulmonary arrest and in the setting of life-threatening clinical deterioration other than a full cardiopulmonary arrest; and (4) whether all medical interventions other than comfort measures were intended to be withheld regardless of the clinical circumstances.
Example of a computer-generated do-not-resuscitate communication sheet. For quick recognition, specific interventions ordered to be withheld are listed in a strike-through font with "withhold" written at the right side of the intervention.
The DNR communication sheet included a definition of cardiopulmonary arrest (an extreme alteration of cardiopulmonary or circulatory function that will result in a patient's death within several minutes unless resuscitative efforts are initiated) and a life-threatening clinical deterioration other than a full cardiopulmonary arrest (less extreme instances of ineffective respiration, cardiac rhythm, or hypotension that may compromise vital organ perfusion and be premonitory of a full cardiopulmonary arrest but by themselves will not cause death within minutes).
The computer printed a cover letter to attending physicians asking them to review the computer communication sheet and to notify the ICU if it did not accurately represent the attending physician's intent. The computer also printed a third sheet that contained excerpts from the hospital's policy for withholding and withdrawing life support that reminded physicians to document the patient's DNR status in the progress notes in the medical record. The cover letter, DNR communication sheet, and policy excerpt sheet were either handed to the attending physicians if they were in the ICU when the DNR order was written or sent by facsimile to the physician's office.
A copy of the DNR communication sheet was also provided to the resident and nurse caring for the patient. A copy of the DNR communication form was placed in a plastic sleeve and attached by a plastic cord to the back of the patient's headboard out of public view where it could be rapidly accessed in an emergency.
Clerks and nurses estimated that an additional 10 minutes were required to access the computer system, send facsimile copies of the DNR communication sheet to physicians, and place materials in the charts and patients' rooms.
Nurses and residents were instructed to refer to the computer communication sheet whenever they needed to communicate a patient's DNR status. They were specifically instructed to review a patient's form during change-of-shift report, whenever they were asked about the patient's DNR status, and as soon as possible during a serious change in the patient's clinical status that may require life-support interventions. Nurses and residents were instructed never to recall from memory a patient's DNR status without referring to the DNR communication sheet regardless of circumstances.
A research nurse trained in nondirective interviewing collected the study data. Within 24 hours after identification of a patient with a new DNR order, the nurse administered a questionnaire to the attending physician, nurse, and resident in charge of the patient's care. The questionnaire assessed the attending physician's intent in writing the DNR order and the nurse's and resident's understanding of the patient's DNR status. Attending physicians, nurses, and residents were asked the following questions: (1) Does the DNR order only apply to a full cardiopulmonary arrest and to no other clinical situation? (2) Does the DNR order withhold all components of advance cardiopulmonary life support including external pacing, electrical cardioversion, bolus epinephrine administration, chest compression, and endotracheal intubation? (3) In addition to withholding some or all components of advance cardiopulmonary life support, does the DNR order also withhold other types of care, such as intravenous vasoactive drugs, increases in infusion rates of existing vasoactive drugs, rapid fluid infusions for sudden hypotension, transfusion of blood products, and/or diagnostic blood drawings? Intertesting variability was monitored and found to be small. The research nurse also observed whether the nurse and resident responded to questions from memory or by retrieving the patient's DNR status from the DNR communication sheet.
Attending physicians, nurses, and residents who cared for patients with DNR orders during period 3 were asked to rate on a 5-point Likert scale the efficacy of the computer-based communication process.
Two-way concordances between attending physicians and residents and between attending physicians and nurses for each of the questionnaire items were determined by the κ statistic expressed as κ ± SE.9 A composite measure of agreement across all questionnaire items was measured by calculation of an overall value of κ by defining agreement as concordance of all 3 items between caregiver groups.9 Comparisons between κ statistics were made with a z test.10 The κ results were analyzed for agreement as follows: less than 0, "poor" agreement; 0 to 0.2, "slight" agreement; 0.2 to 0.4, "fair" agreement; 0.4 to 0.6, "moderate" agreement; 0.6 to 0.8, "substantial" agreement; and 0.8 to 1.0, "almost perfect" agreement.11
Overall 2-way and 3-way proportions of agreement were reported for each questionnaire item and the composite DNR order across questionnaire items.10
Continuous variables with a nonparametric distribution were analyzed by the Mann-Whitney U test. Differences between nominal variables were determined by χ2 or the Fisher exact test if groups had 5 or less responses in a cell. Data were analyzed using the JMP version 3.1 software (JMP Software, Cary, NC) with P<.05 considered statistically significant.
The study was approved by the Institutional Review Board for Human Research. Requirements for written informed consent for the caregiver participants were waived.
Data were analyzed from 147 patients (78 men and 69 women) entered into the study, with 40 patients in period 1, 36 patients in period 2, and 71 patients in period 3. Eighteen patients had been screened but were excluded from analysis because of unavailability of the attending physician for the questionnaire interview. The age ranges of patients were similar between the study periods: period 1, median age was 71 years (range, 18-96 years); period 2, median age was 76 years (range, 19-92 years); and period 3, median age was 72 years (range, 24-97 years).
Eighty-five attending physicians, 91 nurses, and 65 residents cared for the 147 patients. Because not all patients were assigned to house staff, only 113 patients were analyzed for concordance of caregiver combinations that included residents. The study results were representative of the entire caregiver groups' practice patterns as demonstrated by the broad distribution of patients among attending physicians (median of 1 patient per attending physician [middle 80%, 1-4 patients per attending physician]), residents (median 1 patient per attending physician [middle 80%, 1-4 patients per attending physician]), and nurses (median of 1 patient per attending physician [middle 80%, 1-3 patients per attending physician]).
Table 1 displays the 2-way concordances between attending physicians and residents and attending physicians and nurses across each of the 3 questionnaire items. Concordances in period 1 ranged from "fair" to "moderate" agreement. As previously reported,5 the procedure-specific DNR order form in period 2 improved concordances to levels of "moderate," "substantial," and "nearly perfect" agreement across each questionnaire item between all caregiver groups except for attending physician–nurse understanding regarding whether the DNR order only applies to a full cardiopulmonary arrest. The concordance for this item remained "fair" (period 1 κ=0.36 ± 0.14 [67.5% agreement] vs period 2 κ=0.39 ± 0.15 [75.0% agreement]).
The computer-based system and the educational efforts in period 3 resulted in further improvement in concordance in some but not all questionnaire items. Concordance as to whether the DNR order only applied to a full cardiopulmonary arrest increased from "moderate" to "substantial" between attending physicians and residents and from "fair" to "substantial" between attending physicians and nurses. Concordance as to whether other treatments were to be limited for nonarrest events increased from "substantial" to "almost perfect" between attending physicians and nurses. As shown in Table 1, however, some patients continued to be misclassified throughout all study periods with the overall 2-way proportions of agreement ranging from 81.4% to 93.7%.
Table 1 shows the proportions of agreement for the composite of all 3 questionnaire items in 2-way comparisons between caregiver groups. The proportions increased during all 3 study periods but significant differences existed only between periods 1 and 3 for attending physician–resident (P<.001) and attending physician–nurse (P<.001) agreement.
The overall 3-way proportions of agreement (complete concordance) for all 3 components of the DNR order between all 3 caregiver groups increased in each of the 3 study periods, but the differences were statistically significant only for the comparison of period 3 with period 1. In period 1, complete concordance existed for 6 (22%) of 27 patients. This proportion was 11 (48%) of 23 patients in period 2 (P=.06 vs period 1). Complete concordance for period 3 increased to 39 (62%) of 63 patients (P=.24 compared with period 2; P<.001 vs period 1).
Do-not-resuscitate progress notes were considered complete in more patients in period 2 (69.1%) compared with period 1 (47.8%; P<.005) or period 3 (42.3%; P<.005). In period 3, residents accessed the DNR communication sheet for 34 (54%) of 63 patients when asked by the study nurse to respond to the questionnaire items. Nurses accessed the DNR communication sheet for 53 (75%) of 71 patients when asked to define the patients' DNR status. When not referring to the DNR communication sheet, residents and nurses recollected patients' DNR status from memory.
Caregivers interviewed during period 3 had a similarly favorable opinion regarding the efficacy and acceptability of the computer-based system as shown by their responses to the 5-point Likert question (1=low efficacy, 5=high efficacy). Attending physicians had a median response of 4.0 (middle 50%, 3.0-4.5), residents had a median response of 5 (middle 50%, 4.0-5.0), and nurses had a median response of 4 (middle 50%, 3.0-5.0).
During period 3, attending physicians modified their DNR orders for 7 (10%) of 71 patients immediately after receiving a copy of the DNR communication sheet. These physicians stated that the DNR communication sheet allowed them to recognize that they or the residents had incorrectly completed the procedure-specific DNR order form.
Accurate communication among caregivers of a patient's DNR status has assumed greater importance during the last decade. Up to 70% of hospitalized patients have a DNR order written at the time of death12 and, in some centers, 90% of patients dying in the ICU undergo withholding or withdrawal of life-support interventions.1 Combined with the increased frequency of their use, DNR orders have become progressively more complex, restricting some but not all components of cardiopulmonary resuscitation for certain but not other life-threatening events in combinations that vary among patients. Adding further complexity, DNR orders may also withhold various treatment interventions, such as antibiotics or nutrition, that are beyond the scope of cardiopulmonary resuscitative measures.
Our previously reported5 findings from periods 1 and 2, which serve as baseline data for the present study, combined with the observations of others4 demonstrate that only "fair" to "moderate" concordance between caregivers exists in understanding of complex DNR orders when orders are written in unstructured physician order forms. It has been recommended that procedure-specific DNR order forms can improve the communication of complex DNR orders,2 but limited data exist to support this contention. We observed improved concordances with the structured, procedure-specific DNR order form for some but not all components of the DNR order with agreement ranging from "fair" to "almost perfect." However, complete concordance of all components of the DNR order between all 3 categories of caregivers existed for less than 50% of patients.
To further enhance understanding of DNR orders within our ICU, we designed a computer-based communication system and an ongoing educational program that encouraged caregivers to rely on a computer-generated DNR communication sheet to convey patients' DNR status. We found further improvement in concordance of understanding for several but not all of the 3 components of the DNR order between caregiver groups. We also observed in period 3 a significant improvement in the proportion of agreement for the composite DNR order (all 3 components of the DNR order combined) compared with period 1 for attending physicians and residents (period 3 [81.0%] vs period 1 [44.4%]; P<.001) and attending physicians and nurses (period 3 [72.9%] vs period 1 [40.0%]; P<.001). The overall proportions of agreement (complete concordance of all components of the DNR order between all caregiver groups) in period 3 also improved compared with the use of an unstructured order form in period 1 (period 1 [22.2%] vs period 3 [61.9%]; P<.001).
We speculate that the computer-based system improved concordance of understanding between caregivers because it translated the instructions of the procedure-specific DNR order form (a relatively complex input tool for placing information into the medical record) into declarative phrases with interventions intended to be withheld listed with the visual aid of a strike-through line for quick recognition in emergencies (a relatively simple output tool for disseminating information to caregivers). We also suggest that placement of the DNR communication sheet at the head of the bed with copies provided to physicians and nurses on the day the order was written further improved understanding of the order's content.
We had intended, however, to achieve perfect concordance of understanding by training nurses and residents to review the DNR communication sheet whenever a sudden change in their patients' condition occurred and whenever they were queried about their patients' DNR status. We found, however, that residents 46% of the time and nurses 25% of the time recalled from memory their patients' DNR status without referring to the DNR communication sheet. It was more difficult than expected to change the manner by which DNR information was accessed and communicated within the ICU. Although it is suggested that levels of agreement between caregivers interpreting common diagnostic tests should be considered adequate with a κ statistic at or above 0.40,11,13 we believe that perfect agreement should be sought for DNR orders because of their critically important therapeutic implications. Unclear understanding of DNR orders may result in a reductionistic interpretation by bedside caregivers that equates a partial DNR order, which may intend to withhold some but not all life-support measures, with an order for comfort care only.14
Although the computer-based system provided the attending physician with excerpts from the hospital's policy on initiating DNR orders, it failed to improve the frequency by which physicians documented the essential elements of a DNR progress note in the medical record. Guidelines for appropriate use of DNR orders emphasize the importance of documenting in the medical record the basis for the order's implementation.15 Previous investigations have demonstrated the resistance of physicians to interventions designed to improve documentation of the rationale for a DNR status and the physician's discussion of the DNR order with the patient and family.2,8,16
The computer-generated DNR communication sheet provided an unanticipated benefit by assisting physicians in identifying mistakes in DNR order entry. Errors were noted in 10% of the DNR orders during period 3 after attending physicians reviewed the DNR communication sheet. This error rate is considerable considering that the DNR order form had been previously reviewed and signed by physicians in all instances. After noting the errors, attending physicians telephoned the ICU and corrected the DNR order. This observation demonstrates that some physicians may not understand how to complete necessarily complex procedure-specific DNR order forms even though these forms effectively improve communication. In light of this observation, future efficacy studies of DNR order forms should provide data to support their reliability and validity.
Even though the computer-based system added extra steps to establishing a DNR order, caregiver satisfaction with the system was high. Eighty-five percent of caregivers gave the system a "moderately effective" to "highly effective" rating on a 5-point Likert scale; the median evaluation was 4 of 5 for each of the 3 caregiver groups.
The study is limited by its design that examined concordance only for 3 categories of DNR status. Had we measured concordance for all details of the DNR order, such as the specific interventions to be withheld, we anticipate that a lower concordance would have been observed. It is likely, however, that concordance for these specific items would have been greater with the computer-based system compared with an unstructured DNR order form.
An additional limitation of the study is its examination of concordance between caregivers in their understanding of the DNR order without regard to the patients' or families' understanding of the resuscitative treatment plans. Previous studies demonstrate poor communication between physicians and their hospitalized patients during the terminal phases of their lives.12,17- 19 Had we examined concordance between the patients' or families' understanding and expectations of the DNR order and the caregivers' interpretation of the patients' and families' wishes, we anticipate that a lower measured concordance would have been found. Perhaps a DNR communication sheet written in lay terms and provided to patients and their families would aid communication and provide a safeguard against misinterpretation of patients' end-of-life wishes.
In conclusion, DNR orders are often miscommunicated among caregivers but communication can be enhanced with a procedure-specific DNR order form supplemented by a computer-based system and educational intervention. The unique nature and complexity of the DNR order, however, presents hindrances to achieving complete concordance between caregivers. Ongoing monitoring procedures should be established in hospitals to ensure that the bedside caregivers' understanding of a patient's DNR status accurately reflect the wishes and expectations of patients and their families.
Accepted for publication August 7, 1997.
Corresponding author: John E. Heffner, MD, St Joseph's Hospital and Medical Center, 350 W Thomas Rd, Phoenix, AZ 85001 (e-mail: firstname.lastname@example.org).
Thank you for submitting a comment on this article. It will be reviewed by JAMA Internal Medicine editors. You will be notified when your comment has been published. Comments should not exceed 500 words of text and 10 references.
Do not submit personal medical questions or information that could identify a specific patient, questions about a particular case, or general inquiries to an author. Only content that has not been published, posted, or submitted elsewhere should be submitted. By submitting this Comment, you and any coauthors transfer copyright to the journal if your Comment is posted.
* = Required Field
Disclosure of Any Conflicts of Interest*
Indicate all relevant conflicts of interest of each author below, including all relevant financial interests, activities, and relationships within the past 3 years including, but not limited to, employment, affiliation, grants or funding, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. If all authors have none, check "No potential conflicts or relevant financial interests" in the box below. Please also indicate any funding received in support of this work. The information will be posted with your response.
Register and get free email Table of Contents alerts, saved searches, PowerPoint downloads, CME quizzes, and more
Subscribe for full-text access to content from 1998 forward and a host of useful features
Activate your current subscription (AMA members and current subscribers)
Purchase Online Access to this article for 24 hours
Some tools below are only available to our subscribers or users with an online account.
Download citation file:
Web of Science® Times Cited: 12
Customize your page view by dragging & repositioning the boxes below.
and access these and other features:
Enter your username and email address. We'll send you a link to reset your password.
Enter your username and email address. We'll send instructions on how to reset your password to the email address we have on record.
Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.