Development and Validation of a Score to Assess Risk of Adverse Drug Reactions Among In-Hospital Patients 65 Years or Older:  The GerontoNet ADR Risk Score FREE

In Western countries, drug-related illnesses are an important medical problem, resulting in 3% to 5% of all hospital admissions, accounting for 5% to 10% of in-hospital costs, and being associated with a substantial increase in morbidity and mortality.^1- 8 Older patients are particularly vulnerable to drug-related illnesses because they are usually on multiple drug regimens, which expose them to the risk of drug interactions,⁹ and because age is associated with changes in pharmacokinetics and pharmacodynamics.¹⁰

One potential strategy for preventing adverse drug reactions (ADRs) is to identify those patients who are at risk of an ADR and to target additional resources toward this group. An example of this approach might be that when a patient is identified as being at risk, the physician and the pharmacist could pay extra attention to all the medications that he or she receives. Several patient characteristics that may make an ADR more likely, including age, number of drugs that the patient is receiving, alcohol use, comorbidity, and factors that alter drug distribution or metabolism, such as renal or hepatic insufficiency, heart failure, and anemia, have been suggested in various studies.^1,11- 15 Although identification and quantification of risk factors for iatrogenic illness are judged to be a public health priority,¹⁶ to our knowledge there are no empirical data that allow stratification of patients according to likelihood of an ADR. Indeed, in the care of older patients, numerous scales are used to identify risks (eg, cardiovascular), disorders (eg, depression), and dysfunctions (eg, cognitive problems and disability in activities of daily living or instrumental activities of daily living), but a scale to detect those patients who are at risk for an ADR is not yet available.

Hospitalized older adults are usually “frail” and present with acute diseases, which may increase their susceptibility to ADRs and intensify the severity of drug-related illnesses.¹ Moreover, in-hospital patients, who often have a genuine need for many drugs, are usually the victims of a “prescribing cascade” that leads to an increased likelihood of ADRs.¹⁷ Also, because of these complexities in prescribing, older adults often receive inappropriate drugs whose risks outweigh the benefits.¹⁸

The hospital is an ideal setting to study ADRs because the pharmacological noncompliance is reduced and the daily evaluation of patients, as well as the constant review of charts and medical records, provides an opportunity for careful reporting of all suspected ADRs. This opportunity makes the in-hospital population a perfect group to study ADRs and to develop a score to assess the risk of drug-related illness.

Therefore, based on these considerations, the objective of the present study was to develop and validate a practical, efficient, and simple method of identifying patients who are at increased risk of an ADR in a population of in-hospital older adults. To address this issue, a score was developed based on data from the medical literature and on secondary analysis of the database of the Gruppo Italiano di Farmacoepidemiologia nell’Anziano (GIFA) (Italian Group of Pharmacoepidemiology in the Elderly), a study that was specifically designed to collect data about ADRs among in-hospital patients in Italy. Then, this score was validated in a population of older adults consecutively admitted to 4 university hospitals in Europe by geriatricians who are members of the GerontoNet group, a network of academic departments of geriatric medicine in the European Union.¹⁹

The GIFA comprises a group of investigators who operate in community and university-based hospitals throughout Italy. As already described in detail elsewhere,^1,20 after obtaining written informed consent, a study physician with specific training completed a questionnaire for each patient on admission to the hospital and updated it daily. The study periods were May 1 to June 30, 1988, and September 1 to December 31, 1988; May 15 to June 15, 1991; and May 1 to June 30 and September 1 to October 31 in 1993, 1995, 1997, and 1998. Data recorded included sociodemographic characteristics, indicators of physical function and cognitive status, clinical diagnoses at admission and at discharge, medications taken before admission and during hospital stay, and medications prescribed at discharge.

An ADR was defined as any noxious, unintended, and undesired effect of a drug, excluding therapeutic failures, intentional and accidental poisoning, and drug abuse.²¹ Each ward participating in the study was visited daily by a study physician who personally asked the nurses and the attending physicians about any possible ADR that had occurred in the preceding 24 hours. The medical and nursing records were also carefully examined daily. Study physicians were instructed to consider any new clinical event as a potential ADR. For each suspected ADR, the study physician coded clinical description, severity, and eventual outcome. Also, he or she collected detailed information about the drug(s) identified as the potential culprit. The causality of the relationship between drug use and ADR was assessed based on the scores of the Naranjo algorithm.²² The ADRs were classified as definite (score, 9-12 points), probable (score, 5-8 points), possible (score, 1-4 points), or doubtful (score, 0 point). Only definite and probable ADRs occurring during hospital stay were considered for this study. The ADRs that were observed at hospital admission or that caused hospital admission were excluded. The ADR was rated as severe if it caused discontinuation of treatment with the suspected drug, led to the administration of an antidote medication, or caused the death of the patient.

Drugs were coded according to the Anatomical Therapeutic and Chemical codes.²³ Only drugs used during hospital stay were considered for the present study. Clinical diagnoses were recorded by the study physicians, who gathered information from the patients and the attending physicians and carefully reviewed charts, x-ray films, laboratory parameters, medical histories, and clinical documentation. Diagnoses were coded according to the International Classification of Diseases, Ninth Revision, Clinical Modification, codes.²⁴ Anemia was defined by the World Health Organization criteria as follows: hemoglobin concentration below 12.0 g/dL (to convert to grams per liter, multiply by 10.0) in women and below 13.0 g/dL in men.²⁵ The glomerular filtration rate was computed using the Modification of Diet and Renal Disease Study formula²⁶:

170 × (Serum Creatinine)^–0.999 × (Age)^–0.176 × (Serum Urea Nitrogen)^–0.170 × (Serum Albumin)^0.318

For women, the result was multiplied by 0.762. A glomerular filtration rate of less than 60 mL/min was used to define renal failure. Nutritional status was assessed by measuring body mass index and serum albumin levels. Activity of daily living disability was defined as a need for assistance to perform more than 1 of the following tasks: eating, dressing, bathing, transferring, and toileting. Falls were defined as an unintentional change in position from a lying, sitting, or standing position, resulting in the patient coming to rest on the ground or other lower level. A history of ADRs was defined as any ADR that occurred in the past.

For the present analysis, we considered only patients 65 years or older who received at least 1 medication during hospital stay; who had been admitted during the 1993, 1995, and 1997 survey periods; and for whom complete data on predicting variables were available (n = 5997). In the 1988, 1991, and 1998 surveys, data on 1 or more variables were not collected, and for this reason, patients enrolled in these periods were not included in the present analysis. From this population, we excluded 61 participants who were receiving anticancer medications, resulting in a final sample of 5936 participants.

The baseline characteristics of the study participants, according to presence of ADRs, were compared using a χ² test. Variables associated with an ADR at P ≤ .10 in the univariate analyses were entered into a stepwise logistic regression model. The stepwise procedure added and retained independent variables to the model if they were significant at P ≤ .10. Variables retained in the final model were used to compute the ADR risk score. A score of 1 was assigned to variables associated with an ADR with an odds ratio (OR) between 1.00 and 1.99; a score of 2, to those with an OR between 2.00 and 2.99; a score of 3, to those with an OR between 3.00 and 3.99; and a score of 4, to those with an OR of 4.00 or more. The ADR risk score was computed based on the sum of scores of individual variables. To evaluate the predictive ability of the ADR risk score, receiver operator characteristic (ROC) curves were constructed, and areas under the curve (AUC) were calculated. Analyses were performed with SPSS version 16.0 (SPSS Inc, Chicago, Illinois).

The score developed in the GIFA sample was applied for validation among 483 in-hospital older adults who were admitted to 4 geriatric and internal medicine wards participating in the study (Catholic University of the Sacred Heart, Rome, Italy; Ghent University Hospital, Ghent, Belgium; Brighton and Sussex Hospitals Trust, Brighton, England; and Erasmus University Medical Center, Rotterdam, the Netherlands).

Patients who were admitted to participating wards between September 2008 and December 2008 were enrolled and followed up until discharge. Exclusion criteria were age younger than 65 years and an unwillingness to participate in the study. For each participant, a questionnaire was completed at admission and updated daily by a study physician who received specific training. Data recorded included sociodemographic characteristics, indicators of physical function and cognitive status, clinical diagnoses at admission and at discharge, medications taken before admission and during hospital stay, medications prescribed at discharge, and biochemical and blood parameters. Data on ADRs were collected using the methods already described for the GIFA study (see above). Only definite and probable ADRs that occurred during hospital stay were considered for this study. Any ADRs that were observed at hospital admission or that caused hospital admission were excluded from the study. Study physicians who were responsible for collecting data were not informed regarding variables that were entered into the ADR risk score. In this sample, ROC analysis was performed to assess the ability of the risk score to predict ADRs.

The mean (SD) age of 5936 patients who participated in the GIFA study was 78.0 (7.2) years, and the mean (SD) number of drugs that were used during the patients' hospital stay was 6.3 (3.6). Overall, 383 patients (6.5%) experienced an ADR defined as probable or definite based on the Naranjo algorithm during hospital stay; 345 participants experienced only 1 ADR (90.1% of all ADRs); 30 (7.8%), 2 ADRs; and 8 (2.1%), 3 or more. Overall, cardiovascular and arrhythmic complications (n = 97; 25.3% of all ADRs) were the most frequent ADRs, followed by gastrointestinal (n = 69; 18%), neurologic and neuropsychiatric (n = 68; 17.8%), electrolytic (n = 50; 13.1%), and dermatologic/allergic (n = 45; 11.7%) complications. The ADRs were rated as severe in 221 cases (64% of all ADRs).

As shown in Table 1, at the univariate analysis the presence of 4 or more comorbid conditions, renal failure, heart failure, liver disease, depression, number of drugs, and history of an ADR were associated with increased risk of ADRs, with P ≤ .10. Among these variables, the number of drugs, history of an ADR, presence of 4 or more comorbid conditions, heart failure, liver disease, and renal failure were retained in the stepwise regression model and used to compute the score (Table 2). The use of 8 or more drugs was the variable most strongly associated with ADRs, and it was scored as 4 points, followed by a history of an ADR, which was scored as 2 points. All other variables received a score of 1 point. The range of the score was from 0 to 10; the mean (SD) was 3.2 (2.2); the median was 3; and the interquartile range was 1-5. When we repeated the analysis considering only severe ADRs, we observed that the number of drugs was still the strongest predictor of an ADR (5-7 vs <5 drugs: OR, 1.58; 95% CI, 0.99-2.52; ≥8 vs <5 drugs: OR, 4.09; 95% CI, 2.65-6.31), followed by history of an ADR (OR, 2.18; 95% CI, 1.50-3.16), presence of 4 or more comorbid conditions (OR, 1.72; 95% CI, 1.26-2.35), heart failure (OR, 1.61; 95% CI, 1.17-2.23), and liver disease (OR, 1.32; 95% CI, 0.95-1.82), while the variable for renal failure was no longer retained in the model.

The Figure (A) shows that rate of ADRs progressively increased as the ADR risk score increased, going from 2.0% among patients with a score of 0 to 1 to 21.7% among those with a score of 8 or more. The area under the ROC curve, which assesses the ability of the risk score to predict ADRs in the whole population, was 0.71 (95% CI, 0.68-0.73), and this result was substantially confirmed when only severe ADRs were considered (AUC, 0.73; 95% CI, 0.69-0.76) and after stratification of the study population by year of survey (1993 AUC, 0.72; 95% CI 0.68-0.75; 1995 AUC, 0.68; 95% CI, 0.63-0.74; and 1997 AUC, 0.72; 95% CI, 0.67-0.78). A cut point between 3 and 4 seemed to have a good balance between sensitivity (68%) and specificity (65%) and could be used to identify patients who are at high risk for ADRs.

The age of 483 patients entering the validation study was 80.3 (7.6) years; the number of drugs that were used during the hospital stay was 11.0 (7.0); and an ADR was observed in 56 participants (11.6%). Table 3 presents the distribution of variables that were included in the score according to the ADRs in this population. With the exception of the presence of heart failure, all the variables in the score were associated with an increased rate of ADRs. As shown in the Figure (B), an ADR was observed in 2 of 44 participants (4.5%), with an ADR risk score of 0 to 1; in 3 of 72 participants (4.2%) with a score of 2 to 3; in 10 of 143 participants (7.0%) with a score of 4 to 5; in 15 of 131 participants (11.5%) with a score of 67; and in 26 of 93 participants (28%) with a score of 8 or more. The AUC, which assesses the ability of the risk score to predict ADRs in this sample, was 0.70 (95% CI, 0.63-0.78).

The present study proposes a practical and simple method of identifying patients who are at increased risk for an ADR. To our knowledge, this approach has never been adopted before in the field of ADRs, and it may be useful in clinical practice as a tool to identify patients who are at risk and in research to target a population that can benefit from interventions that are aimed to reduce drug-related illness.

In a previous study, using the following 2 approaches Bates et al¹³ tried to develop a risk stratification model for patients who are likely to experience an adverse drug event: (1) a cohort analysis that uses limited information that is readily available electronically and (2) a case-control study. In their study, the authors identified a few independent predictors of ADRs, which had relatively little power, making their attempt to develop a risk score unsuccessful. More recently, Johnston et al²⁷ tried to identify specific patient characteristics that are associated with an increased risk for the development of an ADR or a medication error. They showed that age, clinical diagnoses, admission sources, types of insurance, and the use of specific medications or medication classes were associated with outcome. However, their study was retrospective and relied on voluntarily reported ADRs, which may have resulted in underreporting. Other studies have tried to identify risk factors for ADRs among in-hospital patients and to develop a risk score, but they were limited by a small sample size or were not specifically focused on older adults.^28- 29

In the present study, we identified several risk factors for the development of ADRs. First, our study concurs with previous findings showing that the absolute number of concurrently used drugs is the strongest risk factor for the development of ADRs.^{1,3- 5,10- 14} Older patients may, of course, have a genuine need for more drugs, and the coadministration of multiple drugs can lead to drug-drug interactions, contributing to an increased rate of ADRs.¹⁷ Noticeably, few older patients who take multiple drugs are included in pharmacological trials; therefore, the safety profile of many drugs in an older frail population, especially when used in combination, is still debated. Also, older patients who have a valid need for more medications are often victims of a “prescribing cascade” that frequently includes inappropriate medications, which may play a relevant role in the development of ADRs.^17- 18 Furthermore, a history of an ADR is also a strong risk factor for drug-related illnesses, suggesting that there is a group of patients who are more susceptible to the negative effects of drugs because of ethnic, genetic, or cultural factors.³⁰

In this study, we have chosen not to assess the risk for ADRs according to individual drug classes, because patterns of drug use may change across settings and over time. Also, the distinction of drugs in “classes” is made difficult because drugs with similar therapeutic effects may have different safety profiles. The increased rate of ADRs observed among patients with comorbidities may be related to drug-disease interactions: drugs that are indicated for a certain disease may worsen or exacerbate another condition. Furthermore, several conditions, such as heart failure or hepatic disease, may alter drug distribution and metabolism, leading to an increased rate of ADRs.^31- 32 The association between renal function and iatrogenic illness is well documented,^33- 34 but we found that impaired renal function had only a borderline association with ADRs in the GIFA sample. One possible explanation may be that, while drug metabolism is clearly altered by impaired renal function, the drugs and the dosages used in older patients are adjusted to account for this condition most of the time.

We also did not find any association between age and ADR risk. Indeed, advancing age is associated with an increased incidence of diseases and with polypharmacy, and, as a result, older patients are more prone to experience ADRs. Therefore, the association between advancing age and ADRs that has been described in previous studies is probably not a direct one but is most likely mediated by these factors.^35- 36 In addition, we studied a population of patients who were 65 years or older; therefore, we were not able to assess differences in the risk for ADRs compared with younger populations.

Most of the risk factors for ADRs identified in this study were already assessed in other populations, but, to our knowledge, this is the first study to combine these variables in a score with the aim of identifying high-risk patients who can benefit from specific interventions that are aimed to reduce drug-related illness. The GIFA is one of the largest studies to examine ADRs in older adults with a dedicated database. The assessment of ADRs was done by study personnel during hospital stay, allowing for a correct evaluation of the clinical presentation. An important limitation of the study relates to the fact that data on the preventability of ADRs were not collected. Indeed, the identification of risk factors for preventable ADRs may be particularly important to target a high-risk population that can benefit from interventions that are aimed to reduce drug-related illnesses. Another limitation is represented by the generalizability of the results. Our findings, which are based on a hospitalized population 65 years or older, cannot be extrapolated to younger persons who are living in the community. Therefore, further studies are needed to validate the ADR risk score in other countries and settings. Also, the prescribing patterns are different in various countries, as is the epidemiology of disease burden. The data that we used to develop the risk score were collected between 1993 and 1997. In past decades, the medical complexity of older adults who are admitted to the hospital and the awareness of the treatment of the elderly population, including methods of managing acute geriatric problems, have increased.

Therefore, a validation study was performed in a sample of older adults who were hospitalized in 4 academic hospitals in Europe in 2008. Participants in the validation study were sicker compared with those in the GIFA sample: they were older, had more comorbid conditions, used more drugs, and had a higher rate of ADRs. However, the predictive ability of the score in both samples was similar, suggesting that it is equally effective in predicting ADRs in populations with different characteristics. Heart failure was the only variable in the risk score that was not associated with an increased risk for ADRs in the validation study. This result may be related to the small sample size and the reduced power of this study.

In the past 10 years, the incidence of ADRs in older hospital patients has not decreased, indicating that there is a problem of identification of those at risk. Although geriatricians are now more aware of the adverse effects of medication and of drug interactions, and pharmacists who are trained in elderly care are now available in various geriatric centers, these changes will need to be implemented in the broad hospital setting to have an impact on the incidence of ADRs in hospitals. The ADR risk score is likely to contribute to a better identification of ADRs and those at risk for ADRs.

The present study proposes a practical, efficient, and simple method of identifying patients who are at increased risk for an ADR and who may represent a target for interventions aimed at reducing drug-related illnesses. Further studies are needed to validate this tool in different populations and settings.

Correspondence: Graziano Onder, MD, PhD, Centro Medicina dell'Invecchiamento, Università Cattolica del Sacro Cuore–Policlinico A. Gemelli, Largo Francesco Vito 1, 00168 Roma, Italy (graziano_onder@rm.unicatt.it).

Accepted for Publication: December 9, 2009.

Author Contributions: All authors, external and internal, had full access to all the data (including statistical reports and tables) in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Onder, Petrovic, Tangiisuran, Rajkumar, Bernabei, and van der Cammen. Acquisition of data: Petrovic, Tangiisuran, Meinardi, Markito-Notenboom, Somers, Rajkumar, and van der Cammen. Analysis and interpretation of data: Onder, Petrovic, Tangiisuran, Somers, Rajkumar, and van der Cammen. Drafting of the manuscript: Onder, Tangiisuran, Meinardi, Markito-Notenboom, and Rajkumar. Critical revision of the manuscript for important intellectual content: Petrovic, Tangiisuran, Somers, Rajkumar, Bernabei, and van der Cammen. Statistical analysis: Onder. Obtained funding: Onder, Petrovic, and Rajkumar. Administrative, technical, and material support: Onder, Tangiisuran, Meinardi, Markito-Notenboom, Somers, and Rajkumar. Study supervision: Onder, Petrovic, Tangiisuran, Rajkumar, Bernabei, and van der Cammen.

Financial Disclosure: None reported.

Funding/Support: This study was funded by a grant from the GerontoNet Group, a network of academic departments of geriatric medicine in the European Union, supported by Servier.

Role of the Sponsors: The sponsors had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.