Accuracy of a Pharmacovigilance Algorithm in Diagnosing Drug Hypersensitivity Reactions FREE

Many people who have experienced a drug hypersensitivity reaction are designated as allergic to the drug without any further investigation. Drug hypersensitivity reactions can indeed be life-threatening,^1- 2 and drug reintroduction in these patients may cause reactions that may be even more severe.^2- 3 On the other hand, overdiagnosis due to common fear of anaphylaxis is frequent, and nonhypersensitive patients do not need to avoid these drugs in the future. It is therefore important to diagnose drug hypersensitivity reactions. Confirmation of the diagnosis should be rigorous and based on clinical history and a physical examination, possibly followed by skin tests and drug provocation tests. The strategy used to confirm the diagnosis of a suspected drug hypersensitivity has been standardized by the European Academy of Allergology and Clinical Immunology.^4- 6 However, this strategy requires sophisticated and potentially dangerous allergy tests. We recently demonstrated that drug provocation tests in individuals with suspected drug allergy performed in carefully controlled settings is cumbersome but can confirm drug hypersensitivity.⁷

Probability scales to assess the relationship (or imputability) between an adverse drug reaction and therapy of a patient are frequently used by the National Committee on Safety of Medicine. Many of these methods of assessment of causality have been published.^8- 12 They use algorithms based on the patient’s clinical history taken either prospectively or retrospectively by medical record review. These methods combine several criteria: chronology and symptoms of the reaction and, in some cases, scientific literature. In France and other European countries, the currently used method was developed in 1978 by Dangoumau et al¹³ and updated in 1985 by Begaud et al.^8- 9 In the United States¹⁰ and Canada,¹¹ other methods are used. These methods are of great importance but, to our knowledge, have never been tested in drug hypersensitivity reactions.

In the present study, we analyzed patients with suspected drug hypersensitivity reactions to β-lactams, nonsteroidal anti-inflammatory drugs (NSAIDs), paracetamol (acetaminophen), local anesthetics, macrolides, quinolones, and some other drugs. We estimated the diagnostic accuracy of a pharmacovigilance algorithm by comparing the probability scores obtained retrospectively by the method of Begaud et al^8- 9 to the results of the drug allergy diagnosis. To improve the evaluation of the drug hypersensitivity diagnosis, we performed a multivariate analysis using a logistic regression model to look for independent clinical risk factors.

Between January 2000 and January 2003, we included all patients with 1 or more clinical histories of drug hypersensitivity who consulted at our allergy clinic (University Hospital of Montpellier, Montpellier, France). According to the referring physicians, the therapy was likely to be responsible for the hypersensitivity reaction, and since their patients needed the drugs in question, allergy testing was required.

We included patients with immediate (occurring within 1 hour after the last administration of the drug) and nonimmediate hypersensitivity reactions that occurred during treatment. As previously described,⁷ we excluded patients who had experienced severe life-threatening skin reactions, drug-induced autoimmune disease, specific organ hypersensitivity manifestations, aspirin-induced asthma, or cardiovascular disease (contraindicating epinephrine therapy); patients who declined the whole drug allergy evaluation; and pregnant women. We also excluded patients with chronic urticaria and latex and food allergy and patients with human immunodeficiency virus (HIV) infection, usually representing a different problem.^7,14 Patients were considered to be atopic when they had at least 2 positive skin prick test results to the common aeroallergens of their living area.

At least 4 to 6 weeks after the resolution of clinical symptoms and at least 4 weeks from any illness, patients underwent a standardized evaluation, which included a standardized questionnaire,⁴ skin tests for β-lactams and local anesthetics,⁵ and provocation tests.^6- 7 Precautions for patients with a history of anaphylaxis are described in detail elsewhere.⁷ The drug allergy testing was the reference standard. For β-lactams, the diagnosis was based on positive skin test results or, when these results were negative, on a positive oral challenge result.^15- 16 For NSAIDs, paracetamol, local anesthetics, macrolides, quinolones, and other drugs, skin tests were not performed because the results are not accurate.⁵ These diagnoses were based on a positive oral challenge result.^6- 7

The following data were collected for the analysis: age, sex, time lapse since the clinical history, time lapse between the last intake of the drug and the reaction, atopy, prescription of multiple drugs during the episode, and possible infection as diagnosed by the prescribing physician. For drug allergy diagnosis,⁷ our institutional policy does not require authorization from an ethics committee but does require the patient's written informed consent, which was obtained for every patient.

In this study, the algorithm of Begaud et al^8- 9 was used in all clinical reactions by trained investigators unaware of the diagnosis of drug hypersensitivity. Data on clinical reactions were collected retrospectively.

The algorithm calculates the relationship between an adverse effect and the patient’s therapy. This is performed independently for each drug taken. Drug combination is not taken into account. This assessment of causality (or imputability) depends on 7 criteria that are divided into 2 groups: chronology and symptoms or signs. The appreciation of the results does not involve knowledge from the literature. The criteria of chronology (Table 1) include time lapse between the administration of the drug and the reaction and the effect of drug cessation and drug reintroduction. The combination of these 3 criteria makes up an intermediate score of chronology (C) with 4 possible results (C3, suggestive chronology; C2, possible; C1, dubious; and C0, incompatible). The clinical criteria (Table 2) involve symptoms, possible contributing factors (if validated), another possible differential diagnosis, and results of reliable and specific laboratory tests. The results of the combination of these 4 criteria also constitute an intermediate symptom score (S) with only 3 possible results (S3, suggestive history; S2, possible; and S1, dubious). The crossing of these scores results in 5 final possible imputability (I) scores (Table 3): I0 (unlikely imputability), I1 (dubious imputability), I2 (possible imputability), I3 (likely imputability), and I4 (very likely imputability).

The imputability method of Begaud et al considers each drug as an individual case. It defines reintroduction as the intake of the same drug using the same doses and in the same conditions as the initial administration. Consequently, the reintroduction of the same drug or of a drug of the same class but in different conditions is not considered a real reintroduction. By contrast, the effect of this type of reintroduction is considered to be important by the allergologist. All of the reactions in our study were thus reanalyzed using the method of Begaud et al.^8- 9 The reintroduction of the same drug or of another drug of the same therapeutic class was considered a real reintroduction.

For each of the qualitative variables, we calculated the frequency. For each of the quantitative variables, we provided the interquartile range.

The statistical link between 2 qualitative variables was performed with the χ² or the Fisher exact test for small sample sizes. For continuous variables, comparisons were performed with the Mann-Whitney nonparametric test because of nonnormal distribution. The diagnostic value of the algorithm was calculated in terms of sensitivity, specificity, and predictive values, with approximate 95% confidence intervals (CIs). To do this, the 5 conclusions of the algorithm of Begaud et al^8- 9 were arbitrarily pooled into 2 classes (I0 + I1 vs I3; I2 and I4 were never found in our study). Sensitivity was defined as the probability of being I3 in hypersensitivity reactions and specificity as the probability of being I0 or I1 in nonhypersensitivity reactions. The positive predictive value was defined as the probability that an I3 reaction is a hypersensitivity reaction, and the negative predictive value was defined as the probability that an I0 or I1 reaction was a nonhypersensitivity reaction.

To appreciate independent clinical risk factors linked to the hypersensitivity diagnosis, a multivariate analysis was performed using a logistic regression model with a stepwise selection. This enabled us to adjust the odds ratio (OR) and to establish 95% CIs. The adequacy of the model was checked by the percentage of concordance between predicted probabilities and observed responses and by the Hosmer-Lemeshow test.¹⁷ The statistical analysis was performed with the help of SAS software, version 8 (SAS Institute Inc, Cary, NC).

During the study period, 677 patients with suspected drug allergy were included (Table 4). Forty-one patients were excluded: 27 because of a clinical history incompatible with a drug hypersensitivity reaction, 6 because of HIV, 2 because of chronic urticaria, 4 because of food or latex allergy, and 2 because of severe cutaneous reactions. The included patients consisted of 229 men (33.8%) and 448 women (66.2%). Their mean ± SD age was 36.5 ± 20.5 years. A total of 226 (33.4%) were atopic. There were 221 patients who had more than 1 suspected drug hypersensitivity reaction (1001 suspected reactions in 677 patients). A total of 220 patients experienced more than 2 hypersensitivity reactions. Each clinical reaction was analyzed as a separate event. A total of 611 clinical reactions (61%) involved more than 1 drug, but the drug allergy testing discriminated hypersensitivity from nonhypersensitivity reactions.

Thus, 1001 hypersensitivity reactions were analyzed (Table 4): 408 involving β-lactams (40.7%), 291 NSAIDs (29.1%), 112 paracetamol (11.2%), 38 local anesthetics (3.8%), 71 macrolides (7.1%), 27 quinolones (2.7%), and a few other drugs (5.4%). A total of 641 (64%) of the reactions occurred while the patients were treated for an infection (mostly ear, nose, and throat infection). There were 189 immediate reactions (18.9%) (ie, arising less than 60 minutes after the last drug intake) and 608 nonimmediate reactions (60.8%). For the remaining 204 clinical histories (20.4%), the chronology of the reaction was imprecise. The different symptoms observed during these reactions are given in Table 4.

In 175 clinical reactions (17.5%), a diagnosis of drug hypersensitivity was confirmed. For the other reactions, the responsibility of the drug was eliminated. When the reactions with or without a diagnosis of drug hypersensitivity were compared, no significant difference occurred in terms of sex, age, atopy status, number of drugs administered, infection, and time delay between the reaction and the drug allergy testing (Table 4). However, significant differences occurred for the clinical delay and presentations: proven drug hypersensitivity reactions were more likely to be immediate reactions, such as anaphylactic shock and angioedema (P<.001), than reactions for which the responsibility of the drug was eliminated.

In 204 reactions (20.4%), the imputability method of Begaud et al^8- 9 was not attributable because of the absence of 1 of the criteria required for scoring (Table 5). Forty-five reactions (4.5%) were given an I0 score (unlikely imputability), 720 (71.9%) an I1 score (dubious imputability), and 32 (3.2%) an I3 score (likely imputability). The reactions that scored I3 were always immediate. Sensitivity of the algorithm was 10.3% (95% CI, 5.8%-14.8%) and specificity was 76.9% (95% CI, 74.0%-79.8%). When the “unknown scores” were excluded, sensitivity and specificity were 12.0% (95% CI, 6.8%-17.2%) and 97.8% (95% CI, 96.7%-98.9%), respectively. The positive predictive value was 56.2% (95% CI, 39.0%-73.4%), and the negative predictive value was 83.0% (95% CI, 80.3%-85.7%). Although there were 1.7% false-positive scores (ie, I3 for nonhypersensitivity reactions), there were no false-negative scores (ie, I0, unlikely imputability for true hypersensitivity reactions).

The clinical reactions for which an imputability score could be given (797) were scored once again considering the effect of a possible reintroduction of any drug of the same class (Table 5, step 2). This notion of drug reintroduction concerned 46 reactions, which were upgraded from an I1 to an I3 score (14 hypersensitivity reactions and 32 nonhypersensitivity reactions), given 3.9% of the false-positive results. False-negative results did not change. Corresponding sensitivity, specificity, and positive and negative predictive values were 26.3% (95% CI, 9.8%-32.8%), 74.7% (95% CI, 71.7%-77.7%), 58.9% (95% CI, 48.0%-69.8%), and 85.8% (95% CI, 83.2%-88.4%), respectively. When the unknown scores were excluded, sensitivity and specificity were 31.0% (95% CI, 23.5%-38.5%) and 95.0% (95% CI, 93.3%-96.7%), respectively. The algorithm’s performance for clinical histories of the β-lactam group did not differ from that of the NSAID group: sensitivities were 11.3% (95% CI, 3.4%-19.2%) and 14.3% (95% CI, 5.1%-23.5%), respectively, and specificities were 98.7% (95% CI, 97.2%-100%) and 98.5% (95% CI, 96.8%-100%), respectively, when the unknown scores were excluded.

A multivariate analysis was performed using a logistic regression model to look for independent clinical risk factors linked to the hypersensitivity diagnosis. The variables of the clinical reactions included in the multiple regression model were immediate reaction (yes vs others), reintroduction of the drug (yes vs others), Begaud et al score (I3 vs others), and polymedication (yes vs no). Two interactions terms were tested (I3 score and immediate reaction and I3 score and reintroduction) and found not to be statistically significant. The analysis found 3 parameters linked to drug hypersensitivity: I3 likely score (OR, 4.2; 95% CI, 1.9-9.3; P<.001), reintroduction in clinical history (OR, 2.8; 95% CI, 1.5-5.4; P=.001), and delay between reaction and last drug intake of less than 1 hour (OR, 1.6; 95% CI, 1.0-2.5; P=.03).

The aim of the present study was to determine whether reliable diagnoses of drug hypersensitivity reaction could be made using pharmacologic algorithms with clinical history. If this was possible, then drug allergy diagnosis would become unnecessary. We used drug allergy testing, including drug provocations, as a reference standard.⁷ We followed the Standards for Reporting of Diagnostic Accuracy statement for the reporting of studies of diagnostic accuracy.¹⁸ Because of the need for a definite diagnosis for the comparison, we had to exclude severe life-threatening skin reactions from this study, where, owing to ethical reasons, the drug could not be reintroduced. Even though drug allergy diagnosis is not easy, drug provocation is considered to be the most appropriate method when skin tests are not available or not validated.^6- 7,15 However, it is always possible that some patients with mild sensitivity or a long delay between testing and drug hypersensitivity reaction may have been missed. As we previously demonstrated,⁷ drug hypersensitivity reactions were confirmed in a few patients (17.5%) with a history suggestive of possible drug allergy. However, patients with mild reactions could possibly experience a more severe reaction if the drug was to be reintroduced. It is therefore important to diagnose these patients’ conditions.

We found that many clinical histories (20.4%) could not have a probability score because of a lack of criteria. It is possible that the lack of clinical information for these cases was related to the clinicians being able to refer patients for drug allergy testing. The causality assessment method of Begaud et al^8- 9 is not accurate for the diagnosis of drug hypersensitivity reactions and cannot replace drug allergy testing. Although the algorithm did not enable us to distinguish true drug hypersensitivity reactions from nonhypersensitivity reactions in most cases, it found a few false-positive results (1.7%) and no false-negative results. The positive predictive value (56.2%) and the negative predictive value (83%) may allow, in some situations, elimination of a drug allergy evaluation. The multivariate analysis found parameters linked to drug hypersensitivity: I3 (likely imputability) (OR, 4.1), reintroduction in clinical history (OR, 2.8), and immediate delay (OR, 1.5). When a complete drug allergy diagnosis is not possible, their consideration is crucial.

Nevertheless, routine causality assessment methods performed by pharmacologists are not designed to make definite decisions but rather to classify adverse drug reactions on the basis of the likelihood of a causal relationship between the drug and the reaction. Their use is essential for the filing of reports and for facilitating database analyses.¹⁹ Although there are few differences among the different available algorithms, we have previously demonstrated that the imputability method of Begaud et al^8- 9 is more efficient than the imputability methods of Jones¹⁰ and Naranjo et al¹¹ with regard to the diagnosis of drug hypersensitivity reactions (S.B. and P.D., unpublished data, 2005).

From a clinical perspective, the 2 main pitfalls of causality assessment methods are for the reactions where a score cannot be given because of a lack of sufficient information (more than 20%) and for the files without exploitable conclusions (more than 70% of dubious imputability). The only solution for the first pitfall (which is common daily practice) is drug allergy testing. The second pitfall could be improved by a more serious consideration of the clinical history parameters (which we have shown to be of importance in the final diagnosis) and notably the immediate chronology and reintroduction of a drug of the same class. Nevertheless, drug allergy diagnosis remains crucial. From a pharmacovigilance perspective, we have suggested a possibility of improving the French causality assessment method by taking into account the reintroduction of a drug of the same class.

Correspondence: Pascal Demoly, MD, PhD, Exploration des Allergies, Maladies Respiratoires, Hôpital Arnaud de Villeneuve, University Hospital of Montpellier, 34295 Montpellier CEDEX 5, France (demoly@montp.inserm.fr).

Accepted for Publication: March 6, 2005.

Financial Disclosure: None.

Funding/Support: This study was supported by an institutional grant (Projet Hospitalier de Recherche Clinique 7679) from the University Hospital of Montpellier.

Acknowledgment: We thank Anna Bedbrook, BSc, for English correction.