Author Affiliations: Laboratory Medicine, University Hospitals Leuven, Leuven, Belgium.
The accuracy of a diagnostic test is traditionally reported in terms of sensitivity and specificity and predictive values in scientific articles, systematic reviews, and test inserts. Physicians tend, however, to confuse sensitivity and specificity, which are independent of the prevalence, with positive and negative predictive values, which are dependent on the prevalence.1
An alternative way of expressing diagnostic accuracy is the likelihood ratio (LR).2 Likelihood ratios offer the advantage that they are not dependent on the prevalence of the disease, in contrast to positive and negative predictive values, and can be used for nondichotomous test results.3- 5 Steurer and coworkers1 found that reporting diagnostic accuracy as LRs expressed in nontechnical language is more intelligible to clinicians, although a subsequent randomized trial by the same group found no difference in posttest probability estimates between LRs in nontechnical language and sensitivity and specificity.6
Most practicing physicians do not use the Bayesian approach to calculate posttest probability because they consider it impractical and difficult to calculate.7 The use of a graphic representation of the posttest probability as a function of the pretest probability could overcome these problems. This approach has, however, not been studied. While such approach is ill suited for printed laboratory reports, the current advances in information technology allow laboratories to provide this graphic representation “one mouse click away” to clinicians when results are checked electronically.
We developed a 3-item questionnaire. Each question asked the respondent to estimate the posttest probability and included a pretest probability in the form of a prevalence and results of diagnostic accuracy. Depending on the question, the diagnostic accuracy was presented as sensitivity and specificity, LR in nontechnical terms, or graphically. The questionnaire, in English translation, is shown in Figure 1.
The questionnaire and the answers by clinicians (given as number and percentage of responses) to questions 1, 2, and 3.
The questionnaire was distributed to 117 general practitioners and 55 specialists in internal medicine, including trainees, at 2 separate conferences on continuing medical education. The program did not include any specific information regarding the performance of diagnostic tests. The chairman asked participants to complete the questionnaire during the small breaks between the speakers. The questionnaires were collected immediately after the last speaker of the session.
Of the 172 eligible clinicians, 123 filled in the questionnaire (response rate, 72%). The clinicians answering the questions had on average more than 25 years of experience as a physician and had on average more than 40 patient contact hours per week. The correct answer to question 1 was “approximately 20%” (19.6% exactly). The second question described the same test as the first question, but the diagnostic accuracy information was described as an LR in nontechnical language (Figure 1). Since the LR equals sensitivity/(1 − specificity), the LR for the first test was 9.5. The correct answer for question 2 was also “approximately 20%” (20.4% exactly).
The third question provided a graphic representation of the posttest probability as a function of the pretest probability. The test depicted was relatively similar to the tests in the 2 previous questions, with an LR of 12.4 for a positive result. Given the higher prevalence of 20%, the correct answer was “approximately 80%” (75.5% exactly). The test depicted in question 3 is an IgA antitissue transglutaminase assay used in our laboratory (diagnostic accuracy in routine clinical practice).
The proportion of physicians who gave a correct answer was calculated for each of the 3 questions. Physicians who indicated that they did not know the answer to a particular question were not included for this calculation. A Fisher exact test was used to evaluate significant differences in proportions. Calculations were performed using GraphPad QuickCalcs (http://www.graphpad.com) from GraphPad Software (GraphPad Software Inc, La Jolla, California).
Most clinicians grossly overestimated the probability of disease when the diagnostic accuracy information was given as sensitivity and specificity (question 1). Most clinicians (81%) indicated that the probability of disease was approximately 90%. The response by younger (≤40 years old) and older physicians (≥41 years old) was comparable (7% vs 8% correct answers, respectively).
When the diagnostic accuracy information of the same test was provided as an LR in nontechnical terms (question 2), the number of correct answers increased significantly from 8% to 34% (P < .001; Figure 1). Of the 93 clinicians who answered questions 1 and 2, 43 (46%) gave the same estimate twice, 41 (44%) gave a lower estimate for question 2 than for question 1, and only 9 (10%) gave a higher estimate for question 2 than for question 1.
With a graphic approach (question 3), 73% of the responding clinicians gave the correct answer (P < .001 vs questions 1 and 2). The number of nonresponders for question 3 (38%) was higher than for question 1 (15%) (P = .002) and question 2 (22%) (P = .03).
Our study demonstrates that a graphic approach is the most efficient way to convey diagnostic accuracy information to clinicians. The lack of familiarity with a graphic approach, indicated by the higher proportion of persons who could not answer question 3 compared with question 1, is hardly surprising, since previous studies found that most clinicians also lack familiarity with LRs.7 Because the graphic approach does not require calculations to estimate the posttest probability, in contrast to the use of sensitivity and specificity or LRs, we believe that the graphic approach is much easier to explain to clinicians (Figure 2). Since most physicians in industrialized countries consult their laboratory results electronically, a graphic representation could be provided “one mouse click away” to clinicians.
A graphic depiction of pretest vs posttest probability of the disease. Using this type of graph, one can calculate the posttest probability of the disease for any given pretest probability for a specific laboratory test. For example, if the pretest probability is 10% (the patient has a 10% chance of having the target disorder) and the depicted test result is positive, then the posttest probability of the disease is 60% (the patient has a 60% chance of having the target disorder if the test result is positive).
Correspondence: Dr Vermeersch, Laboratory Medicine, University Hospitals Leuven, Herestraat 49, B-3000 Leuven, Belgium (firstname.lastname@example.org).
Author Contributions:Study concept and design: Vermeersch and Bossuyt. Acquisition of data: Vermeersch. Analysis and interpretation of data: Vermeersch. Drafting of the manuscript: Vermeersch. Critical revision of the manuscript for important intellectual content: Vermeersch and Bossuyt. Statistical analysis: Vermeersch. Study supervision: Bossuyt.
Financial Disclosure: None reported.
Thank you for submitting a comment on this article. It will be reviewed by JAMA Internal Medicine editors. You will be notified when your comment has been published. Comments should not exceed 500 words of text and 10 references.
Do not submit personal medical questions or information that could identify a specific patient, questions about a particular case, or general inquiries to an author. Only content that has not been published, posted, or submitted elsewhere should be submitted. By submitting this Comment, you and any coauthors transfer copyright to the journal if your Comment is posted.
* = Required Field
Disclosure of Any Conflicts of Interest*
Indicate all relevant conflicts of interest of each author below, including all relevant financial interests, activities, and relationships within the past 3 years including, but not limited to, employment, affiliation, grants or funding, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. If all authors have none, check "No potential conflicts or relevant financial interests" in the box below. Please also indicate any funding received in support of this work. The information will be posted with your response.
Some tools below are only available to our subscribers or users with an online account.
Download citation file:
Web of Science® Times Cited: 6
Customize your page view by dragging & repositioning the boxes below.
Enter your username and email address. We'll send you a link to reset your password.
Enter your username and email address. We'll send instructions on how to reset your password to the email address we have on record.
Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.