0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letter |

Medicine’s Uncomfortable Relationship With Math:  Calculating Positive Predictive Value

Arjun K. Manrai, AB1,2; Gaurav Bhatia, MS1,3; Judith Strymish, MD4; Isaac S. Kohane, MD, PhD1,2; Sachin H. Jain, MD4,5,6
[+] Author Affiliations
1Harvard-MIT Health Sciences and Technology, Cambridge, Massachusetts
2Center for Biomedical Informatics, Harvard Medical School, Boston, Massachusetts
3The Eli and Edythe L Broad Institute of MIT and Harvard, Cambridge, Massachusetts
4Department of Veterans Affairs (VA) Boston Healthcare System, Harvard Medical School, Boston, Massachusetts
5Merck Medical Information and Innovation, Merck and Company, Boston, Massachusetts
6Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts
JAMA Intern Med. 2014;174(6):991-993. doi:10.1001/jamainternmed.2014.1059.
Text Size: A A A
Published online

Extract

In 1978, Casscells et al1 published a small but important study showing that the majority of physicians, house officers, and students overestimated the positive predictive value (PPV) of a laboratory test result using prevalence and false positive rate. Today, interpretation of diagnostic tests is even more critical with the increasing use of medical technology in health care. Accordingly, we replicated the study by Casscells et al1 by asking a convenience sample of physicians, house officers, and students the same question: “If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease, assuming you know nothing about the person's symptoms or signs?”

Figures in this Article

Topics

mathematics

Sign in

Create a free personal account to sign up for alerts, share articles, and more.

Purchase Options

• Buy this article
• Subscribe to the journal

First Page Preview

View Large
First page PDF preview

First Page Preview

View Large
First page PDF preview

Figures

Place holder to copy figure label and caption
Figure.
Distribution of Responses to Survey Question Provided in the Article Text

Of 61 respondents, 14 provided the correct answer of 2%. The most common answer was 95%, provided by 27 of 61 respondents. The median answer was 66%, which is 33 times larger than the true answer.

Graphic Jump Location

Tables

References

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment
The authors are not entirely correct in their solution
Posted on May 7, 2014
Paul Gerrard, MD
Spaulding Rehabilitation Hospital
Conflict of Interest: None Declared
The authors are actually wrong about the 2% correct answer. The correct answer to the math problem that the authors presented is actually incalculable.Positive Predictive Value = True Positives / Number of Positive Test Results.The math problem in the study give the false positive rate and the disease prevalence. However, unless I am missing something, it doesn't give the number of those who truly have the disease who will have a positive test result (True positive). Without this, an accurate answer cannot be calculated.To come up with the 2% calculation, the authors made an additional assumption neither stated in the problem nor clearly implied: They had to assume that there is 1 true positive in each 1,000 person sample. (Which will yield 1/51 = 1.96%)It is a bit problematic to say that people got this question wrong, when a correct answer is not calculable without making an additional assumption. Perhaps what is most telling about this article is not that so many subjects got the problem \"wrong,\" but that the authors managed to get it published in one of the most prestigious medical journals despite the fact that a \"right\" answer was not calculable.
Authors are correct and this is a frequent error
Posted on August 27, 2014
Dave Curtis
University College London
Conflict of Interest: None Declared
Actually, the authors do say that the test is assumed to be perfectly sensitive so they have not made the error Paul Gerrard accuses them of.I have seen this same error even in high profile scientific publications. In a news article in the Lancet entitled \"At-home HIV test poses dilemmas and opportunities\" the original version reported the false positive rate as if it was one minus the specificity. We wrote in and the article has now been corrected so there is no sign of the original error in the online version, just a note to say there has been a correction. (http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(12)61585-2/fulltext)It is difficult enough to have sensible conversations about the advantages and disadvantages of screening and diagnostic tests without such widespread innumeracy among health professionals. I congratulate the authors for highlighting this issue.
Learning to make assumptions while practising medicine, for there is no certainties in life except births and deaths; we don't need level 1 evidence for everything‏
Posted on September 25, 2014
Shyan Goh
Sydney, Australia
Conflict of Interest: None Declared
The basis of the 1978 paper by Casscells et al was the equation:PPV = (p x Se) / [(p x Se) + (1 - p)(1 - Sp)]where PPV is Positive Predictive Value, p is disease prevalence, Se is test Sensitivity, Sp is test SpecificityRestrictly speaking using the equation from this paper, p = 1/1000, (1- Sp) = False Positive Rate = 5/100If Se is 0, then PPV is 0If Se is 1, then PPV is 100/5095 or 0.0196Regardless of whatever true value of Se is (ie no assumption required), PPV cannot be more than 0.0196However this equation is not commonly used in this format.Whereas in the \"common sense\" reasoning from the actual Casscells et al paper uses the usual equation for PPV beingPPV = TP / (TP + FP)whereby TP is True Positive and FP is False PositiveBased on this equation:1 out of any 1000 people has the diseaseSo 999 people does not have the diseaseFalse Positive Rate of a test for the disease is 5%Of 999 people without the disease, about 0.05 X 999 = just under 50 people would have tested positive on the test.Of 1000 people tested, there can only be maximum of 1 person who is tested positive with the disease (true positive)This (just under) 50 people (false positive) plus 1 person (max possible number of people with the disease to have positive test in a group of 1000 ie maximum possible true positive) means there is a maximum possible just under 51 people who would have tested positive out of 1000 So PPV is TP / (TP + FP) = (maximum possible 1) / (just under 51) = not more than 1/51The issue therefore at hand is how respondents being tested is willing to make some assumption and extrapolation within reason using what is known as possible.The trouble is that not many of us are taught to make some reasonable assumption nowsdays given the direction of medical litigation is going plus the fanatical (and sometimes senseless) call to demand for evidence for every thing we do.The fact is, it IS safe to assume, for example, everyone who jump off the top of a skyscraper, is going to die and does not require level 1 evidence base to support that assumption.Anyone who does survive this fall is a very very lucky person as much luck being winning the grand prize in the biggest El Gordo lottery.
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 3

Sign in

Create a free personal account to sign up for alerts, share articles, and more.

Purchase Options

• Buy this article
• Subscribe to the journal

Related Content

Customize your page view by dragging & repositioning the boxes below.

See Also...
Articles Related By Topic
Related Collections
PubMed Articles
Jobs
brightcove.createExperiences();