Systematic Review of Clinical Prediction Rules for Neuroimaging in the Evaluation of Dementia FREE

DEMENTIA has profound effects on health outcomes and is associated with high medical care utilization and costs.¹ The identification of potentially reversible causes of dementia is critical, given the adverse consequences of delayed diagnosis. Neuroimaging of the brain, commonly computed tomography (CT) or magnetic resonance imaging (MRI), is a high-cost technology that can identify potentially reversible causes of dementia such as subdural hematomas, normal pressure hydrocephalus, and tumors.² These causes are estimated to account for less than 5% of all cases of dementia.³

To date, 9 clinical practice guidelines on the evaluation of dementia have been published and endorsed by 12 different organizations or groups.^4- 12 Eight guidelines contain a recommendation about neuroimaging; of these, none recommend the routine use of neuroimaging as part of the diagnostic evaluation of every case of dementia (Table 1).^5- 12 Seven of the 8 guidelines recommend the use of clinical criteria (ie, a clinical prediction rule) to identify patients for whom a neuroimaging study is recommended.^6- 12 These prediction rules are aimed at selecting patients with a higher likelihood of having a potentially reversible cause of dementia that can be diagnosed by neuroimaging (hereafter referred to as potentially reversible cause of dementia).

Despite the uniform recommendation for selective neuroimaging, the recommended clinical prediction rules vary widely across these guidelines (Table 1). Because of the observed variation of the clinical prediction rules for selective neuroimaging in dementia, we undertook an analysis of the evidence underlying these rules. The objective was to review systematically primary research studies on clinical prediction rules for neuroimaging in the evaluation of dementia, and to assess the sensitivity, specificity, and diagnostic performance of the rules.

We conducted a MEDLINE search of articles published from January 1, 1983, through December 31, 1998. Using the keyword dementia and medical subject headings dementia diagnosis or dementia etiology and tomography, x-ray computed, or magnetic resonance imaging, we identified 886 articles. After reviewing titles and abstracts of these articles, we selected for in-depth review 51 articles with possible information about a clinical prediction rule for neuroimaging in dementia. We identified an additional 6 articles for in-depth review from bibliographies of textbook chapters, review articles, and the guidelines about dementia.

We included articles that met the following criteria: (1) they reported a clinical prediction rule for the use of neuroimaging in dementia; (2) the clinical variables in the prediction rule were explicit and presented in sufficient detail to apply consistently in clinical practice; (3) every patient underwent a neuroimaging study (ie, CT or MRI); (4) all subjects were categorized based on the neuroimaging results as having or not having a potentially reversible cause of dementia; and (5) enough data were presented to calculate the sensitivity and specificity of the clinical prediction rule.

For each study meeting eligibility criteria, we abstracted information on the age, sex, dementia severity, and clinical setting of the sample. We also abstracted the clinical variables in each prediction rule, who performed the clinical evaluation, and whether the clinical variables were collected prospectively or retrospectively (ie, medical record review). We also recorded the type of neuroimaging study used and whether the physicians applying the prediction rule were aware of the neuroimaging findings.

For each study, we abstracted or calculated the number of patients (1) with and without a potentially reversible cause of dementia as defined by the neuroimaging results; (2) who had at least 1 of the clinical characteristics contained in the prediction rule (rule-positive findings); and (3) who did not have any of the clinical characteristics contained in the prediction rule (rule-negative findings).

We calculated the sensitivity, specificity, and 95% confidence interval (CI) for each rule. Sensitivity was defined as the proportion of patients with a neuroimaging-defined potentially reversible disorder and rule-positive findings. Specificity was defined as the proportion of patients with no neuroimaging-defined potentially reversible disorder and with rule-negative findings. We tested for heterogeneity between studies that evaluated the same prediction rule by constructing a summary receiver operating characteristic curve comparing the 95% CIs for the sensitivity plotted against 1 − specificity for each rule, and by comparing sensitivities and specificities for each rule using χ² or Fisher exact test.²⁵

Using a hypothetical cohort of 1000 patients with dementia, we estimated the positive and negative predictive values for each clinical prediction rule at different prevalences of neuroimaging-defined potentially reversible disorders (ie, 1%, 5%, 10%, and 15%). Since physicians and patients are frequently concerned that clinical prediction rules may miss treatable conditions,^26- 27 we also reported the proportion of neuroimaging-defined potentially reversible disorders in patients with rule-negative findings (ie, 1 − negative predictive value) and the number of potentially reversible cases of dementia missed by applying each prediction rule to the hypothetical cohort.

Of the 57 articles obtained for in-depth review, 7 met eligibility criteria for inclusion in this analysis^{13- 14,16- 17,20- 23,28- 31} (Table 2). One study evaluated 4 different prediction rules with the use of the same patient population,¹³ whereas another study compared 2 prediction rules in the same patient population.¹⁷ The other 5 studies evaluated 1 prediction rule each. From these 7 studies, we identified 6 different clinical prediction rules that we labeled as the Dietch, Larson high-risk, Larson low-risk, Bradshaw, American Academy of Neurology (AAN)–Chui, and Canadian Consensus Conference rules (Table 3). Four of the 6 rules were developed from clinical data collected through chart review or standardized examinations (Dietch, Larson high-risk, Larson low-risk, and Bradshaw), whereas the other 2 rules (AAN-Chui and Canadian Consensus Conference) were derived from existing consensus-based guidelines.

Two of the remaining 50 articles evaluated a prediction rule for neuroimaging in dementia, although they did not meet all of our inclusion criteria.^29- 30 One study only included patients with potentially reversible causes of dementia; thus, the specificity could not be calculated.²⁹ The other study did not include a sufficiently explicit set of clinical variables to allow others to consistently apply the prediction rule in clinical practice.¹⁴ The other 48 articles were excluded because they did not assess prediction rules.

Each study examined a consecutive series of patients with dementia who were referred to a dementia clinic^{13,16- 17,22} or to a radiology unit^20,28; the total number of patients undergoing evaluation per study ranged from 98 to 500 (Table 2). The overall frequency of potentially reversible causes of dementia detectable by neuroimaging was lower in patient populations drawn from dementia and geriatric clinics (0%-3.9%) compared with studies where patients were identified from radiology units (6.5%-10.4%). The average age of patients in the 7 studies ranged from 63 to 76 years. In 3 studies, the sex distribution was not specified. In the remaining 4 studies, the percentage of female subjects studied ranged from 53% to 71%. The severity of dementia, as measured by the Mini-Mental Status Examination, varied from an average of 15.4 to 23.4, but was not reported in 3 studies. In 6 studies, every patient received a CT scan; only 1 study used MRI. The use of contrast was reported in only 2 studies. In 3 studies, the prediction rules were assessed by collection of data abstracted from patient medical records, and no information was given as to whether the person abstracting the information was aware of the results of the CT scan of the head. The specialty of the physicians examining the patients varied within and between each study.

The Bradshaw, Larson high-risk, and Canadian Consensus Conference rules identified patients who should undergo a CT scan, whereas the Dietch, Larson low-risk, and AAN-Chui rules identified patients who need not undergo a CT scan. To compare all 6 prediction rules, we reworded the clinical variables in each rule so that neuroimaging is recommended if any of the variables are present (Table 3).

The Canadian Consensus Conference and Dietch rules included the largest number of clinical variables, whereas both Larson rules contained the fewest (Table 3). Although none of the 6 prediction rules were identical, they included common variables. All included a variable on the duration of dementia symptoms or acuity of change in cognitive function; 4 criteria included focal signs and symptoms. Clinical variables that might increase the likelihood of finding a subdural hematoma (eg, head trauma) or normal pressure hydrocephalus (eg, gait apraxia or urinary incontinence) were specified in only 3 rules.^10,21,28 Only 2 of the 6 prediction rules included an age cutoff as a criterion variable.^10,21 However, 3 of the 4 rules were developed in patient populations that excluded patients who were younger than 50 years²¹ or 60 years.^2,22

Table 4 presents the raw data from 7 studies and the sensitivity and specificity for each rule. Depending on the study, sensitivity varied widely, from 12.5% to 100.0%. In addition, for the Bradshaw and both Larson rules, sensitivity varied considerably (ie, differences ≥50%), depending on the sample to which the rule was applied. Similarly, specificity also ranged widely from 37.2% to 85.7%. Only 1 rule consistently had high sensitivity (>85%) across all studies evaluating the same rule; no rule consistently had high specificity (>85%) (Table 4).

Based on our analysis of heterogeneity across studies evaluating the same rule, these studies were too heterogenous to allow for pooling of results (data not shown). However, one of the studies applied 4 rules (Dietch, both Larson, and Bradshaw rules) to the same patient population.¹³ Thus, to estimate additional aspects of the diagnostic performance of all 6 prediction rules in a hypothetical cohort of patients, we selected the sensitivity and specificity for these 4 rules from this study. The sensitivity and specificity for each of the other 2 rules were drawn from the single study in which they were evaluated.

The Canadian Consensus Conference and Dietch rules performed the best, missing the fewest number of patients with potentially reversible causes of dementia across the range of prevalences evaluated (Table 5). However, this comes at the expense of subjecting more patients to imaging, compared with the other 5 rules. Application of the Dietch rule would result in 63% of all patients presenting with dementia undergoing imaging, whereas for the other rules, smaller proportions of patients would undergo imaging (24% for Larson low-risk rule; 36% for Larson high-risk rule; 21% for Bradshaw rule; 58% for Canadian Consensus Conference rule; and 37% for the AAN-Chui rule). The positive predictive value of all 6 rules was low, regardless of the prevalence of potentially reversible causes of dementia (Table 5). Of patients with rule-negative findings, most would not have a potentially reversible cause of dementia; however, the Dietch and Canadian Consensus Conference rules had the lowest proportion of patients with rule-negative findings and a potentially reversible cause of dementia (Table 5).

A clinical prediction rule used to identify dementia patients with a potentially reversible disorder should possess a high sensitivity to minimize the proportion of false-negative findings.^26- 27 Most clinicians and patients want to know how the application of a given rule affects the risk of missing a potentially reversible cause of dementia. Our review indicates that wide variation exists in the content and diagnostic performance of the 6 prediction rules reported in the literature. One rule (Dietch) had relatively high sensitivity (100.0% and 87.5% in both studies evaluating the Dietch rule); the other rules had lower sensitivities with larger variability between studies evaluating the same rule. Alexander and colleagues²⁹ also reported a high sensitivity for the Dietch criteria, but the study was excluded from our review because their study design did not enable calculation of specificity. They applied the Dietch criteria to 83 patients in a large health management organization population of elderly patients who had a brain tumor, subdural hematoma, or normal pressure hydrocephalus and who initially presented with some form of possible dementia. Of these 83 patients, 79 met the Dietch criteria for imaging (ie, 95.2% sensitivity [95% CI, 88.8%-99.2%]). The desired high sensitivity of the Dietch criteria comes at the expense of a low specificity and, hence, the least reduction in neuroimaging use, relative to a setting in which all patients with dementia undergo imaging. Therefore, at the population level, application of the Dietch rule vs the other study rules would have the smallest effect in decreasing utilization.

The current estimates of sensitivity and specificity are imprecise, having wide CIs. For example, most of the primary studies evaluating the performance of a prediction rule include fewer than 15 cases of potentially reversible causes of dementia (despite having total sample sizes ranging from 100-500). To obtain more precise estimates about a rule's performance, larger sample sizes will be required.²⁷ This imprecision in these estimates may explain in part the lack of consensus in the literature about whether all patients with dementia should undergo neuroimaging.^{23,30- 31,34}

It is not clear, by applying even the high-sensitivity rules, if the number of missed cases is acceptable to physicians and patients. For example, if the prevalence of potentially reversible causes of dementia is as low as 1%, then applying the highest-sensitivity rule (Dietch) would miss only 1 patient of 10 in a cohort of 1000 patients with dementia. However, as the prevalence of potentially reversible cases increases, even at 10%, the number of missed cases is 13 of 50. Nevertheless, this represents a small proportion (3.6%) of all patients with rule-negative findings. Many clinicians may consider this rate as unacceptably high—particularly if current practice is that all new dementia patients undergo imaging—given concerns about quality of care and medicolegal ramifications. Adoption of less sensitive rules or even the Dietch rule may lead to underutilization of neuroimaging and underdetection of potentially reversible causes of dementia.

To apply these rules effectively, physicians must be proficient in eliciting a neurologic history and in performing a neurologic examination. However, some authorities believe that nonneurologists receive an insufficient amount of training in neurology.^35- 37 This belief is further supported by a recent study that found that 76% of patients with moderate to severe cognitive impairment were not recognized as possibly having a dementia syndrome by their primary care physician.³⁸ Thus, patients with a potentially reversible cause of dementia who have clinical findings consistent with criteria for neuroimaging may be misclassified as not meeting the criteria by physicians who do not perform a detailed neurologic history and physical examination.³⁴

None of the clinical prediction rules evaluated in the primary studies were completely identical to any of the rules recommended by existing published guidelines on the evaluation of dementia. Three of the guidelines^7,9,11 contained imprecise definitions of clinical variables about the indications for neuroimaging (eg, "etiology of cognitive changes are not apparent after history, physical, or laboratory testing"), which precluded comparison with the clinical prediction rules from the primary studies. Only 4 guidelines cited a source for their prediction rule,^{7- 8,11- 12} 3 of which cited at least 1 of the 6 studies we identified.^7- 8,12 However, none cited all 6 studies, although 5 of them were published before or during the year (1987) the oldest guideline⁹ was issued. In addition, no guideline included a summary of the data on the diagnostic utility of the previously published clinical prediction rules or estimated the possible diagnostic performance of the clinical prediction rule that they recommend. This is not in keeping with the recently published literature that recommends that guideline recommendations should be based on a comprehensive literature review and should present the evidence and rationale supporting each recommendation.^39- 42 None of the guidelines included a discussion about the implications of implementing their recommendations in clinical practice, information that most physicians say they want in practice guidelines.⁴³

It is unclear what effect application of any of the rules recommended by the clinical practice guidelines would have because of differences in content between the guideline rules and the rules evaluated in the literature. However, only the AAN⁶ and the New York Department of Health¹² guidelines contained a set of clinical variables similar to—but not as comprehensive as—the Canadian Consensus Conference or Dietch rule, the highest-sensitivity rules evaluated in the literature.

The principal reason for identifying a neuroimaging-defined potentially reversible cause of dementia is to improve patient outcomes. However, none of the guidelines reviewed contain a discussion about the benefits to patients of identifying potentially treatable causes of dementia. This may result in part from lack of data on the effectiveness that treating potentially reversible causes has on cognitive function, quality of life, or survival. In a comprehensive review that pooled data from 11 studies, 42% of patients with potentially reversible cause of dementia detectable by neuroimaging had some improvement after treatment, whereas only 8% exhibited complete resolution of their dementia.³ Early detection of normal pressure hydrocephalus or subdural hematomas is more likely to improve outcomes after surgery compared with patients with tumors causing dementia.^3,20,29,44 Further studies evaluating the impact of neuroimaging on physicians' management decisions and on patient outcomes are needed to evaluate the role neuroimaging has in the evaluation of dementia.

Research studies are needed to evaluate the trade-offs in costs and health outcomes associated with application of a prediction rule relative to subjecting all dementia patients to imaging. Research is also needed to better understand what false-negative rates and costs society, physicians, and patients would tolerate and the impact of malpractice concerns on clinical decision making and receptiveness to use of these rules by primary care physicians and by specialists. These studies should also compare the different kinds of neuroimaging technology currently available, eg, the use of MRI compared with CT scanning to detect potentially reversible causes of dementia. In addition, these technologies should be evaluated for their ability to rule in other forms of dementia (eg, Alzheimer disease or vascular dementia), and the impact neuroimaging results have on patient outcomes. Although MRI may be more sensitive than CT scanning, it is unclear if MRI is superior to CT scanning in the routine evaluation of dementia, and further studies are needed to determine the role of MRI in dementia.^2,5,7,45

We found that the body of research evaluating the clinical prediction rules for neuroimaging in dementia was insufficient to understand the risk and benefits with certainty. The inadequacy of the existing literature may have contributed partially to the considerable variation in the clinical prediction rules recommended within current clinical practice guidelines. The evidence suggests that application of these rules might result in underdetection of potentially reversible causes of dementia. Given the rising prevalence of dementia, the cost of this technology, and the potential adverse consequences of underusing this technology, there is an urgent need for large, well-designed studies evaluating the utility of neuroimaging in patients with dementia.

Accepted for publication April 4, 2000.

This work was supported by a grant from the New York State Department of Health, Albany, to the American Academy of Neurology, Minneapolis, Minn (New York State Comptroller C-012600).

Presented as a poster at the 1998 Annual Meeting for the Society of General Internal Medicine, Chicago, Ill, April 24, 1998.

We thank Michael Stein, MD, Martin Shapiro, MD, PhD, Deidre S. Gifford, MD, MPH, and Al Mushlin, MD, for providing comments on early versions of this manuscript. We also thank Ashley Crittenden for help with manuscript preparation.

The opinions contained herein represent the views of the authors only and do not necessarily reflect the position of the American Academy of Neurology; the New York State Department of Health; the University of California–Los Angeles; Brown University, Providence, RI; or the University of Rochester, Rochester, NY.

Corresponding author and reprints: David R. Gifford, MD, MPH, Division of Geriatrics, Rhode Island Hospital, 593 Eddy St, Providence, RI 02903 (e-mail: David_Gifford@Brown.edu).