0
Research Letters |

Validation and Comparison of a Novel Screening Guideline for Kidney Disease: KEEPing SCORED FREE

Heejung Bang, PhD; Madhu Mazumdar, PhD; Lisa M. Kern, MD, MPH; David A. Shoham, PhD; Phyllis A. August, MD, MPH; Abhijit V. Kshirsagar, MD, MPH
Arch Intern Med. 2008;168(4):432-435. doi:10.1001/archinternmed.2007.122.
Text Size: A A A
Published online

Chronic kidney disease (CKD) is one of the world's major public health problems. Nearly 1 in 9 adults (20 million people) in the United States have CKD, and it is estimated that another 20 million are at increased risk.1 Given the asymptomatic nature of kidney disease, affected individuals and health care providers may be unaware of the condition in patients. Identifying individuals with early kidney disease would be a useful first step in preventing progression to end-stage renal disease as well as reducing morbidity and mortality from cardiovascular disease (CVD).

We recently published an instrument (SCreening for Occult REnal Disease [SCORED]) to systematically identify individuals with a high likelihood of prevalent CKD.2 Derived from the National Health and Nutrition Examination Survey (NHANES) 1999-2002 and (partially) validated in the Atherosclerosis Risk in Communities (ARIC) study, SCORED identified 9 demographic and medical variables that could be assigned integer values and then entered into a scoring algorithm. The scoring algorithm was intentionally simplified to be accessible to lay persons and health care providers.

To our knowledge, SCORED is the only algorithm derived from scientific modeling, rather than expert scientific opinion, targeted for general population screening. Alternatively, the National Kidney Foundation has encouraged screening strategies targeted at high-risk groups. Through the Kidney Early Evaluation Program (KEEP), the National Kidney Foundation recommends using the following characteristics to identify individuals with a high likelihood of kidney disease: “if a person is 18 years or older and has one or more of the following: diabetes; high blood pressure; or a family history of diabetes, high blood pressure or kidney disease.”3,4

It is unclear how the SCORED algorithm compares with the KEEP guidelines. Such comparisons would be useful for clinicians and others. Since the publication of SCORED, we have performed an additional validation study using new, independent samples: (1) NHANES 2003-2004 and (2) a combined cohort of the ARIC study and Cardiovascular Heart Study (ARIC/CHS). We report the findings herein.

The ARIC study enrolled 15 792 participants aged 45 to 64 years between 1987 and 1989, and CHS recruited 5201 subjects 65 years and older between 1989 and 1990. Both are community studies, and detailed descriptions have been published previously.2,5,6 Some data disparities and limitations to be noted are summarized in a footnote of the Table.

Table Graphic Jump LocationTable. Diagnostic Performance of SCORED vs KEEP Using Validation Datasets (NHANES 2003-2004 [N = 4298] and ARIC/CHS [N = 21 221])a
SCORED Model

The SCORED model is a multivariable mathematical function that gives an estimated probability of having CKD as follows:

Probability(CKD) = 1/[1 + exp(−β" × x)],

where

β" × x = −5.4 + 1.55 × I(Age 50-59 years) + 2.31 × I(Age 60-69 years) + 3.23 × I(Age ≥70 years) + 0.29 × I(Female) + 0.93 × I(Anemia) + 0.45 × I(Hypertension) + 0.44 × I(Diabetes) + 0.59 × I(History of CVD) + 0.45 × I(History of Heart Failure) + 0.74 × I(Peripheral Vascular Disease) + 0.83 × I(Proteinuria).

The total integer score is a weighted sum, ranging from 0 to 12, and is calculated as follows:

Total score = 2 × I(Age 50-59 years) + 3 × I(Age 60-69 years) + 4 × I(Age ≥70 years) + 1 × I(Female) + 1 × I(Anemia) + 1 × I(Hypertension) +1 × I(Diabetes) + 1 × I(History of CVD) + 1 × I(History of Heart Failure) + 1 × I(Peripheral Vascular Disease) + 1 × I(Proteinuria),

where I(A) is an indicator taking 1 for condition A and 0 for otherwise. If the total score is greater than or equal to 4, a confirmatory blood examination (eg, serum creatinine and/or urinalysis) by a physician is strongly recommended. A user-friendly questionnaire is also provided in the original article.2

Measures for Validation and Comparison

Creatinine level was calibrated in all studies as recommended by the National Kidney Disease Education Program and the National Center for Health Statistics.79 Kidney function was quantified by glomerular filtration rate from the 4-variable Modification of Diet in Renal Disease study equation, and CKD was defined as a glomerular filtration rate lower than 60 mL/min/1.73m2, which corresponds to stage 3 or higher kidney disease.3,10

The SCORED model was evaluated in the validation datasets using standard measures: percentage of people identified to be at high risk by the given rule, sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve (AUC).

Because the ARIC study and CHS did not measure urinary protein level for proteinuria, we created 3 different approaches to handling missing or incomplete information and assessed sensitivity of the estimates of the selected measures. First, we started the analysis with ARIC/CHS datasets without proteinuria, which reflects the situation that no one reported that she or he had proteinuria. Second, we substituted urinary protein level measured at the year 5 follow-up visit for CHS subjects if available or imputed the measurement via a missing data analysis method otherwise.11 Third, we defined the highest 10 percentiles of uric acid level as proteinuria (to compare, 9.5% of subjects had proteinuria in NHANES). (The rationale for using high uric acid level as a surrogate marker for proteinuria, at least for sensitivity checking, is that it is used in the diagnosis and treatment of renal disorders, including renal failure or injury and is associated with proteinuria.12)

We compared SCORED vs KEEP using the same set of diagnostic measures. It is important to note that none of the datasets we examined contained information on “family history of kidney disease,” a criterion needed for KEEP. As a result, the SCORED model could not examine the effect of this potentially important factor. The absence of this data was also a problem for the direct comparison between KEEP vs SCORED. To improve comparability, we repeated our analysis in 2 scenarios: (1) ignoring this criterion and (2) after applying the fill-in method based on missing data imputation and replacement by surrogate variables. These statistical strategies, although imperfect, would offer a sound method of sensitivity analysis in practice.

For the NHANES dataset, sensitivity, specificity, positive predictive value, and negative predictive value of SCORED were estimated to be 95%, 65%, 20%, and 99%, respectively, with an AUC of 0.88. The application to ARIC/CHS provided sensitivity of 88% to 90%, specificity of 50% to 52%, positive predictive value of 13% to 14%, and negative predictive value of 98%, with an AUC of 0.78 to 0.80 (Table). In contrast, KEEP tended to place larger proportions of subjects into the high-risk category than SCORED (67%-77% vs 40%-53%), and the sensitivity for detecting CKD was 86% to 92%, with a specificity of 24% to 35%, positive predictive value of 3% to 12%, negative predictive value of 95% to 98%, and an AUC of 0.65 to 0.77. Results were robust under the different assumptions and approaches we adopted. Our comparisons give some indication about the expected performance of the 2 screening programs in real-world settings.

We report the diagnostic characteristics of the SCORED model and its performance compared with the KEEP guidelines. Ninety-five percent of individuals with CKD were identified by the SCORED algorithm as being at high risk (cutoff of ≥4); among those who did not have CKD, 65% scored less than 4 in NHANES. Those values were somewhat reduced in the ARIC/CHS samples, possibly owing to differences in population characteristics, variable definitions, and available data.

The SCORED model exhibited improved performances in all the criteria we examined. For KEEP as well as SCORED, NHANES showed better results than ARIC/CHS. Based on the higher AUC of the SCORED model compared with the KEEP guidelines, the improved performance characteristics are likely because of the use of additional variables and different weights for age groups. Indeed, questions about underlying CVD, common in the CKD population as demonstrated by the KEEP data,13 may improve their screening efforts.

There are also some limitations. Some variables that are a part of SCORED or KEEP were unavailable or incomplete in the databases we used, including family history of kidney disease and proteinuria. We addressed this issue with different statistical strategies. The absence of these variables in well-known studies may reflect low awareness of these important risk factors. We encourage the collection of such information in future epidemiological studies including the next iteration of NHANES. Furthermore, there are known negative consequences of screening. Overdiagnosis, misdiagnosis, anxiety, cost, and creating a false sense of security are some adverse effects of screening and render screening in some diseases controversial.1418 We propose that SCORED would minimize the potential adverse outcomes associated with screening by identifying approximately 25% fewer screeners as being at high risk compared with KEEP. The greater specificity of SCORED may reduce overdiagnosis and enhance resource use.

Screening is a public health strategy for identifying an unrecognized disease in asymptomatic populations, in which subjects are asked questions or offered a test, to identify those individuals who are more likely to be helped than harmed by further tests or treatment to reduce the risk of a disease or its complications. Diseases suitable for screening are those with serious consequences, those in which treatment is more effective at an earlier stage, and conditions with a long preclinical phase.16 Chronic kidney disease fulfills these criteria; however, it is not known whether screening will in fact result in improved outcomes and reduced progression to end-stage renal disease.17 The benefits of screening for CKD and its effectiveness and cost-effectiveness are yet to be determined.19

In conclusion, SCORED and KEEP are simple and inexpensive to use and carry some, albeit, minor, consequences of a false-positive result. Because of its general population sampling, we believe that SCORED has greater applicability and could also serve as an educational tool to raise CKD awareness. There is ample evidence that CKD is underdiagnosed and undertreated and that the burden of CKD is increasing worldwide.20 Not many people are aware that undiagnosed CKD can lead to serious consequences, including kidney failure, dialysis, and even death, and is associated with CVD. We propose that increased screening would identify greater numbers of individuals who may potentially benefit from strategies known to slow the progression of renal disease, a hypothesis that warrants testing in prospective clinical trials. We should “act on the best available evidence—as opposed to waiting for the best possible evidence.”21(p111)

Correspondence: Dr Bang, Division of Biostatistics and Epidemiology, Department of Public Health, Weill Medical College of Cornell University, 411 E 69th St, New York, NY 10021 (heb2013@med.cornell.edu).

Author Contributions: Dr Bang had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Bang, Mazumdar, Shoham, and Kshirsagar. Acquisition of data: Bang. Analysis and interpretation of data: Bang, Mazumdar, Kern, Shoham, August, and Kshirsagar. Drafting of the manuscript: Bang, Shoham, August, and Kshirsagar. Critical revision of the manuscript for important intellectual content: Bang, Mazumdar, Kern, and Kshirsagar. Statistical analysis: Bang, Mazumdar, and Shoham. Obtained funding: Mazumdar. Administrative, technical, and material support: Kern and Kshirsagar. Study supervision: August and Kshirsagar.

Financial Disclosure: None reported.

Funding/Support: The ARIC and CHS studies are conducted and supported by the National Heart, Lung, and Blood Institute (NHLBI) in collaboration with ARIC and CHS investigators. Drs Bang and Mazumdar were partially supported by the Tolly Vinik Trust through the Institute of Clinical Research of Weill Medical College of Cornell University. Dr Mazumdar was additionally supported by grant R25CA105012 (Collaborative Program in Nutrition and Cancer Prevention). Dr Shoham is a postdoctoral trainee supported by NHLBI institutional training grant 5T-32HL007055-29.

Disclaimer: This manuscript was prepared using a limited-access dataset obtained from the NHLBI and does not necessarily reflect the opinions or views of the ARIC/CHS or NHLBI.

Additional Contributions: Kylie Braynt, MS, provided CHS data setup and Sean Coady, MA, at NHLBI assisted in data issues and clarifications. We thank the staff and participants of the NHANES, ARIC study, and CHS for their important contributions and valuable information for health research.

Coresh  JByrd-Holt  DAstor  BC  et al.  Chronic kidney disease awareness, prevalence, and trends among US adults, 1999 to 2000. J Am Soc Nephrol 2005;16 (1) 180- 188
PubMed
Bang  HVupputuri  SShoham  DA  et al.  SCreening for Occult REnal Disease (SCORED). Arch Intern Med 2007;167 (4) 374- 381
PubMed
National Kidney Foundation, K/DOQI clinical practice guidelines for chronic kidney disease. Am J Kidney Dis 2002;39 (2) ((suppl 1)) S1- S266
PubMed
National Kidney Foundation, Kidney Early Evaluation Program. Am J Kidney Dis 2005;45 (2) ((suppl 2)) S1- S135
PubMed
The ARIC investigators, The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. Am J Epidemiol 1989;129 (4) 687- 702
PubMed
Fried  LPBorhani  NOEnright  P  et al.  The Cardiovascular Health Study: design and rationale. Ann Epidemiol 1991;1 (3) 263- 276
PubMed
Coresh  JAstor  BCMcQuillan  G  et al.  Calibration and random variation of the serum creatinine assay as critical elements of using equations to estimate glomerular filtration rate. Am J Kidney Dis 2002;39 (5) 920- 929
PubMed
Myers  GLMiller  WGCoresh  J  et al.  Recommendations for improving serum creatinine measurement. Clin Chem 2006;52 (1) 5- 18
PubMed
Weiner  DETighiouart  HAmin  MG  et al.  Chronic kidney disease as a risk factor for cardiovascular disease and all-cause mortality: a pooled analysis of community-based studies. J Am Soc Nephrol 2004;15 (5) 1307- 1315
PubMed
Levey  ASGreen  TKusek  JBeck  GMDRD Study Group, A simplified equation to predict glomerular filtration rate from serum creatinine [abstract]. J Am Soc Nephrol 2000;11(suppl)155A
SAS Institute Inc, SAS/STAT User's Guide Version 8.2.  Cary, NC SAS Institute Inc2000;
Lee  JEKim  Y-GChoi  Y-HHuh  WKim  DJOh  HY Serum uric acid is associated with microalbuminuria in prehypertension. Hypertension 2006;47 (5) 962- 967
PubMed
McCullough  PAJurkovitz  CTPergola  PE  et al.  Independent components of chronic kidney disease as a cardiovascular risk state. Arch Intern Med 2007;167 (11) 1122- 1129
PubMed
McClellan  WMRamirez  SPBJurkovitz  C Screening for chronic kidney disease: unresolved issues. J Am Soc Nephrol 2003;14 (7) ((suppl 2)) S81- S87
PubMed
Gansevoort  RTBakker  SJLde Jong  PE Early detection of progressive chronic kidney disease: is it feasible? J Am Soc Nephrol 2006;17 (5) 1218- 1220
PubMed
Wilson  JJungner  G The Principles and Practice of Screening for Disease.  Geneva, Switzerland World Health Organization1968;
Hallan  SIDahl  KOien  CM  et al.  Screening strategies for chronic kidney disease in the general population. BMJ 2006;333 (7577) 1047
PubMed
Freedman  DAPetitti  DBRobins  JM On the efficacy of screening for breast cancer. Int J Epidemiol 2004;33 (1) 43- 55
PubMed
Bang  H Medical cost analysis: application to colorectal cancer data from the SEER Medicare database. Contemp Clin Trials 2005;26 (5) 586- 597
PubMed
Meguid El Nahas  ABello  AK Chronic kidney disease: the global challenge. Lancet 2005;365 (9456) 331- 340
PubMed
Kaplah  JPedLiverman  CTedKraak  VIed Preventing Childhood Obesity: Health in the Balance.  Washington, DC Institute of Medicine2004;

Figures

Tables

Table Graphic Jump LocationTable. Diagnostic Performance of SCORED vs KEEP Using Validation Datasets (NHANES 2003-2004 [N = 4298] and ARIC/CHS [N = 21 221])a

References

Coresh  JByrd-Holt  DAstor  BC  et al.  Chronic kidney disease awareness, prevalence, and trends among US adults, 1999 to 2000. J Am Soc Nephrol 2005;16 (1) 180- 188
PubMed
Bang  HVupputuri  SShoham  DA  et al.  SCreening for Occult REnal Disease (SCORED). Arch Intern Med 2007;167 (4) 374- 381
PubMed
National Kidney Foundation, K/DOQI clinical practice guidelines for chronic kidney disease. Am J Kidney Dis 2002;39 (2) ((suppl 1)) S1- S266
PubMed
National Kidney Foundation, Kidney Early Evaluation Program. Am J Kidney Dis 2005;45 (2) ((suppl 2)) S1- S135
PubMed
The ARIC investigators, The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. Am J Epidemiol 1989;129 (4) 687- 702
PubMed
Fried  LPBorhani  NOEnright  P  et al.  The Cardiovascular Health Study: design and rationale. Ann Epidemiol 1991;1 (3) 263- 276
PubMed
Coresh  JAstor  BCMcQuillan  G  et al.  Calibration and random variation of the serum creatinine assay as critical elements of using equations to estimate glomerular filtration rate. Am J Kidney Dis 2002;39 (5) 920- 929
PubMed
Myers  GLMiller  WGCoresh  J  et al.  Recommendations for improving serum creatinine measurement. Clin Chem 2006;52 (1) 5- 18
PubMed
Weiner  DETighiouart  HAmin  MG  et al.  Chronic kidney disease as a risk factor for cardiovascular disease and all-cause mortality: a pooled analysis of community-based studies. J Am Soc Nephrol 2004;15 (5) 1307- 1315
PubMed
Levey  ASGreen  TKusek  JBeck  GMDRD Study Group, A simplified equation to predict glomerular filtration rate from serum creatinine [abstract]. J Am Soc Nephrol 2000;11(suppl)155A
SAS Institute Inc, SAS/STAT User's Guide Version 8.2.  Cary, NC SAS Institute Inc2000;
Lee  JEKim  Y-GChoi  Y-HHuh  WKim  DJOh  HY Serum uric acid is associated with microalbuminuria in prehypertension. Hypertension 2006;47 (5) 962- 967
PubMed
McCullough  PAJurkovitz  CTPergola  PE  et al.  Independent components of chronic kidney disease as a cardiovascular risk state. Arch Intern Med 2007;167 (11) 1122- 1129
PubMed
McClellan  WMRamirez  SPBJurkovitz  C Screening for chronic kidney disease: unresolved issues. J Am Soc Nephrol 2003;14 (7) ((suppl 2)) S81- S87
PubMed
Gansevoort  RTBakker  SJLde Jong  PE Early detection of progressive chronic kidney disease: is it feasible? J Am Soc Nephrol 2006;17 (5) 1218- 1220
PubMed
Wilson  JJungner  G The Principles and Practice of Screening for Disease.  Geneva, Switzerland World Health Organization1968;
Hallan  SIDahl  KOien  CM  et al.  Screening strategies for chronic kidney disease in the general population. BMJ 2006;333 (7577) 1047
PubMed
Freedman  DAPetitti  DBRobins  JM On the efficacy of screening for breast cancer. Int J Epidemiol 2004;33 (1) 43- 55
PubMed
Bang  H Medical cost analysis: application to colorectal cancer data from the SEER Medicare database. Contemp Clin Trials 2005;26 (5) 586- 597
PubMed
Meguid El Nahas  ABello  AK Chronic kidney disease: the global challenge. Lancet 2005;365 (9456) 331- 340
PubMed
Kaplah  JPedLiverman  CTedKraak  VIed Preventing Childhood Obesity: Health in the Balance.  Washington, DC Institute of Medicine2004;

Correspondence

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
NOTE:
Citing articles are presented as examples only. In non-demo SCM6 implementation, integration with CrossRef’s "Cited By" API will populate this tab (http://www.crossref.org/citedby.html).
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 7

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Topics
PubMed Articles