[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.159.202.12. Please contact the publisher to request reinstatement.
[Skip to Content Landing]
Download PDF
Table. 
Diagnostic Performance of SCORED vs KEEP Using Validation Datasets (NHANES 2003-2004 [N = 4298] and ARIC/CHS [N = 21 221])a
Diagnostic Performance of SCORED vs KEEP Using Validation Datasets (NHANES 2003-2004 [N = 4298] and ARIC/CHS [N = 21 221])a
1.
Coresh  JByrd-Holt  DAstor  BC  et al.  Chronic kidney disease awareness, prevalence, and trends among US adults, 1999 to 2000. J Am Soc Nephrol 2005;16 (1) 180- 188
PubMedArticle
2.
Bang  HVupputuri  SShoham  DA  et al.  SCreening for Occult REnal Disease (SCORED). Arch Intern Med 2007;167 (4) 374- 381
PubMedArticle
3.
National Kidney Foundation, K/DOQI clinical practice guidelines for chronic kidney disease. Am J Kidney Dis 2002;39 (2) ((suppl 1)) S1- S266
PubMedArticle
4.
National Kidney Foundation, Kidney Early Evaluation Program. Am J Kidney Dis 2005;45 (2) ((suppl 2)) S1- S135
PubMed
5.
The ARIC investigators, The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. Am J Epidemiol 1989;129 (4) 687- 702
PubMed
6.
Fried  LPBorhani  NOEnright  P  et al.  The Cardiovascular Health Study: design and rationale. Ann Epidemiol 1991;1 (3) 263- 276
PubMedArticle
7.
Coresh  JAstor  BCMcQuillan  G  et al.  Calibration and random variation of the serum creatinine assay as critical elements of using equations to estimate glomerular filtration rate. Am J Kidney Dis 2002;39 (5) 920- 929
PubMedArticle
8.
Myers  GLMiller  WGCoresh  J  et al.  Recommendations for improving serum creatinine measurement. Clin Chem 2006;52 (1) 5- 18
PubMedArticle
9.
Weiner  DETighiouart  HAmin  MG  et al.  Chronic kidney disease as a risk factor for cardiovascular disease and all-cause mortality: a pooled analysis of community-based studies. J Am Soc Nephrol 2004;15 (5) 1307- 1315
PubMedArticle
10.
Levey  ASGreen  TKusek  JBeck  GMDRD Study Group, A simplified equation to predict glomerular filtration rate from serum creatinine [abstract]. J Am Soc Nephrol 2000;11(suppl)155A
11.
SAS Institute Inc, SAS/STAT User's Guide Version 8.2.  Cary, NC SAS Institute Inc2000;
12.
Lee  JEKim  Y-GChoi  Y-HHuh  WKim  DJOh  HY Serum uric acid is associated with microalbuminuria in prehypertension. Hypertension 2006;47 (5) 962- 967
PubMedArticle
13.
McCullough  PAJurkovitz  CTPergola  PE  et al.  Independent components of chronic kidney disease as a cardiovascular risk state. Arch Intern Med 2007;167 (11) 1122- 1129
PubMedArticle
14.
McClellan  WMRamirez  SPBJurkovitz  C Screening for chronic kidney disease: unresolved issues. J Am Soc Nephrol 2003;14 (7) ((suppl 2)) S81- S87
PubMedArticle
15.
Gansevoort  RTBakker  SJLde Jong  PE Early detection of progressive chronic kidney disease: is it feasible? J Am Soc Nephrol 2006;17 (5) 1218- 1220
PubMedArticle
16.
Wilson  JJungner  G The Principles and Practice of Screening for Disease.  Geneva, Switzerland World Health Organization1968;
17.
Hallan  SIDahl  KOien  CM  et al.  Screening strategies for chronic kidney disease in the general population. BMJ 2006;333 (7577) 1047
PubMedArticle
18.
Freedman  DAPetitti  DBRobins  JM On the efficacy of screening for breast cancer. Int J Epidemiol 2004;33 (1) 43- 55
PubMedArticle
19.
Bang  H Medical cost analysis: application to colorectal cancer data from the SEER Medicare database. Contemp Clin Trials 2005;26 (5) 586- 597
PubMedArticle
20.
Meguid El Nahas  ABello  AK Chronic kidney disease: the global challenge. Lancet 2005;365 (9456) 331- 340
PubMedArticle
21.
Kaplah  JPedLiverman  CTedKraak  VIed Preventing Childhood Obesity: Health in the Balance.  Washington, DC Institute of Medicine2004;
Research Letters
February 25, 2008

Validation and Comparison of a Novel Screening Guideline for Kidney Disease: KEEPing SCORED

Arch Intern Med. 2008;168(4):432-435. doi:10.1001/archinternmed.2007.122

Chronic kidney disease (CKD) is one of the world's major public health problems. Nearly 1 in 9 adults (20 million people) in the United States have CKD, and it is estimated that another 20 million are at increased risk.1 Given the asymptomatic nature of kidney disease, affected individuals and health care providers may be unaware of the condition in patients. Identifying individuals with early kidney disease would be a useful first step in preventing progression to end-stage renal disease as well as reducing morbidity and mortality from cardiovascular disease (CVD).

We recently published an instrument (SCreening for Occult REnal Disease [SCORED]) to systematically identify individuals with a high likelihood of prevalent CKD.2 Derived from the National Health and Nutrition Examination Survey (NHANES) 1999-2002 and (partially) validated in the Atherosclerosis Risk in Communities (ARIC) study, SCORED identified 9 demographic and medical variables that could be assigned integer values and then entered into a scoring algorithm. The scoring algorithm was intentionally simplified to be accessible to lay persons and health care providers.

To our knowledge, SCORED is the only algorithm derived from scientific modeling, rather than expert scientific opinion, targeted for general population screening. Alternatively, the National Kidney Foundation has encouraged screening strategies targeted at high-risk groups. Through the Kidney Early Evaluation Program (KEEP), the National Kidney Foundation recommends using the following characteristics to identify individuals with a high likelihood of kidney disease: “if a person is 18 years or older and has one or more of the following: diabetes; high blood pressure; or a family history of diabetes, high blood pressure or kidney disease.”3,4

It is unclear how the SCORED algorithm compares with the KEEP guidelines. Such comparisons would be useful for clinicians and others. Since the publication of SCORED, we have performed an additional validation study using new, independent samples: (1) NHANES 2003-2004 and (2) a combined cohort of the ARIC study and Cardiovascular Heart Study (ARIC/CHS). We report the findings herein.

Methods

The ARIC study enrolled 15 792 participants aged 45 to 64 years between 1987 and 1989, and CHS recruited 5201 subjects 65 years and older between 1989 and 1990. Both are community studies, and detailed descriptions have been published previously.2,5,6 Some data disparities and limitations to be noted are summarized in a footnote of the Table.

SCORED Model

The SCORED model is a multivariable mathematical function that gives an estimated probability of having CKD as follows:

Probability(CKD) = 1/[1 + exp(−β" × x)],

where

β" × x = −5.4 + 1.55 × I(Age 50-59 years) + 2.31 × I(Age 60-69 years) + 3.23 × I(Age ≥70 years) + 0.29 × I(Female) + 0.93 × I(Anemia) + 0.45 × I(Hypertension) + 0.44 × I(Diabetes) + 0.59 × I(History of CVD) + 0.45 × I(History of Heart Failure) + 0.74 × I(Peripheral Vascular Disease) + 0.83 × I(Proteinuria).

The total integer score is a weighted sum, ranging from 0 to 12, and is calculated as follows:

Total score = 2 × I(Age 50-59 years) + 3 × I(Age 60-69 years) + 4 × I(Age ≥70 years) + 1 × I(Female) + 1 × I(Anemia) + 1 × I(Hypertension) +1 × I(Diabetes) + 1 × I(History of CVD) + 1 × I(History of Heart Failure) + 1 × I(Peripheral Vascular Disease) + 1 × I(Proteinuria),

where I(A) is an indicator taking 1 for condition A and 0 for otherwise. If the total score is greater than or equal to 4, a confirmatory blood examination (eg, serum creatinine and/or urinalysis) by a physician is strongly recommended. A user-friendly questionnaire is also provided in the original article.2

Measures for Validation and Comparison

Creatinine level was calibrated in all studies as recommended by the National Kidney Disease Education Program and the National Center for Health Statistics.79 Kidney function was quantified by glomerular filtration rate from the 4-variable Modification of Diet in Renal Disease study equation, and CKD was defined as a glomerular filtration rate lower than 60 mL/min/1.73m2, which corresponds to stage 3 or higher kidney disease.3,10

The SCORED model was evaluated in the validation datasets using standard measures: percentage of people identified to be at high risk by the given rule, sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve (AUC).

Because the ARIC study and CHS did not measure urinary protein level for proteinuria, we created 3 different approaches to handling missing or incomplete information and assessed sensitivity of the estimates of the selected measures. First, we started the analysis with ARIC/CHS datasets without proteinuria, which reflects the situation that no one reported that she or he had proteinuria. Second, we substituted urinary protein level measured at the year 5 follow-up visit for CHS subjects if available or imputed the measurement via a missing data analysis method otherwise.11 Third, we defined the highest 10 percentiles of uric acid level as proteinuria (to compare, 9.5% of subjects had proteinuria in NHANES). (The rationale for using high uric acid level as a surrogate marker for proteinuria, at least for sensitivity checking, is that it is used in the diagnosis and treatment of renal disorders, including renal failure or injury and is associated with proteinuria.12)

We compared SCORED vs KEEP using the same set of diagnostic measures. It is important to note that none of the datasets we examined contained information on “family history of kidney disease,” a criterion needed for KEEP. As a result, the SCORED model could not examine the effect of this potentially important factor. The absence of this data was also a problem for the direct comparison between KEEP vs SCORED. To improve comparability, we repeated our analysis in 2 scenarios: (1) ignoring this criterion and (2) after applying the fill-in method based on missing data imputation and replacement by surrogate variables. These statistical strategies, although imperfect, would offer a sound method of sensitivity analysis in practice.

Results

For the NHANES dataset, sensitivity, specificity, positive predictive value, and negative predictive value of SCORED were estimated to be 95%, 65%, 20%, and 99%, respectively, with an AUC of 0.88. The application to ARIC/CHS provided sensitivity of 88% to 90%, specificity of 50% to 52%, positive predictive value of 13% to 14%, and negative predictive value of 98%, with an AUC of 0.78 to 0.80 (Table). In contrast, KEEP tended to place larger proportions of subjects into the high-risk category than SCORED (67%-77% vs 40%-53%), and the sensitivity for detecting CKD was 86% to 92%, with a specificity of 24% to 35%, positive predictive value of 3% to 12%, negative predictive value of 95% to 98%, and an AUC of 0.65 to 0.77. Results were robust under the different assumptions and approaches we adopted. Our comparisons give some indication about the expected performance of the 2 screening programs in real-world settings.

Comment

We report the diagnostic characteristics of the SCORED model and its performance compared with the KEEP guidelines. Ninety-five percent of individuals with CKD were identified by the SCORED algorithm as being at high risk (cutoff of ≥4); among those who did not have CKD, 65% scored less than 4 in NHANES. Those values were somewhat reduced in the ARIC/CHS samples, possibly owing to differences in population characteristics, variable definitions, and available data.

The SCORED model exhibited improved performances in all the criteria we examined. For KEEP as well as SCORED, NHANES showed better results than ARIC/CHS. Based on the higher AUC of the SCORED model compared with the KEEP guidelines, the improved performance characteristics are likely because of the use of additional variables and different weights for age groups. Indeed, questions about underlying CVD, common in the CKD population as demonstrated by the KEEP data,13 may improve their screening efforts.

There are also some limitations. Some variables that are a part of SCORED or KEEP were unavailable or incomplete in the databases we used, including family history of kidney disease and proteinuria. We addressed this issue with different statistical strategies. The absence of these variables in well-known studies may reflect low awareness of these important risk factors. We encourage the collection of such information in future epidemiological studies including the next iteration of NHANES. Furthermore, there are known negative consequences of screening. Overdiagnosis, misdiagnosis, anxiety, cost, and creating a false sense of security are some adverse effects of screening and render screening in some diseases controversial.1418 We propose that SCORED would minimize the potential adverse outcomes associated with screening by identifying approximately 25% fewer screeners as being at high risk compared with KEEP. The greater specificity of SCORED may reduce overdiagnosis and enhance resource use.

Screening is a public health strategy for identifying an unrecognized disease in asymptomatic populations, in which subjects are asked questions or offered a test, to identify those individuals who are more likely to be helped than harmed by further tests or treatment to reduce the risk of a disease or its complications. Diseases suitable for screening are those with serious consequences, those in which treatment is more effective at an earlier stage, and conditions with a long preclinical phase.16 Chronic kidney disease fulfills these criteria; however, it is not known whether screening will in fact result in improved outcomes and reduced progression to end-stage renal disease.17 The benefits of screening for CKD and its effectiveness and cost-effectiveness are yet to be determined.19

In conclusion, SCORED and KEEP are simple and inexpensive to use and carry some, albeit, minor, consequences of a false-positive result. Because of its general population sampling, we believe that SCORED has greater applicability and could also serve as an educational tool to raise CKD awareness. There is ample evidence that CKD is underdiagnosed and undertreated and that the burden of CKD is increasing worldwide.20 Not many people are aware that undiagnosed CKD can lead to serious consequences, including kidney failure, dialysis, and even death, and is associated with CVD. We propose that increased screening would identify greater numbers of individuals who may potentially benefit from strategies known to slow the progression of renal disease, a hypothesis that warrants testing in prospective clinical trials. We should “act on the best available evidence—as opposed to waiting for the best possible evidence.”21(p111)

Back to top
Article Information

Correspondence: Dr Bang, Division of Biostatistics and Epidemiology, Department of Public Health, Weill Medical College of Cornell University, 411 E 69th St, New York, NY 10021 (heb2013@med.cornell.edu).

Author Contributions: Dr Bang had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Bang, Mazumdar, Shoham, and Kshirsagar. Acquisition of data: Bang. Analysis and interpretation of data: Bang, Mazumdar, Kern, Shoham, August, and Kshirsagar. Drafting of the manuscript: Bang, Shoham, August, and Kshirsagar. Critical revision of the manuscript for important intellectual content: Bang, Mazumdar, Kern, and Kshirsagar. Statistical analysis: Bang, Mazumdar, and Shoham. Obtained funding: Mazumdar. Administrative, technical, and material support: Kern and Kshirsagar. Study supervision: August and Kshirsagar.

Financial Disclosure: None reported.

Funding/Support: The ARIC and CHS studies are conducted and supported by the National Heart, Lung, and Blood Institute (NHLBI) in collaboration with ARIC and CHS investigators. Drs Bang and Mazumdar were partially supported by the Tolly Vinik Trust through the Institute of Clinical Research of Weill Medical College of Cornell University. Dr Mazumdar was additionally supported by grant R25CA105012 (Collaborative Program in Nutrition and Cancer Prevention). Dr Shoham is a postdoctoral trainee supported by NHLBI institutional training grant 5T-32HL007055-29.

Disclaimer: This manuscript was prepared using a limited-access dataset obtained from the NHLBI and does not necessarily reflect the opinions or views of the ARIC/CHS or NHLBI.

Additional Contributions: Kylie Braynt, MS, provided CHS data setup and Sean Coady, MA, at NHLBI assisted in data issues and clarifications. We thank the staff and participants of the NHANES, ARIC study, and CHS for their important contributions and valuable information for health research.

References
1.
Coresh  JByrd-Holt  DAstor  BC  et al.  Chronic kidney disease awareness, prevalence, and trends among US adults, 1999 to 2000. J Am Soc Nephrol 2005;16 (1) 180- 188
PubMedArticle
2.
Bang  HVupputuri  SShoham  DA  et al.  SCreening for Occult REnal Disease (SCORED). Arch Intern Med 2007;167 (4) 374- 381
PubMedArticle
3.
National Kidney Foundation, K/DOQI clinical practice guidelines for chronic kidney disease. Am J Kidney Dis 2002;39 (2) ((suppl 1)) S1- S266
PubMedArticle
4.
National Kidney Foundation, Kidney Early Evaluation Program. Am J Kidney Dis 2005;45 (2) ((suppl 2)) S1- S135
PubMed
5.
The ARIC investigators, The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. Am J Epidemiol 1989;129 (4) 687- 702
PubMed
6.
Fried  LPBorhani  NOEnright  P  et al.  The Cardiovascular Health Study: design and rationale. Ann Epidemiol 1991;1 (3) 263- 276
PubMedArticle
7.
Coresh  JAstor  BCMcQuillan  G  et al.  Calibration and random variation of the serum creatinine assay as critical elements of using equations to estimate glomerular filtration rate. Am J Kidney Dis 2002;39 (5) 920- 929
PubMedArticle
8.
Myers  GLMiller  WGCoresh  J  et al.  Recommendations for improving serum creatinine measurement. Clin Chem 2006;52 (1) 5- 18
PubMedArticle
9.
Weiner  DETighiouart  HAmin  MG  et al.  Chronic kidney disease as a risk factor for cardiovascular disease and all-cause mortality: a pooled analysis of community-based studies. J Am Soc Nephrol 2004;15 (5) 1307- 1315
PubMedArticle
10.
Levey  ASGreen  TKusek  JBeck  GMDRD Study Group, A simplified equation to predict glomerular filtration rate from serum creatinine [abstract]. J Am Soc Nephrol 2000;11(suppl)155A
11.
SAS Institute Inc, SAS/STAT User's Guide Version 8.2.  Cary, NC SAS Institute Inc2000;
12.
Lee  JEKim  Y-GChoi  Y-HHuh  WKim  DJOh  HY Serum uric acid is associated with microalbuminuria in prehypertension. Hypertension 2006;47 (5) 962- 967
PubMedArticle
13.
McCullough  PAJurkovitz  CTPergola  PE  et al.  Independent components of chronic kidney disease as a cardiovascular risk state. Arch Intern Med 2007;167 (11) 1122- 1129
PubMedArticle
14.
McClellan  WMRamirez  SPBJurkovitz  C Screening for chronic kidney disease: unresolved issues. J Am Soc Nephrol 2003;14 (7) ((suppl 2)) S81- S87
PubMedArticle
15.
Gansevoort  RTBakker  SJLde Jong  PE Early detection of progressive chronic kidney disease: is it feasible? J Am Soc Nephrol 2006;17 (5) 1218- 1220
PubMedArticle
16.
Wilson  JJungner  G The Principles and Practice of Screening for Disease.  Geneva, Switzerland World Health Organization1968;
17.
Hallan  SIDahl  KOien  CM  et al.  Screening strategies for chronic kidney disease in the general population. BMJ 2006;333 (7577) 1047
PubMedArticle
18.
Freedman  DAPetitti  DBRobins  JM On the efficacy of screening for breast cancer. Int J Epidemiol 2004;33 (1) 43- 55
PubMedArticle
19.
Bang  H Medical cost analysis: application to colorectal cancer data from the SEER Medicare database. Contemp Clin Trials 2005;26 (5) 586- 597
PubMedArticle
20.
Meguid El Nahas  ABello  AK Chronic kidney disease: the global challenge. Lancet 2005;365 (9456) 331- 340
PubMedArticle
21.
Kaplah  JPedLiverman  CTedKraak  VIed Preventing Childhood Obesity: Health in the Balance.  Washington, DC Institute of Medicine2004;
×