SCreening for Occult REnal Disease (SCORED)A Simple Prediction Model for Chronic Kidney Disease

Background  Despite the wide availability and low cost of serum creatinine measurement, at-risk populations are not routinely tested for chronic kidney disease (CKD).

Methods  We used a cross-sectional analysis of a nationally representative, population-based survey to develop a system, SCORED (SCreening for Occult REnal Disease), that uses routinely available demographic and medical information to identify individuals with an increased likelihood of CKD. The analysis included 8530 adult participants in the National Health and Nutrition Examination Surveys conducted from 1999 to 2000 and 2001 to 2002 in the United States. Chronic kidney disease was defined as a glomerular filtration rate less than 60 mL/min per 1.73 m². Univariate and multivariate associations between a comprehensive set of risk factors and CKD were examined to develop a prediction model. The optimal characteristics of the model were examined with internal measures. External validation was performed using the Atherosclerosis Risk in Communities study. A model-based numeric scoring system was developed.

Results  Age (P<.001), female sex (P = .02), and various health conditions (hypertension [P = .03], diabetes [P = .03], and peripheral vascular disease [P = .008]; history of cardiovascular disease [P = .001] and congestive heart failure [P = .04]; and proteinuria [P<.001] and anemia [P = .003]) were associated with CKD. The multivariate model was well validated in the internal and external data sets (area under the receiver operating characteristic curve of 0.88 and 0.71, respectively). A score of 4 or greater was chosen by internal validation as a cutoff point for screening based on the diagnostic characteristics (sensitivity, 92%; specificity, 68%; positive predictive value, 18%; and negative predictive value, 99%).

Conclusion  This scoring system, weighted toward common variables associated with CKD, may be a useful tool to identify individuals with a high likelihood of occult kidney disease.

identification of individuals with chronic kidney disease (CKD) should be simple given the wide availability and low cost of serum creatinine measurement. However, during the past 2 decades, studies have demonstrated that at-risk populations are not routinely tested^1- 4 for CKD. As recently as 2003, only 22% of individuals with diabetes mellitus and 28% of individuals with hypertension underwent measurement of serum creatinine levels.⁵ Not surprisingly, awareness of CKD remains low,^6,7 even among family members of patients with end-stage kidney disease (ESKD),⁸ and the proportion of individuals with new CKD identified at or near ESKD has not significantly declined during the last 15 years.^9- 12

Detection of CKD at earlier stages of disease offers the opportunity to initiate therapies known to attenuate progressive nephropathy.^13- 19 Furthermore, detection of occult CKD may also help attenuate the large burden of cardiovascular morbidity and mortality.²⁰ Treating individuals with early CKD has the potential to delay ESKD by almost 2 years²¹ among young and middle-aged individuals.

Given the difficulty of identifying individuals with CKD and the known benefits of treatment, we sought to develop a simple method to prompt health care professionals and laypersons to screen for kidney disease. We had 2 requirements for this model-based system: (1) the use of routinely available and minimally intrusive demographic and medical variables that are understood by laypersons and health care professionals and (2) the use of variables that cumulatively affect prevalent CKD.^22,23

The National Health and Nutrition Examination Surveys (NHANESs) are national surveys conducted since 1975 by the National Center for Health Statistics of the Centers for DiseaseControl and Prevention. Participants in NHANES are identified through a complex, multistage clustering sample design of the civilian noninstitutionalized population. We combined data from 2 independent surveys, NHANES 1999-2000 and 2001-2002, available on a public domain Web site (http://www.cdc.gov/nchs/nhanes.htm). For our analysis, we restricted the NHANES population to men and women 20 years or older.

The NHANES used trained personnel to ascertain medical and health information from participants via direct interview, examination, and blood samples. We chose comprehensive demographic and clinical variables as potential determinants of CKD based on the literature.^22- 24 These variables included age, sex, race, marital status, anemia, low-density lipoprotein cholesterol, high-density lipoprotein cholesterol, triglycerides, hypertension, diabetes mellitus, peripheral vascular disease, history of cardiovascular disease, history of congestive heart failure, proteinuria, smoking status, physical activity, body mass index (calculated as weight in kilograms divided by the square of height in meters), educational and income levels, and health insurance status. A complete description of the definitions is available on request from the corresponding author.

Serum creatinine concentration was determined by the modified kinetic Jaffe method. Glomerular filtration rate (GFR) was estimated using the abbreviated Modification of Diet in Renal Disease formula:

GFR (mL/min per 1.73 m²) = 186 × Serum Creatinine (mg/dL)^–1.154 × Age (years)^–0.203 × 1.212 (If Black) × 0.742 (If Female).

An adjustment factor was used to align the NHANES serum creatinine values to the creatinine assay used to develop the Modification of Diet in Renal Disease formula.²⁵ For the 1999-2000 NHANES data set, 0.13 was added to the serum creatinine measurement.⁶ The adjustment factor for the 2001-2002 group is +0.02.²⁶

Kidney disease was defined as a GFR less than 60 mL/min per 1.73 m². This range corresponds to stage 3 or higher CKD by the National Kidney Foundation's classification scheme and helps identify individuals with clinically significant CKD.^22,27

The split-sample method was used for risk equation and score development and internal validation.^28,29 Eligible participants from the data set were randomly allocated to development (67%) and validation (33%) sample sets. Logistic regression was used to create a prediction model in the development data set.

We first analyzed the univariate associations between the independent variables and CKD using participants in the development data set. For multivariate modeling, the same covariates were considered the main effects. We used the backward elimination technique to reach the final model, in which factors with the largest P value are deleted one at a time until all the predictors in the model are significant at P<.05. We also tested 2-way interactions of significant prognostic factors in the final multivariate regression with age, sex, and race.

Once the most parsimonious model was defined, we tested diagnostic properties via the validation data set. Using the regression coefficients in the risk function, we estimated the patient-specific probability of having CKD and established a rule to characterize different degrees of risk based on cutoff points of the probability distribution.

A numerical scoring scheme was derived by rounding up the estimates of the corresponding regression parameters obtained from the same model (to the smallest integer that was greater than the estimate). This method is based on β-coefficients (or log of odds ratios) rather than odds ratios, which can be excessively influenced by only a few factors.²⁹

The prediction models were evaluated in the validation data set based on several measures: percentage of positive cases, sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve (AUC). We also estimated 95% confidence intervals for diagnostic characteristics.^30,31

Performance of the prediction model was evaluated in an independent data set, the Atherosclerosis Risk in Communities (ARIC) study. Between 1987 and 1989, the ARIC study recruited a population-based cohort of 15 792 men and women 45 to 64 years of age from 4 US communities. A detailed description of the ARIC study design has been published.³² Variables in the ARIC study were defined as closely as possible to the NHANES variables. We included all participants who were present at the baseline visit and had complete covariate information (N = 12 096). We used a constant of −0.22 derived from a method of indirect calibration of serum creatinine values from the ARIC–Life Course Socioeconomic Status cohort using NHANES III data.³³

We conducted various analyses to evaluate the validity and robustness of the prediction model that we developed. First, we repeated the analysis after omitting 2 variables that may not be readily available without the involvement of health care personnel: (1) peripheral vascular disease (derived from ankle brachial index) and (2) hemoglobin level (which is a part of the definition of anemia). Second, we ascertained the AUC from unweighted analyses. Third, we reran the same model after excluding the patients with a GFR less than 15 mL/min per 1.73 m² (n = 17), which is regarded as kidney failure or stage 5 chronic kidney disease. Fourth, we repeated model derivation and validation by 100 different selections of random splits. Fifth, we ran the model excluding individuals with proteinuria from the development set and eliminating proteinuria as an independent variable.

All analyses were performed using survey procedures in SAS statistical software, version 9.1, for correct weighted analysis (SAS Institute Inc, Cary, NC). To this end, options of strata, cluster, and weight (4 years) were used. Two-sided hypotheses and tests were adopted for all statistical inferences.

Combining NHANES 1999-2000 and 2001-2002 resulted in a data set with 10 291 individuals who were at least 20 years of age. The final data set consisted of 8530 observations after excluding individuals with missing serum creatinine measurements (n = 1472) and other missing covariates (n = 289).

Important characteristics of the study population and its univariate association with kidney disease are presented in Table 1. A total of 601 of 8530 participants (weighted proportion, 5.4%) had kidney disease. Multivariable modeling demonstrated that only 9 variables had statistically significant associations with kidney disease in the development data set (Table 2). The numeric values assigned to each of these final variables reflect the magnitude of the log of the odds ratio.

Table 3 gives the performance of the prediction model in the validation data set. The sensitivity and specificity of the model changed with increasing prevalence of kidney disease. Varying the cutoff point of the total score also changed the sensitivity, specificity, positive predictive value, and negative predictive value of the prediction model. At one extreme, for a score of 6 or higher, the sensitivity was 68% and the specificity was 87%; at the other extreme, for a score of 3 or higher, the sensitivity was high (96%) but the specificity was low (58%). The negative predictive value remained uniformly high (≥97%) for various scenarios. A prediction score of 4 or higher was chosen to be the rule underlying the screening guideline based on both diagnostic and qualitative criteria and practical implementation considerations; cutoff points of 5 and 4 give the comparable values of the Youden index, 0.62 vs 0.60, respectively, whereas a cutoff point of 4 offers significantly higher sensitivity.³⁴Figure 1 shows the unweighted and weighted proportions of people with each score with concurrent CKD.

Minimal attenuation was found in the accuracy measure of the prediction model with the omission of peripheral vascular disease and hemoglobin level (AUC = 0.87 vs 0.88). The same analysis after excluding the patients with GFRs less than 15 mL/min per 1.73 m² or analysis without weighting resulted in the same AUC. Replication of 100 different random splits using the same ratio yielded the same scoring rule, as determined by the median value of individual scores for 9 factors. In addition, subgroup analyses by race and sex yielded identical or higher values for AUC, in the range of 0.88 to 0.91. Elimination of individuals with proteinuria from the data set changed the model fit slightly (AUC = 0.878).

Model fits from the ARIC study and NHANES were highly consistent. A few major differences between NHANES and the ARIC study should be noted: (1) age ranged from 45 to 65 years in the ARIC study as opposed to 20 to 85 years in our NHANES analysis, (2) proteinuria or microalbuminuria information was not collected in the ARIC study, (3) in the ARIC study, medication use for heart failure was ascertained only for the past 2 weeks, thereby resulting in a low prevalence (0.6%) and power, and (4) NHANES data were collected from 1999 through 2001, whereas the ARIC study visit 1 data were collected from 1987 through 1989. Table 4 indicates that the AUC is 0.71. We suspect that this lower AUC is primarily due to these data set differences, especially the difference in the age range of the 2 data sets.

We have developed and validated a systematic method to screen for kidney disease from a well-defined population sample. The model-based system makes use of a parsimonious set of medical and demographic characteristics to identify individuals with a high likelihood of CKD before any evaluation with serum laboratory analysis. These characteristics are often present concurrently and cumulatively affect underlying kidney disease. Furthermore, age, hypertension, diabetes mellitus, cardiovascular disease (divided into coronary artery disease, congestive heart failure, and peripheral vascular disease), proteinuria, and anemia are easily identified by the general public and health care professionals. Using a cutoff score of 4 or higher, this model demonstrates a high sensitivity and negative predictive value of 92% and 99%, respectively. The specificity and positive predictive value are admittedly low. Only 18% of patients with scores of 4 or higher will have CKD. However, the potential financial and psychological consequences are arguably minimal. Confirmatory testing (serum creatinine measurement) is inexpensive and reliable and does not require invasive or time-consuming measurements.

This instrument could serve as an antecedent screening test that would enhance the pretest probability of developing CKD and complement existing formulas that estimate GFR based on serum creatinine levels. We envision a broad range of potential scenarios in which the model may be applied: (1) mass screenings sponsored by governmental and nongovernmental agencies, (2) private and public primary care clinics, (3) medical emergency departments, (4) public education initiatives, and (5) interactive, Web-based medical information sites. A sample questionnaire is presented in Figure 2. Among individuals scoring 4 or higher in any of these settings, confirmatory testing could then be obtained using a common and relatively inexpensive measurement, serum creatinine concentration.

We purposefully chose to define CKD for the prediction equation using a GFR of less than 60 mL/min per 1.73 m² rather than less than 90 mL/min per 1.73 m² for 2 reasons. First, we wanted to minimize the detection of individuals with an age-related physiological decline in kidney function. Second, the Modification of Diet in Renal Disease estimation formula was derived among individuals with a baseline GFR of less than 60 mL/min per 1.73 m² and is most accurate for individuals with a GFR in this range.³⁵

Currently, no other systematic methods exist that predict prevalent or incident CKD.³⁶ Clinical practice guidelines (evidence based and expert opinion) for the treatment of CKD^22,27,37 recommend regular screening of individuals with risk factors for CKD, such as diabetes mellitus, hypertension, family history of kidney failure, or concurrent cardiovascular diseases. These recommendations focus on single risk factors and do not quantify the cumulative effect of multiple risk factors. However, individuals often present for evaluation with multiple comorbid conditions that may each contribute additively to the presence of CKD. Our method makes use of multiple concurrent risk factors for CKD. Future screening programs for CKD will focus on multiple risk factors in both the general population and those at high risk.

Practice patterns suggest that recommendations for evaluation of kidney disease are not routinely followed.³⁸ For example, data from the United States suggest that most primary care practices screen less than 20% of their diabetic Medicare patients for the presence of kidney disease.^38- 40 Even among individuals with known risk factors for CKD, kidney disease may be underrecognized.^5,41

Unexpectedly, we found that female sex but not race was associated with prevalent CKD. Although the racial differences in incident and prevalent ESKD are well documented,⁴² some of the racial differences observed in the prevalence of CKD may be due to differences in the rate of progression among black vs white patients. In the NHANES III and NHANES 1999-2000 data, the black population had a lower age-adjusted prevalence of CKD than the white population.⁶ In the NHANES 1999-2002 data used in this study, the prevalence of CKD was similarly higher among white compared with black patients. Baseline results from the Racial Differences in the Prevalence of Chronic Kidney Disease among Participants in the Reasons for Geographic and Racial Differences in Stroke cohort support these findings.⁴³ One possible explanation may be that black patients progress more rapidly from early stages of CKD to ESKD. However, the cross-sectional nature of these data limits any speculation on this hypothesis.

Our study has some limitations. The model is heavily weighted toward the common risk factors for kidney disease, advanced age, diabetes mellitus, and hypertension, as well as toward comorbid cardiovascular disease and anemia. Weighting has 2 important consequences. First, a high proportion of elderly individuals will be identified, especially if such individuals are older than 70 years. We specifically chose a GFR outcome of less than 60 mL/min per 1.73 m² rather than less than 90 mL/min per 1.73 m² in recognition of the physiological changes in renal function occurring with age. Nevertheless, elderly individuals represent the fastest-growing segment of the ESKD population.⁴⁴ identifying such individuals not only would allow the implementation of therapies to delay progressive CKD but also may facilitate long-term discussions about the feasibility and practicality of dialytic therapy, a decision of last choice.

Second, weighting toward common risk factors may prevent effective screening for autosomal dominant polycystic kidney disease and glomerulonephritis with this model. However, most cases of autosomal dominant polycystic kidney disease are nonsporadic, and families with this condition are often aware of their inherited risk of kidney disease and frequently seek medical advice. Glomerulonephritides include a disparate group of diseases with the protean clinical findings of hematuria, proteinuria, and hypertension. The prediction rule includes a variable for proteinuria (albuminuria) and hypertension but lacks any measurement of hematuria. Individuals with underlying glomerular disease often become symptomatic, especially with edema, prompting them to seek medical care.

Another limitation is the inability to determine family history of kidney disease. Family history of ESKD may modify the effect of diabetes mellitus and hypertension.^45,46 Currently, only the Kidney Early Evaluation Program,^47,48 targeted at populations at high risk for ESKD, surveys the impact of family history during its community-based screenings. The addition of family history of kidney disease to the next iteration of NHANES and other data sets would allow investigators to better understand its contribution to CKD and ESKD.

The cross-sectional nature of this study should also be noted. A low score does not rule out the possibility of developing CKD in the future. A prediction model for incident disease is an important next step, and our research group is currently investigating such methods. However, the scoring system is able to identify individuals who, based on their current condition, should receive screening for CKD. It may also prompt individuals and health care professionals to perform simple tests (such as an office urine dipstick) to assess variables such as proteinuria. The current scoring system may underestimate CKD because self-reported levels of proteinuria may not be reliable and are often not assessed, even among high-risk populations.⁸ Furthermore, updating scores as component conditions change over time will alert individuals and health care professionals to the need for further screening.

Finally, we used a statistical method that was incapable of investigating complicated effect modifications among the risk and protective factors. For such interactions, classification tree regression may be better suited,^49,50 yet no tree-based algorithms for complex sampling design exist. Of particular note, no significant interactions of the covariates were found with age, sex, or race when testing the 2-way interactions. Subgroup analyses by sex and race also yielded highly similar AUCs.

Several strengths of the analysis should be noted. First, a broad representation of sex, ethnic and racial groups, age, and low income levels was achieved by weighted sample design. Second, the large sample size afforded us the power to conduct subset and sensitivity analyses and added to the robustness of the findings. Third, we were able to validate the prediction model in a large, independent community-based data set (ARIC study). Although we were limited by some unavoidable data conditions, we reached reasonably consistent results, providing further validation to our prediction model.

In summary, we have developed SCORED (SCreening for Occult REnal Disease), a system to prompt health care professionals and laypersons to consider underlying kidney disease. The timely detection of CKD can benefit patients with ESKD and society from the burgeoning costs of the disease. In the future, we plan to test SCORED in several settings, including a community-based screening program.

Correspondence: Abhijit V. Kshirsagar, MD, MPH, University of North Carolina, Campus Box 7155, Room 7017, Burnett-Womack Hall, Chapel Hill, NC 27599-7155 (sagar@med.unc.edu).

Accepted for Publication: November 2, 2006.

Author Contributions: Dr Bang had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Bang, Shoham, Klemmer, Falk, Mazumdar, Gipson, Colindres, and Kshirsagar. Acquisition of data: Bang, Mazumdar, and Kshirsagar. Analysis and interpretation of data: Bang, Vupputuri, Shoham, Mazumdar, and Kshirsagar. Drafting of the manuscript: Bang, Vupputuri, Shoham, Mazumdar, Gipson, and Kshirsagar. Critical revision of the manuscript for important intellectual content: Bang, Vupputuri, Shoham, Klemmer, Falk, Mazumdar, Gipson, and Colindres. Statistical analysis: Bang, Shoham, and Mazumdar. Obtained funding: Falk. Administrative, technical, and material support: Kshirsagar. Study supervision: Klemmer, Gipson, Colindres, and Kshirsagar.

Financial Disclosure: None reported.

Funding/Support: The ARIC study is conducted and supported by the National Heart, Lung, and Blood Institute (NHLBI) in collaboration with the ARIC study investigators.

Disclaimer: This article was prepared using a limited access data set obtained from the NHLBI and does not necessarily reflect the opinions or views of the ARIC study investigators or the NHLBI.

Acknowledgment: We thank the staff and participants of NHANES and the ARIC study for their important contributions. We also thank Lisa Kern, MD, at Weill Medical College for her invaluable guidance in understanding the NHANES data sets and intellectual discussions. Without her, this article would have been significantly delayed. Finally, we appreciate the help of Anita Mesi, BA, in generating a figure and Sean Coady, BA, at the NHLBI for his assistance with the ARIC study data set.

This article was corrected for typographical errors on 2/19/07.