Receiver operating characteristic curves showing the weighted (area under the curve = 0.885) and unweighted (area under the curve = 0.872) models.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Nguyen MT, Herrine SK, Laine CA, Ruth K, Weinberg DS. Description of a New Hepatitis C Risk Assessment Tool. Arch Intern Med. 2005;165(17):2013–2018. doi:10.1001/archinte.165.17.2013
Because of the low prevalence of hepatitis C virus (HCV) infection in the general population, mass screening would be expensive and of low yield. Some researchers advocate targeted screening of persons at elevated HCV risk.
This cross-sectional study aimed to develop a patient-administered tool to assess HCV infection risk. Two hundred seven patients with unknown HCV status from a general medicine practice and 222 HCV-positive patients from a hepatology practice completed a 72-item survey about demographic, social, and clinical risk factors for HCV infection. General medicine patients also underwent HCV serologic testing.
Three (1.5%) of 207 general medicine patients had positive HCV antibody test results. These patients plus the 222 hepatology patients were significantly more likely than HCV-negative patients to report an array of factors. In a multivariable model, 7 factors remained significantly associated with HCV infection: sex with a prostitute or an injecting drug user, exposure to blood products, refusal as a blood donor or as a life insurance applicant, witnessing illicit drug use, and self-reported HBV infection. A simplified model that assigned 1 point for each factor present predicted HCV infection as well as a weighted model (based on χ2 testing and receiver operating characteristic curve comparison). In a population with a 2% prevalence of HCV infection, people who identified 2 risk factors had a 10% chance of HCV infection, whereas those with 4 or more risk factors had a 50% chance.
A self-administered 72-item questionnaire can stratify patients into HCV risk groups. If validated in other primary care populations, this instrument could help target HCV screening.
Hepatitis C virus (HCV) infection, a leading cause of cirrhosis, hepatocellular carcinoma, and liver transplantation, is the most common cause of hepatic morbidity and mortality in the United States.1,2 Most infected persons are asymptomatic, and fewer than 25% of the estimated 4 million Americans with chronic HCV are aware of their status. A limited understanding of asymptomatic infection has led clinicians to underappreciate HCV as an explanation for chronic liver disease.3,4 Physicians may not ask patients about, or patients may hesitate to disclose, behaviors that increase risk of HCV. The incentive for case identification is further reduced by the perception that available treatments are often intolerable or ineffective. Although demonstrated benefit from the treatment of asymptomatic HCV infection is lacking, clinical outcomes seem to be improved when therapy is initiated in younger patients and at earlier stages of disease.5,6 A major public health challenge is more effective identification of patients with HCV before the development of important clinical sequelae. Previous case-finding efforts have concentrated on high-risk groups or volunteer blood donors.7-13 In community practice, few health care providers target screening toward healthy persons at higher risk for infection.14-16
Because of the relatively low prevalence (approximately 1.8%) of HCV in the general population,2 mass serologic screening for HCV infection would be prohibitively expensive. In 2004, the US Preventive Services Task Force (USPSTF) recommended against routine screening of adults without identified risk factors for HCV, citing the low prevalence of infection, the knowledge that most infected individuals will not develop serious HCV-related disease, and the current absence of evidence that screening improves outcomes.17 The USPSTF also found insufficient evidence to recommend either for or against routine screening of adults at high risk for infection. The systematic review18 that provides background for the USPSTF recommendations concluded that there have been no published efforts investigating the predictive accuracy of screening strategies applied to the general population.
Development of an acceptably sensitive risk assessment questionnaire might facilitate cost-effective screening for HCV. Herein, we describe the derivation process and preliminary performance characteristics of a patient self-administered HCV risk assessment tool.
Based on a literature review regarding HCV risk factors and expert opinion, we developed a 72-item survey that inquires about a range of demographic, social, and clinical factors potentially associated with HCV infection (the survey instrument is available on request). Information obtained from all the participants included age, race, sex, education and income levels, country of origin (United States vs other), and citizenship (United States vs other). Questionnaire domains exploring HCV risk associations included personal and family history of liver and other common diseases, overall health status, occupational history, and lifestyle and behavioral characteristics. The questionnaire also included well-described risk factors (eg, injecting drug use) identified previously. Because our goal was overall HCV risk assessment, we asked more general questions covering a range of potential risk factors, for example, “Have you ever been homeless?” and “Have you ever been imprisoned?” Most questions were posed in a yes/no format.
Before survey administration, we pilot tested the questionnaire with 10 patients with and 10 patients without known HCV infection who fulfilled the study’s participation criteria. We made minor adjustments to improve wording clarity and item flow in response to these initial surveys. Pilot testing responses were excluded from subsequent analysis, and patients who participated were not eligible for later study participation.
After the Thomas Jefferson University institutional review board approved the study, 2 patient groups were approached for possible study enrollment. The first group comprised patients with known HCV infection who received ongoing hepatology care in the Thomas Jefferson University gastroenterology division. The second group was patients who received care in the general internal medicine division at Thomas Jefferson University, were free of clinically apparent liver disease, and had never been tested for HCV infection, to their or their physicians’ knowledge. We excluded general medicine patients with known HCV status or clinically apparent liver disease because screening would not apply to such patients. Participants were required to be aged 18 to 60 years, able to provide consent, and able to complete the written, English-language survey instrument. Research personnel approached consecutive patients until predetermined sample size requirements were met.
On granting consent, each participant received a unique identification number and brief instructions about the written, self-administered survey. All questionnaires were completed on-site and were returned to clearly marked, opaque boxes in either clinic setting. To minimize the underreporting of illicit or embarrassing behavior for respondents who might be uncomfortable providing candid answers under normal circumstances, participant anonymity was maintained throughout the study. In no instance could treating physicians link questionnaire responses or HCV test results to individual study participants.
To ascertain infection status, all general internal medicine participants agreed to undergo HCV antibody testing. To further ensure anonymity, testing was performed at home using a commercially available kit (Home Access Health Corp, Hoffman Estates, Ill). This test system is accurate, with approximately 5% of users being unable to provide adequate samples for testing.19 Following the manufacturer’s instructions, participants provided a small blood sample and mailed it to a central developing location. Participants were provided instructions to acquire a secure password and code to obtain their results anonymously by telephone. Patients with positive HCV test results were provided appropriate counseling about their test results. It was left to the participant’s discretion whether to share test results with a health care provider. All HCV test results were also linked to the participants’ study-specific identification numbers. Questionnaire responses from persons found to be HCV positive were ultimately included in the data analysis along with responses from participants who were HCV positive at study entry.
A computerized database was developed that included the study identification numbers, questionnaire responses, and HCV results for all the participants. Data were analyzed using statistical software (SAS; SAS Institute Inc, Cary, NC). Patients with missing questionnaire responses were excluded from the analyses of that specific item; most items had response rates of 95% or more, minimizing the impact of missing data.
Model development consisted of 2 steps. The first step was to determine the association between HCV status and each questionnaire item using the Fisher exact test, except for questions with ordered responses, where the Wilcoxon rank sum test was used. Only questions with significant differences in responses (P<.01) between HCV-positive and HCV-negative groups at the bivariate level were considered as possible predictors of HCV infection status. Using this set of potential predictors, stepwise logistic regression was performed to identify factors predictive of HCV status (positive or negative). We retained in the model only items associated with HCV status at P<.01 after adjusting for other items in the model. After item selection was complete, we reran the model including only selected items to reduce the number of records excluded for missing values.
In addition to deriving a regression model, in which the relative contribution of each predictor variable was weighted based on estimates of its associated odds ratio (OR), we also constructed a second model in which each variable was weighted equally. In this second model, the number of “higher-risk” responses to the predictor variables was simply totaled, with greater sums denoting greater risk for infection. The overall similarity between the 2 models was compared via a likelihood ratio test. For either model, ORs and associated 95% confidence intervals (CIs) for HCV carrier status were determined from the logistic regression coefficients. Finally, we constructed receiver operating characteristic curves to compare the predictive abilities of the 2 models (separate question weighting vs the total of higher-risk responses). The c statistic, which measures the area under the receiver operating characteristic curve and provides an estimate of predictive ability, was calculated for each model.
Although the sensitivity and specificity of any test depend on the data set used to construct the model, the positive predictive value, in this case, the proportion of persons predicted to be HCV positive (based on the model) who are actually positive, hinges on the population prevalence of the target illness. Therefore, in addition to assessing the model’s test characteristics, we also calculated the positive predictive value using several different potential population prevalence rates for HCV infection.
Of 290 patients without previously identified HCV infection who completed the study questionnaire, 220 (75.9%) also completed HCV antibody testing. Participants who completed HCV testing were demographically similar to those who did not. Thirteen patients were excluded because their blood results were indeterminate or not testable. Three (1.5%) of the remaining 207 patients were HCV positive. Questionnaire results for these 3 individuals are included with the responses of the 222 known HCV-positive participants from the hepatology clinic at Thomas Jefferson University. All subsequent analyses used only data where questionnaire results and HCV status were known.
Table1 provides various characteristics of the study participants based on HCV infection status. Participants with HCV were more likely to be male than HCV-negative participants. Older age and lower terminal points of education were associated with infection. The HCV-positive population was less likely to characterize their general health status favorably compared with HCV-negative patients. Patients with HCV were also more likely than uninfected individuals to know of other family members with HCV infection. As would be predicted based on accepted risk factors, HCV-positive participants were more likely to have used injecting drugs, received a blood transfusion, been refused life insurance, and been refused as blood donors. Table 1 also demonstrates the frequencies of various medical illnesses reported by participants. Individuals with HCV were statistically more likely to be coinfected with hepatitis B virus (HBV), to have reported an episode of jaundice, to have undergone major surgery, and to require kidney dialysis.
A variety of personal behaviors or other factors were significantly associated with HCV infection. Except for having a tattoo, the remainder of the factors centered on illicit behaviors: a history of arrest or imprisonment, the exchange of money for sexual activity, the use or at least observation of illegal drug use, and sex with a person who used injecting drugs. These associations reflect a range of behaviors that might result in viral infection.
In the initial logistic regression model, a separate indicator variable was included for each of the 7 questionnaire items. A self-reported history of sex with a prostitute was the strongest risk factor (OR, 9.76; 95% CI, 2.50-38.13). Other predictors included a history of exposure to potentially infected blood during a transfusion (OR, 8.62; 95% CI, 4.71-15.80), rejection as a blood donor (OR, 2.57; 95% CI, 1.49-4.43), being refused life insurance (OR, 2.75; 95% CI, 1.05-7.25), witnessing the illicit use of injecting drugs (OR, 5.83; 95% CI, 2.93-11.58), having sexual intercourse with an injecting drug user (OR, 5.39; 95% CI, 2.01-14.42), and self-report of HBV infection (OR, 4.71; 95% CI, 1.49-14.83).
In comparison, Table 2 provides the results of the unweighted model. Using the same 7 items as in the previous paragraph, this model assigns 1 point for each factor present and 0 if a factor is not present. As a result, each participant could receive a total score of 0 to 7, with higher scores suggesting greater likelihood of infection. Risk elevations were similar when 4 or more risk factors were present (data not shown). Therefore, for further simplification, the model assigned any patient with 4 or more risk factors to a single group for analysis.
The predictive capacities of the weighted and unweighted models did not differ (likelihood ratio test χ26 = 9.32; P = .16). Receiver operating characteristic curves for the 2 models had similar high areas under the curve, with a possible range of 0.5 for no predictive ability to 1.0 for perfect predictive ability. Visually, the curves are almost totally superimposed (Figure).
Because the simplified model requires no mathematical calculation, clinicians would likely find it more appealing than the weighted model, so we based the calculations in Table 3 and Table 4 on the unweighted model. Table 3 describes the sensitivity and specificity of the unweighted model, whereas Table 4 displays the likelihood of HCV infection depending on the number of positive responses when the unweighted prediction model is used in populations with differing HCV prevalence. For example, when the population prevalence of HCV infection is 2%, a person who identifies 2 risk factors for infection has approximately a 10% chance of HCV infection, whereas someone with 4 or more risk factors has a likelihood of nearly 50%. In populations in which the prevalence of infection is estimated to be 1%, similar risk levels would generate a likelihood of infection of 5% and 33%, respectively.
We identified a small set of patient-reported factors that can be used to estimate risk of HCV infection. Our HCV prediction model may prove useful in efforts to target HCV screening toward higher-risk patients. More effective identification strategies will become important should ongoing studies find a benefit from the treatment of asymptomatic HCV.
Better treatment options might also affect some of the ongoing debate regarding widespread HCV screening.18,20 Previous attempts to increase the efficiency of screening have focused on the identification of high-risk groups. Unlike the USPSTF, the National Institutes of Health Consensus Panel and the Centers for Disease Control and Prevention advocate screening in selected populations, for example, injecting drug users, hemodialysis patients, and recipients of blood transfusions or organ transplants.21,22 The Centers for Disease Control and Prevention and the National Institutes of Health Consensus Panel also recommend screening other various, nonoverlapping groups. The lack of unanimity underscores the difficulties in interpreting the research data on many potential factors associated with infection. This absence of consensus contributes to confusion about screening, especially in the primary care setting.
Perhaps reflecting these issues, the recent USPSTF clinical guidelines concluded that population-based screening is not currently warranted.18 The task force examined 4 large observational, population-based studies of HCV risk factors.23-26 None used a selective screening tool suitable for the prospective screening of asymptomatic populations.18 The task force pointed out the need for studies examining the usefulness of risk factor assessment for HCV infection.
Individuals in favor of routine screening for persons at higher infection risk argue that screening could allow more frequent and effective treatment of the virus,20 although, to our knowledge, no studies currently exist comparing the clinical outcomes of patients screened and not screened for HCV infection. Screening might allow monitoring and counseling of persons with synergistic cofactors, such as alcohol use or HBV infection. Identification of infected persons could also allow focused educational interventions to decrease viral transmission.27
In this article, we describe the derivation and initial testing of a quantitative screening tool that includes some of the known risk factors of HCV and selected behaviors or experiences that might increase infection risk. Like other studies, blood transfusion and sexual intercourse with an injecting drug user were significant risk factors for infection. The other components of the model associated with increased risk—HBV infection, inability to donate blood or to obtain life insurance, sexual contact with a prostitute, and witnessing injecting drug use—presumably capture, directly or indirectly, behavior and lifestyle factors that might result in HCV infection. Study participants provided all their answers anonymously, perhaps facilitating more candid responses to difficult or embarrassing questions.
By incorporating specific high-risk behaviors or credible proxies for higher risk, models such as this may allow for more targeted identification of infected persons. If used as a self-administered questionnaire, it is conceivable that respondents will answer more truthfully. However, the identification of a few questions that can be quickly posed may also facilitate case finding by primary care providers.
It was not the intent of this study to determine which probability threshold for HCV infection is appropriate for triggering screening. Instead, our primary goal was to identify whether risk could be stratified more accurately, especially in populations in which the overall prevalence of HCV infection was low. With a background prevalence of approximately 2% (as estimated for the US population), identification of 2 risk factors increases the likelihood of infection to 10%, whereas 4 or more risk factors raises the probability to approximately 50%.
Two variations of the same model, the first using weighted inputs for each variable and the other simply totaling the number of “positive” responses to assign a risk score, predict disease likelihood with similar accuracy. The latter version is potentially more clinically useful, whether self-administered by patients or used by providers as part of a health screen. Patients might be willing to answer such screens if they are not required to specify which items increased their risk, instead simply noting how many “positive” responses applied.
Several limitations of this study deserve mention. First, the study population may not be representative of other groups because it is possible that the HCV risk factors among infected patients under care in a hepatology practice may differ from those of patients in other settings. However, the prevalence of previously undiagnosed HCV cases in the primary care group (3/207 or 1.5%) approximately parallels the prevalence in the United States, suggesting that this group is similar to the general population with respect to HCV prevalence. Validation of the model in other populations that include only primary care patients with unknown HCV status is necessary. Given the low background prevalence of infection, a large population will be necessary for any validation studies. Second, research personnel closely safeguarded the anonymity of participants, perhaps allowing for more complete and candid answers. Patients may be reluctant to directly disclose to their physicians information about the items in the instrument. An appealing feature is that physicians need only know how many items were present. Third, one of the relevant factors was a positive response to the question, “Has a health care provider ever told you that you have hepatitis B infection?” We purposefully phrased the question to permit response without consulting medical records or providers. Because we did not validate the presence of HBV infection, we performed additional analyses that dropped this variable from the model (data not shown). Similar results, although with modestly reduced predictive accuracy, were seen. Finally, cost-effectiveness, like clinical effectiveness, is an important consideration for population-based screening. We have not addressed the questions of the cost per case of HCV detected or the cost-effectiveness of treatment for populations of patients with minimal liver disease.
We believe that the predictive instrument described herein, if validated, will be appealing to clinicians because it requires no calculation and could easily be placed on a pocket card or personal digital assistant. Physicians would need an estimate of infection prevalence in their patient population to estimate individual risk. Ultimately, patients could assess their own HCV risk. Self-identification, if widely practiced, could be an effective method of case ascertainment. Hepatitis C virus risk assessment instruments may be the first step in general population screening, with antibody testing applied only to individuals above a certain risk threshold.
Correspondence: David S. Weinberg, MD, MSc, Fox Chase Cancer Center, 333 Cottman Ave, Philadelphia, PA 19111 (firstname.lastname@example.org).
Accepted for Publication: May 11, 2005.
Financial Disclosure: None.
Funding/Support: This study was funded in part by Schering-Plough Corp, Kenilworth, NJ.
Role of the Sponsor: Neither Schering-Plough Corp nor its representatives had any role in the design or conduct of the study; in the collection, management, analysis, and interpretation of the data; or in the preparation, review, or approval of this manuscript.
Acknowledgment: We are indebted to Christopher DiMaio, MD, Raymond Rubin, MD, and Michael Lucey, MD, for their thoughtful input.
Create a personal account or sign in to: