Validation of a Hepatitis C Screening Tool in Primary Care FREE

Worldwide, hepatitis C virus (HCV) is one of the most common blood-borne pathogens, with an average prevalence of 3.1%.¹ The HCV antibody has an estimated prevalence of 1.8% in the United States.² In inner-city ambulatory primary care and Veterans Administration populations, however, the rate of HCV infection is as high as 8% and 17%, respectively.^3- 4 The Centers for Disease Control and Prevention (CDC) recommends that all persons be assessed for HCV risk factors and that those with risk factors be screened for HCV antibodies.⁵ Unfortunately, rates at which primary care patients are assessed for historical risk factors, as well as the rates at which patients at increased risk are screened for HCV antibodies, remain below the CDC goals.^6- 7

With the potential for increased clinical benefits with earlier diagnosis,⁸ a new priority for health care providers generally, and primary care providers in particular, is HCV risk factor assessment and antibody testing. While some HCV risk factors are well known, others remain less clearly associated with HCV antibody positivity.⁹ Because universal HCV antibody screening, regardless of risk factors, is neither cost-effective nor practical,¹⁰ assessment of HCV risk factors is a critical tool for primary care providers to identify patients appropriate for antibody testing, particularly in settings with a higher prevalence of disease.¹¹ Well-validated clinical prediction tools have been widely used in other areas of medicine, as they are useful in improving diagnostic accuracy by quantifying the relative contribution of key historical, laboratory, and physical examination data to diagnostic evaluation.¹² Clinical prediction tools for HCV screening have been an underused resource. We performed a prospective validation and refinement of an HCV risk assessment tool that could be used by primary care providers to identify patients who require screening for HCV antibody in at-risk populations.

Our study population comprised randomly selected patients who were attending an adult primary care clinic. The clinic serves approximately 40 000 patients per year and is affiliated with a large academic medical center and tertiary hospital located in the East Harlem neighborhood of New York, New York. The practice's patient population is composed predominantly of persons of African American and Hispanic ethnicity as well as lower socioeconomic status. All patients who were seen in the primary care practice between March 2002 and August 2003 were eligible for the study. With the use of a random number table, potential study subjects were randomly selected among individuals who had come to the clinic either for a scheduled visit with their primary care provider or for an unscheduled visit for an urgent problem. Exclusion criteria included age younger than 18 years or language other than English or Spanish. The study protocol was approved by the institutional review board, and all patients signed informed consent before study participation.

Face-to-face interviews conducted by recruiters in English or Spanish were used to administer a 27-item risk factor assessment questionnaire.^10,13- 15 The screening questionnaire, which was previously developed at Jefferson Medical Center, Philadelphia, Pennsylvania, was based on our clinical experience and on the literature. It has been widely used in clinical settings, including our own, but has never previously been validated or published. Each item in the instrument could be answered yes or no. The questionnaire grouped risk factors into 5 domains: work history, exposure history, personal care history, social history, and medical history. The work history domain questions asked whether subjects had ever had jobs identified in the literature as having a potentially higher risk of HCV exposure: physicians, nurses, medical assistants, home attendants, and laboratory personnel as well as tattoo artists and corrections officers. The exposure history domain questions were about past contact with another persons' blood, eg, during an accident or injury. The personal care history domain items were sharing toothbrushes, receiving tattoos or piercings, and acupuncture. The social history domain questions were about illicit drug use, incarceration, and past and current sexual activity. Medical history domain questions were iatrogenic risks for transmission, including blood transfusions, dialysis, and other medical procedures as well as elevated liver function test results. Collected data included the following subject demographic information: age, sex, race and ethnicity, insurance coverage, and education level.

As part of the baseline survey, participants were also asked to report whether they had ever tested positive for HCV antibodies. The medical records of the patients who reported a history of HCV were reviewed: those with a documented laboratory blood test that was positive for HCV were classified as “positives” for the purposes of the study. A sample of peripheral blood was obtained from all other study subjects for HCV antibody testing (HCV EIA 2.0 Recombinant c100-3, HC31, and HC-34; Abbott Laboratories, Abbott Park, Illinois). Patients who were found to be reactive for HCV antibodies were also considered positives for the analyses.

Mean (SDs) were calculated for continuous variables. Univariate analysis was performed to compare the demographic characteristics among HCV antibody–positive and –negative patients. These analyses were conducted using the χ² test for proportions and the pooled t test for continuous variables. Odds ratios (ORs) with 95% confidence intervals (CIs) were calculated to assess for a potential association between the 5 HCV risk factor domains and the presence of HCV antibodies.

We used logistic regression analysis to identify HCV risk factor domains that were independently associated with the presence of HCV antibody. The 5 risk factor domains were included in the model. Other covariates, such as age, sex, and insurance status, were not included in the model, as these variables are probably proxies for other risk factors, particularly among our study cohort, and were therefore not generalizable to other populations. Variable testing was performed using the Wald test, and the goodness of fit of the final model was evaluated using the Hosmer-Lemeshow test. Adjusted ORs based on this model are given with 95% CIs.

To facilitate the clinical use of the instrument, we created an HCV risk screening tool based on the responses to the 5 domains. Furthermore, we evaluated the diagnostic performance of a simplified rule based only on the domains that were independently associated with HCV antibody on multivariate analysis. Each of the 5 domains was given an equal weight, assigning 1 point to the patient when 1 or more risk factors were present in the domain. Therefore, a total score of 0 to 5 was computed as the sum of the points for each positive domain. Sensitivity and specificity were calculated according to standard methods.^16- 17 We used the receiver operator characteristic (ROC) curve analysis to assess the diagnostic accuracy of the HCV screening instrument. The area under the ROC curve was calculated using the method of Hanley and McNeil.¹⁶

Power calculations showed that a total of 1000 patients were required for the study to have 80% power to identify whether an HCV risk factor domain with a prevalence of at least 15% in the study population was associated with 2.0 increased odds of HCV antibody positivity. All analyses used 2-tailed significance levels of P < .05 and were conducted with SAS statistical software (SAS Institute, Cary, North Carolina).

Between March 2002 and August 2003, a total of 1485 patients who presented to the primary care clinic were asked to participate in the study. Of those, 1000 agreed to participate. Older patients, with a mean age of 55 years (P < .001), and African Americans (P < .001) were less likely to participate. Conversely, patients with Medicaid insurance (P < .001) and Hispanic ethnicity (P < .001) were more likely to participate.

Of the 1000 patients enrolled in the study, 83 were positive for HCV antibodies (8.3%; 95% CI, 6.7%-10.2%). Of these, 58 were known to be HCV antibody positive and 25 were newly diagnosed as being HCV antibody positive. The characteristics of the cohort are described in Table 1. Patients who were HCV antibody positive were older (P = .01), more likely to be male (P = .02), and more likely to be insured by Medicaid (P < .001). Participants with HCV antibodies did not differ from those without HCV antibodies in race/ethnicity distribution, level of education completed, or type of appointment that had brought them to the practice that day (P > .05 for all comparisons).

Univariate analysis of the HCV risk factor domains showed that a positive medical history, exposure history, or social history was significantly associated with HCV antibody positivity (P < .01 for all comparisons [Table 2]). There was a trend toward higher rates of HCV antibody positivity among patients with a positive personal history (P = .06). Conversely, a positive response to the work history domain was not associated with a positive HCV antibody test result (P = .53).

Results of the multivariate analysis showed that a positive response to the medical history (OR, 1.9; 95% CI, 1.1-3.6), exposure history (OR, 3.4; 95% CI, 2.0-5.9), or social history (OR, 6.1; 95% CI, 3.7-10.3) domains was significantly associated with increased odds of HCV antibody positivity (Table 3). A positive work history or personal history, however, was not associated with the results of HCV antibody testing after adjustment for the effect of other risk factor domains. Secondary multivariate analysis limited to patients with a new diagnosis of HCV revealed similar results, except that the medical history domain was no longer significantly associated with HCV risk (P = .42).

The rate of HCV positive antibody was 2%, 25%, 61%, 82%, 90%, and 91% for patients with 0, 1, 2, 3, 4, and 5 positive domains, respectively. The area under the ROC curve for the screening tool based on the 5 domains was 0.73 (95% CI, 0.68-0.79) (Figure 1). Because the work history and personal history domains were not independently associated with the results of the HCV antibody testing, we also evaluated the accuracy of a simplified screening tool based on the medical, exposure, and social history domains. As shown in Figure 2, the simplified instrument had similar operating characteristics to the tool using the 5 risk factor domains (area under the curve, 0.77; 95% CI, 0.71-0.8).

Table 4 lists the operating characteristics of the simplified 3-domain screening tool as a function of the number of positive domains. One or more positive domains indicates the need for further HCV antibody testing. The tool had a sensitivity of 90% and a specificity of 31% for detecting HCV antibodies in participants with 1 or more positive domains. In participants with 3 or more positive domains, the tool had a sensitivity of 34% and a specificity of 97%.

Table 5 shows posttest probabilities based on different HCV antibody prevalence rates. Among populations with a prevalence of 1% to 2%, 3 or more positive domains results in a posttest probability of 17%. When the prevalence is 8%, 2 or more positive domains raises the posttest probability to 23%, while 3 or more positive domains raises the posttest probability to 46%. Based on negative likelihood ratios, a risk factor assessment with no positive domains in a population with 8% prevalence results in a 2% posttest probability.

In this study, we validated a 3-domain HCV risk factor assessment tool in a sample of 1000 inner-city primary care patients. The tool had good operating characteristics and may help assess HCV risk factors and identify patients who require HCV antibody testing. The goal of assessing more patients for HCV risk factors is particularly relevant given that many patients may be unaware that they have HCV infection,¹⁸ and many primary care practitioners are not assessing HCV risk factors in their patients in appropriate numbers.⁶ Therefore, use of this tool may improve the quality of care, particularly among underserved populations, in which HCV antibody prevalence is often higher.

Current CDC guidelines recommend HCV antibody screening in all individuals with risk factors for infection regardless of the setting or patient characteristics.^8,19 Rather than prioritizing HCV risk factor assessment according to the setting, then, it is vital to consider all HCV risk behaviors that occur in all patients within every setting. Adopting risk assessment strategies that identify higher risk behaviors in all patient groups holds the most promise for identifying the greatest number of patients for whom antibody testing is indicated.²⁰ Undertaking this task in all primary care settings is likely to reach the largest number of potential patients.

The potential benefits of HCV screening are multiple. First, earlier identification facilitates virologic suppression, as treatment earlier in the course of the disease is associated with better efficacy.^21- 22 Second, early diagnosis together with patient education and subsequent lifestyle modifications may reduce the risk of transmission of HCV infection to other individuals.^23- 24 Third, routine screening will lead to a fuller determination of HCV prevalence. In our inner-city primary care population, the prevalence rate was more than 4 times the reported national prevalence rate. Improved understanding of prevalence rates in particular communities will allow resources to be focused where they are needed. Finally, treatment earlier in the course of the disease is associated with acceptable cost per quality-adjusted life-years estimates.^25- 26

Our validation demonstrates that a negative result on a risk assessment tool can eliminate the need for HCV antibody testing in the majority of patients assessed. Furthermore, it can accurately identify those patients at moderately increased risk who would benefit from antibody screening. The posttest probability estimates based on the population's prevalence as shown in Table 5 can help in deciding which patients to screen for HCV antibodies. There are large patient populations in which HCV prevalence is low to moderate but in which screening remains low.^3- 4,27 In populations with this risk profile, a screen with 2 or more positive domains can sufficiently alter the posttest probability so the decision can be made as to whether or not a patient needs to be tested for HCV antibodies. A negative risk factor assessment lowers the posttest probability enough among low- to moderate-risk patients that further screening may not be needed.

This study had some limitations. The population surveyed came from an inner-city primary care practice, and the results may therefore not be generalizable to other populations or settings. Despite being an optimal population to study because it is both high risk and underserved, the prediction rule should be validated in other settings and among other populations to confirm its diagnostic accuracy. Furthermore, the self-administered questionnaire assessing risk factor history among patients with known disease presents the possibility of recall bias. However, in the secondary analysis, which included only new, previously unknown diagnoses of HCV, the results were similar except for the medical history. Nevertheless, the likelihood of patients not recalling these usually major aspects of their medical history (eg, transfusion, organ transplantation, or dialysis) is probably low. Finally, there were significant differences between the subjects who chose to participate and those who refused. African Americans and slightly older patients were less likely to participate in the study, while Hispanics and patients with Medicaid were more likely to participate. The effect of this potential sampling bias in our results is unclear and should be evaluated in future studies.

In conclusion, we have demonstrated the value of a simplified tool to assess HCV risk in patients attending a primary care practice. This tool can be used to screen all individuals for HCV risk factors and could diminish the need for antibody screening in patients who are at very low or no risk. Widespread use of the tool may facilitate and increase overall screening and detection of HCV in diverse populations of primary care patients. By targeting only patients at higher risk, it may lead to more cost-effective screening for a disease that is causing significant morbidity and mortality particularly in inner-city populations. An impact analysis or randomized control trial of this model is warranted to demonstrate both clinical value and cost-effectiveness.

Correspondence: Thomas McGinn, MD, MPH, Division of General Internal Medicine, Mount Sinai School of Medicine, 1470 Madison Ave, Box 1087, New York, NY 10029 (thomas.mcginn@mountsinai.org).

Accepted for Publication: April 14, 2008.

Author Contributions: Dr McGinn had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: McGinn, O’Connor-Moore, and Gardenier. Acquisition of data: McGinn, O’Connor-Moore, and Gardenier. Analysis and interpretation of data: McGinn, O’Connor-Moore, Alfandre, Gardenier, and Wisnivesky. Drafting of the manuscript: McGinn, Alfandre, and Gardenier. Critical revision of the manuscript for important intellectual content: McGinn, O’Connor-Moore, Alfandre, Gardenier, and Wisnivesky. Statistical analysis: O’Connor-Moore and Wisnivesky. Obtained funding: McGinn. Administrative, technical, and material support: McGinn, O’Connor-Moore, Alfandre, and Gardenier.

Financial Disclosure: None reported.