The Lack of Utility of Circulating Biomarkers of Inflammation and Endothelial Dysfunction for Type 2 Diabetes Risk Prediction Among Postmenopausal Women:  The Women's Health Initiative Observational Study FREE

Type 2 diabetes mellitus (DM) is a leading cause of morbidity and mortality in the United States.¹ According to the Centers for Disease Control and Prevention, more than 24 million Americans were living with diabetes in 2007,² and this number is forecasted to increase to 48 million by 2050.³ The alarming surge of diabetes incidence highlights the importance of developing effective screening and preventive measures to control this devastating disease. Among high-risk individuals, lifestyle modification and pharmacological interventions have been demonstrated to be effective.^4- 5 It is therefore of clinical and public health importance to identify high-risk individuals for aggressive treatment.

The recent discovery of novel biomarkers for type 2 DM risk not only improves molecular understanding of diabetes etiology but also brings new opportunity for clinical risk stratification and management. Within the Women's Health Initiative Observational Study (WHIOS) and several other large cohort studies, plasma markers of systemic inflammation^6- 19 and endothelial dysfunction^20- 22 appeared to be prospectively associated with risk of type 2 DM (Table 1). These findings support the notion that chronic inflammation and endothelial dysfunction are antecedents of clinical diabetes. However, it remains unknown whether incorporating such markers may improve the performance of risk prediction in apparently healthy individuals. In this study, we examined the added predictive value of plasma inflammation markers (ie, white blood cell count, tumor necrosis factor receptor 2, interleukin 6 [IL-6], and high-sensitivity C-reactive protein [hsCRP]) and endothelial dysfunction markers (E-selectin, soluble intercellular adhesion molecule 1 [ICAM-1], and vascular cell adhesion molecule 1 [VCAM-1]) beyond the traditional diabetes risk factors in postmenopausal women of diverse ethnicities.

The WHIOS is an ongoing longitudinal study designed to examine associations between clinical, socioeconomic, behavioral, and dietary risk factors and subsequent health outcomes, including cardiovascular disease (CVD) and diabetes in multiethnic postmenopausal women (see the eAppendix for the list of Women’s Health Initiative Investigators). Details of the rationale, eligibility, and other design aspects have been published elsewhere.²³ In brief, between September 1994 and December 1998, the WHIOS enrolled a total of 93 676 women aged 50 to 79 years at 40 clinical centers throughout the United States. Women completed study questionnaires and provided fasting blood samples at study baseline, and they were followed up by means of annually mailed self-administered questionnaires and an additional clinical center visit at 3 years after enrollment for obtaining data on disease outcomes and possible exposures, collecting physical and anthropometric measures, and obtaining blood samples. Methods of data collection and measurement characteristics have been reported previously.^23- 25 Of the 93 676 postmenopausal women enrolled into the WHIOS cohort, 82 069 had no history of diabetes or CVD. The WHIOS has been approved by human subjects review committees at each participating institution.

A case-control study nested in the WHIOS was conducted to investigate the associations between novel plasma markers and risk of clinical type 2 DM.^14,21,26Incident type 2 DM was defined as first-time use of oral hypoglycemic agents or insulin assessed by medication use or self-report when medication use data were unavailable. A total of 1584 cases and 2198 controls who were free of reported diabetes and CVD at baseline and who provided adequate blood specimens were selected during a median follow-up of 5.9 years (mean, 5.5 years; range, September 1994 to February 2004). Controls were selected following the principles of risk-set sampling and matched to the cases by age (±2.5 years), race/ethnicity, clinical center, time of blood drawn (±0.10 hours), and length of follow-up.

White blood cell count was measured for all women in the WHIOS. Plasma levels of fasting glucose, insulin, tumor necrosis factor receptor 2, IL-6, hsCRP, E-selectin, ICAM-1, and VCAM-1 were measured only for women selected in the nested case-control study. Details of laboratory procedures and specific assay methods have been published previously.^14,21,26

We first determined the distributions of plasma markers and other risk factors in women enrolled in the WHIOS cohort and those in the case-control study. Correlations between plasma markers were evaluated by calculating Pearson partial correlation coefficients. Continuous variables with skewed distribution were transformed with natural logarithm. We then determined the best functional forms for continuous variables by examining smoothed plots and fractional polynomial models.²⁷ Linear fit appeared to be reasonable for all continuous variables considered. Conditional logistic regression²⁸ was used for building the prediction models. Because of the need for numerous model comparisons in this study, we performed multiple imputations to address the missing data issue²⁹ (14% of the women had missing information on ≥1 of the covariates). In a sensitivity analysis, we repeated all the subsequent analyses, excluding women whose baseline glucose level was 126 mg/dL or greater (737 cases and 27 controls) (to convert glucose levels to millimoles per liter, multiply by 0.0555).

Clinical risk factors included age, race/ethnicity, waist circumference, hypertension (≥130/85 mm Hg³⁰ or treated hypertension), history of high cholesterol levels requiring medication, physical activity, cigarette smoking, alcohol use, and family history of diabetes in 1 or more first-degree relatives. These clinical risk factors coupled with fasting glucose levels (the traditional risk factors) constituted our reference model. Age and race/ethnicity were matching factors and were adjusted in the conditional logistic regression. We chose to include waist circumference as opposed to body mass index or waist to hip ratio because models containing waist circumference resulted in a better model fit measured by the model likelihood ratio χ² statistic.

We also evaluated a secondary reference model including the homeostasis model assessment of insulin resistance (HOMA-IR) and beta cell function (HOMA-B) indices. The HOMA-IR was computed as (fasting insulin level × fasting glucose level)/22.5, with insulin level reported in microinternational units per milliliter and glucose level in millimoles per milliliter. The HOMA-B was computed as (20 × fasting insulin level)/(fasting glucose level − 3.5), with insulin level reported in microinternational units per milliliter and glucose level in millimoles per milliliter. This secondary reference model did not perform better than the model with fasting glucose level, as assessed by global model fit statistics (described in the “Added Predictive Value of Novel Plasma Markers” subsection) and area under the receiver operating characteristic curve (AUC). Therefore, the reference model with fasting glucose level was used for the subsequent analyses.

Model-predicted type 2 DM risk for each woman in the case-control study was calculated using methods described previously.^31- 33 Briefly, we first estimated the overall 6-year survival probability from the entire WHIOS cohort by using the Kaplan-Meier estimator. This overall survival probability was then used as the basis for calculating the individual model-predicted risk with a scaling factor determined by the linear function of model coefficients and the specific covariate values of each woman, corrected by the mean covariate values in the WHIOS. The mean values of plasma markers in the cohort were approximated by the values of these markers among controls.

The predictive value for each of the inflammation and endothelial dysfunction markers beyond traditional risk factors was assessed by comparing the reference model with the model that included the markers added one at a time (as separate models). The predictive value was evaluated on the basis of (1) improvement in global model fit and (2) improvement in model discrimination.^34- 36 For assessing improvement in global model fit, we computed −2 log likelihood, the variable likelihood ratio χ² statistic (the improvement of fit for inclusion of each biomarker separately), the model likelihood ratio χ² statistic (fit for the entire model), and Bayesian information criteria (BIC; fit for the entire model taking into account the number of variables). To compare model discrimination adding specific markers, we calculated the AUC and the integrated discrimination improvement (IDI) recently proposed by Pencina et al³⁶ (a measure that quantifies the improvement of predicted risk for cases and controls). Because of the case-control sampling and the matching design, we calculated AUC and IDI weighted using the stabilized weight of inversed case and control sampling fraction. The inversed sampling fraction (weight) was stabilized to reflect the population distribution of age and ethnicity in cases and controls in the full cohort for unbiased variance.

To further explore the utility of novel plasma markers in type 2 DM risk prediction in different ethnicities, we repeated all analyses for each ethnicity separately.

To evaluate whether novel plasma markers could replace the role of certain traditional risk factors in the prediction equation, we performed the following analyses for model selection: (1) a model that considered traditional risk factors only; (2) a model that considered novel plasma markers and traditional risk factors; and (3) a model that considered novel markers and a subset of traditional risk factors that can be objectively measured (ie, excluding history of smoking, alcohol use, and exercise). Model 3 was fitted because novel plasma markers might replace the need to collect self-reported complex behaviors that were susceptible to differential measurement error. We used backward and stepwise model-building strategies for model selection with a significance criterion level of α = .05. If the two model-selection approaches suggested different models, the one with the smaller BIC was selected. For comparison purposes, a previously proposed clinical model^37- 38 based on the metabolic syndrome–related covariates, including waist circumference, hypertension, history of high cholesterol levels, family history of type 2 DM, and fasting glucose level, was also fitted.

Global model fit statistics (ie, −2 log likelihood and BIC) and model discrimination ability (AUC and IDI) were calculated for these models. In addition, net reclassification improvement, a measure that quantifies the improvement of prediction of clinically meaningful risk categories,³⁶ was also calculated. Clinically meaningful risk categories were adopted and modified from Adult Treatment Panel III guidelines for primary CVD prevention (low-, medium-, and high-risk categories defined as <6%, 6%-20%, and >20% 10-year risk of developing CVD, respectively). We interpolated these cutoffs for 6-year type 2 DM risk (ie, 3.6% and 12.5%) and used less than 5%, 5% to 15%, and greater than 15% as the low-, medium-, and high-risk categories, respectively, for the ease of interpretation and patient communication. Positive and negative predictive values (PPV and NPV) were estimated for each model following the Bayes theorem, which used sensitivity and specificity as well as the background risk in the population. Because a 15% 6-year type 2 DM risk is considered a high risk in this study, we used 15% as the cutoff to calculate PPV and NPV for our models. For sensitivity analysis, 10% and 20% cutoff were also used for the calculation of PPV and NPV. Again, the weighted calculation of AUC, IDI, net reclassification improvement, and predictive values was performed. All analyses were performed with SAS statistical software (version 9.1; SAS Institute Inc, Cary, North Carolina).

Baseline characteristics of the WHIOS cohort participants and the 3782 women in the case-control study are presented in Table 2. Overall, the newly developed type 2 DM case patients had higher prevalence of overweight/obesity, hypertension, hypercholesterolemia, cigarette smoking, and family history of diabetes than controls at baseline. These incident type 2 DM cases also had higher levels of fasting glucose (70.2% of cases had fasting glucose levels ≥110 mg/dL), insulin, HOMA-IR, and inflammation and endothelial dysfunction markers at baseline. The 6-year estimated cumulative risk of developing type 2 DM risk for each age and ethnicity group is based on the entire WHIOS cohort (Table 3). At all ages, minority women had higher diabetes incidence than white women (eg, 8% in blacks vs approximately 3% in whites).

When we assessed the contribution of individual novel plasma markers, levels of white blood cells, IL-6, hsCRP, and ICAM-1 appeared to improve global model fit, with ICAM-1 level improving the model fit the most as suggested by the variable likelihood ratio χ² statistic (Table 4). However, none of the novel plasma markers increased the AUC or the IDI in any significant manner. Although the groups of inflammatory and endothelial dysfunction markers resulted in similar improvement in model fit overall (Table 4), the added contribution by these plasma markers varied across ethnic groups (Table 5). While hsCRP level appeared to enhance model fit in white women, IL-6 and ICAM-1 levels were the only 2 markers that enhanced model fit in black women. However, none of the markers resulted in more favorable model BIC or AUC in all race/ethnic groups. Similar results were obtained when women with baseline glucose levels of 126 mg/dL or more were excluded from the analyses (Table 4, bottom).

Based on the evaluation of BIC, the best prediction model of novel plasma markers (BIC = 891.09; Table 6) had global model fit performance that was similar to the model based on the metabolic syndrome–related covariates and family history of type 2 DM (BIC = 892.77). All 4 types of best prediction models had similar discrimination/reclassification performance, as indicated by the minimal change in AUC (all models achieved an AUC of 0.93), IDI, and net reclassification improvement (Table 6). Based on cutoff values of 10%, 15%, and 20% 6-year predicted risk, PPV and NPV were similarly comparable for all 4 models (Table 6). In the sensitivity analysis excluding women with baseline glucose levels of 126 mg/dL or more, similar results were obtained.

In this study of multiethnic postmenopausal women, plasma markers of systemic inflammation (white blood cells, tumor necrosis factor receptor 2, IL-6, and hsCRP levels) and endothelial dysfunction (E-selectin, ICAM-1, and VCAM-1 levels) did not appear to provide additional values for type 2 DM risk stratification and prediction, despite the fact that they were independent risk factors for this disease. Although the addition of some novel biomarkers significantly improved global model fit, none of these biomarkers led to a meaningful improvement in model discrimination as assessed by AUC and IDI. In particular, the final best model that included novel plasma markers yielded model fit, discrimination, reclassification ability, and predictive values similar to those of the model incorporating only well-established traditional risk factors. The latter model also performed similarly to a simpler model using only the metabolic syndrome–related covariates and family history of diabetes. These results suggest that neither detailed lifestyle factors (eg, history of smoking and alcohol use) nor novel plasma markers are critical for type 2 DM risk stratification in this multiethnic cohort of postmenopausal women.

Few studies have comprehensively and prospectively examined the added predictive value of novel plasma markers for type 2 DM risk stratification in men and women, especially in ethnic minorities. In a cohort of 822 middle-aged men and women followed up for 5 years, Hanley and colleagues³⁹ found no improvement for type 2 DM risk discrimination by adding the hsCRP level (AUC = 0.71) to a model incorporating metabolic syndrome (AUC = 0.69).

Recent studies have clearly associated biomarkers of systemic inflammation and endothelial dysfunction with type 2 DM risk, independent of fasting glucose levels and other clinical risk factors.^{6- 13,15,17- 20,22} However, risk factors that are statistically significant in etiology research are not necessarily useful predictors in screening or predictive tests. In fact, even risk factors with strong effect sizes may fail to perform in risk stratification because of substantial overlapping in the distribution of the specific risk factor in individuals who will and will not ultimately develop the disease.⁴⁰ Conventional approaches for evaluating the performance of model prediction emphasized the use of AUC, which can be directly interpreted as the probability that the model will assign a higher predicted risk to cases than to noncases. However, the implications of this approach are limited because of its lack of consideration of the actual predicted risk in relation to event and clinically meaningfully classification. Recently, Pencina and colleagues³⁶ proposed the use of IDI and net reclassification improvement to capture correct vs incorrect movements in model-predicted risk/risk category among cases and noncases compared with the reference model. Cook^41- 42 and Greenland⁴³ also argued for the application of measures for predictive gain (eg, model calibration and PPV and NPV) because this information is more relevant for clinical decision making. In the present study, we found that inclusion of inflammatory and endothelial dysfunction markers did not improve most of these measures in our model.

Several type 2 DM risk prediction algorithms have been proposed in the literature. Algorithms that use anthropometrics and clinical and lifestyle risk factors were reported to have an AUC of approximately 0.80 in various populations. Examples include the Cambridge risk score⁴⁴ (age, sex, body mass index, use of corticosteroids, treatment for hypertension, and parental history of diabetes), the National Cholesterol Education Program metabolic syndrome criteria,⁴⁵ the San Antonio Heart Study model⁴⁶ (age, sex, ethnicity, fasting systolic blood pressure, plasma glucose and high-density lipoprotein cholesterol levels, body mass index, and parental history of diabetes), and the Finnish Diabetes Risk Score³⁰ (age, body mass index, waist circumference, treatment of hypertension, fasting plasma glucose level, physical activity, and diet). Although these models appear to have reasonable discrimination ability, their calibration and predictive values remain unknown. We reported PPVs of 0.22 to 0.28 for our models when a 6-year risk of 15% was chosen as the cutoff. However, the predictive values are a function of population background risk, and they may not be generalizable to women in different age or racial/ethnic groups or geographic areas.

Several potential limitations need to be considered when interpreting our findings. First, model calibration (accuracy of the predicted risk compared with actual event rate) could not be directly assessed because novel plasma markers were measured only in the nested case-control study and not in the entire WHIOS. Second, because the same study samples were used for model fitting and AUC calculation, the AUC values might have been overestimated.⁴⁷ However, given our sample size, this potential bias is expected to be minimal.⁴⁷ Third, the AUC and other measures of model prediction may be less generalizable to ethnic minorities because of the small sample sizes of minorities in our study (eg, Asian and Hispanic/Latina subjects). Finally, our analyses did not account for differential competing risk (ie, death), although any bias due to differential competing risk should be minimal because death occurred in less than 5% of the noncase women in the cohort during the case-control sampling period.

In conclusion, in this US multiethnic cohort of postmenopausal women prospectively followed up for 6 years, our modeling indicates that novel plasma markers of systemic inflammation and endothelial dysfunction did not have additional value for clinical type 2 DM risk prediction. Further evaluation by other prospective, population-based cohort studies is needed to confirm and assess the generalizability of these findings.

Correspondence: Simin Liu, MD, ScD, Center for Metabolic Disease Prevention, University of California, Los Angeles, Center for Health Sciences 73-265, Campus Box 951772, 650 Charles E. Young Dr S, Los Angeles, CA 90095 (siminliu@ucla.edu).

Accepted for Publication: March 1, 2010.

Author Contributions: Drs Chao and Liu had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Chao, Manson, and Liu. Acquisition of data: Manson, Eaton, Margolis, Rodriguez, Phillips, and Liu. Analysis and interpretation of data: Chao, Song, Cook, Tseng, Manson, Eaton, Margolis, Phillips, and Liu. Drafting of the manuscript: Chao and Liu. Critical revision of the manuscript for important intellectual content: Chao, Song, Cook, Tseng, Manson, Eaton, Margolis, Rodriguez, Phillips, Tinker, and Liu. Statistical analysis: Chao, Cook, Tseng, and Liu. Obtained funding: Manson, Eaton, Rodriguez, and Liu. Administrative, technical, and material support: Song, Manson, Eaton, Rodriguez, Phillips, and Liu. Study supervision: Rodriguez and Liu.

Financial Disclosure: Dr Eaton reports having received research funding from Forest Pharmaceuticals, Inc, and Pfizer, Inc, to conduct research on diabetes.

Funding/Support: This study was supported by grant R01 DK062290 from the National Institute of Diabetes and Digestive and Kidney Disease and the National Institutes of Health. The Women's Health Initiative program is funded by contracts N01WH22110, 24152, 32100-2, 32105-6, 32108-9, 32111-13, 32115, 32118-32119, 32122, 42107-26, 42129-32, and 44221 from the National Heart, Lung, and Blood Institute, National Institutes of Health, US Department of Health and Human Services.

Additional Contributions: Jeff Slezak, MS, Kaiser Permanente Southern California, Department of Research and Evaluation, assisted with statistical analysis. We acknowledge all Women's Health Initiative centers and their principal investigators for their participation in this research. The dedicated and committed participants of the WHIOS made this study possible.