A Randomized Trial of a Pay-for-Performance Program Targeting Clinician Referral to a State Tobacco Quitline FREE

Tobacco use in the United States results in nearly 440 000 premature deaths and over $75 billion in excess medical costs each year.¹ Although brief clinic-based interventions are effective, more intensive interventions are both more effective and cost-effective.² Providing intensive counseling in a busy practice setting can be a considerable challenge.^3- 6 Referral to telephone counseling provides an alternative means to connect patients with evidence-based treatments.⁷ Nearly 50 randomized trials and real-world demonstrations support the efficacy and effectiveness of telephone counseling for smoking cessation.^8- 14 At present, all residents in the United States as well as in several other countries have access to free tobacco quitline services, and quitlines in most US states accept fax referrals from health care providers (eg, physician groups, hospitals, and medical clinics).¹⁵

A study by Bentz et al¹⁶ demonstrated the feasibility of a clinic fax referral system. Wisconsin's experience demonstrates that fax referral programs can make a meaningful contribution to total quitline call volume.¹⁷ Although these results are encouraging, the number of individuals referred may still represent only a small fraction of eligible tobacco users. Innovative strategies are needed to increase referral rates.

Pay-for-performance programs are attracting considerable interest as an approach for improving health care quality.^18- 21 Yet there is currently little scientific evidence supporting this approach.^22- 23 Fewer than 10 randomized trials have assessed the impact of financial incentives on health care quality, and results from these studies have been mixed.^24- 32

This study was a randomized trial of a pay-for-performance program targeting clinician referral to statewide quitline services in Minnesota. Adult primary care clinics were randomly assigned to receive either usual care or financial incentives encouraging referral.

This project took place from September 1, 2005, through June 31, 2006. Project collaborators were the Blue Cross and Blue Shield of Minnesota (BCBSM) Center for Prevention, major Minnesota health plans (Health Partners, Medica, Metropolitan Health Plan, Preferred One, UCare), Clearway Minnesota (an independent nonprofit), and Fairview Physicians Associates (FPA) (a physicians network).

Access to telephone counseling is provided to all Minnesota residents through collaboration between Clearway and Minnesota health plans. Clearway promotes the Quitplan Helpline as a single access number for all callers and provides telephone counseling for callers without other access to this service. Callers with health insurance are transferred directly to plan-sponsored services. The Clearway Quitplan Helpline and most Minnesota health plans offer nicotine replacement therapy for free or with insurance copayment.³³ Individuals who wish to use medications but do not have insurance coverage for these medications are transferred to the Quitplan service.

Fairview Physicians Associates is a large multispecialty group. This project involves the 49 FPA clinics that provide adult primary care services. Thirty-two of the 49 FPA clinics use an electronic medical record system (EMR). This project employed a 2-group clinic randomized design with 25 clinics randomly assigned to the usual care condition and 24 assigned to the intervention condition. Clinic directors and administrators were not blinded as to group assignment of their clinic.

Minnesota health plans and Clearway collaborated to create a unified fax referral system. This involved developing a single fax referral form and telephone number and establishing a system to triage referrals to the Quitplan Helpline or appropriate health plan number. Receipt of a referral resulted in up to 3 outbound telephone calls over a 2-week period by the responsible quitline to offer counseling.

The medical group leadership at FPA mailed information packages to clinic administrators of all participating clinics. This package included a letter describing the fax referral system and copies of the fax referral forms. Fairview Physicians Associates also modified the EMR of all primary care clinics so that the referrals could be completed and submitted within the EMR.

Blue Cross and Blue Shield of Minnesota modified existing contracts with intervention clinics to provide incentives encouraging quitline referral. Clinics that referred 50 smokers would receive a $5000 performance bonus. Clinics would also receive $25 for each referral beyond the initial 50. This incentive amount was arrived at after consultation with the BCBSM network management team and represents an amount that was judged as likely to be meaningful to most clinics in the BCBSM network. All patients referred counted toward the clinic total regardless of health plan coverage (ie, not only BCBSM members). Incentive payments were made to clinics in 1 lump sum at the end of the contract period. Incentive dollars went into each clinic's general operating fund. There were no payments to individual administrators, physicians, and staff as part of this project.

Clinic directors and administrators from intervention clinics were invited by FPA leadership to a project launch meeting to introduce and promote the program. A follow-up visit by a project representative was offered to each intervention clinic. Monthly updates on the number of referrals were provided by e-mail to clinic administrators. Feedback occurred at the clinic level. There was no personalized feedback to individual physicians or staff.

The primary outcome measure for this study was the percentage of the clinic's smokers referred to telephone counseling. This was defined as the number of unique individuals referred divided by the estimated number of smokers seen in the clinic. The number of smokers seen in each clinic was estimated by multiplying the number of unique patients seen in the clinic during the study period by the estimated prevalence of smoking within each clinic. The estimated prevalence of smoking in each clinic was based on a patient satisfaction survey administered in all clinics immediately prior to the start of the project (mean [SD] number of completed surveys per clinic, 594 [617]). This survey included the question, “Do you currently smoke cigarettes every day, some days, or not at all?” An individual was defined as a smoker if they reported smoking cigarettes “every day” or “on some days.”

Information on clinic characteristics was derived from FPA administrative data. This includes the number of physicians in each clinic, the type of practice (eg, family medicine, internal medicine, obstetrics and gynecology, multispecialty), and the presence or absence of an EMR system.

Prior to project launch, FPA leadership also provided a subjective rating of each clinic's history of engagement with quality improvement activities. Clinics were judged to be “very engaged” if they had a consistent history of rapid and successful adoption of quality improvement activities. Clinics that had a history of participating with some but not all quality improvement efforts were judged to be “engaged.” Clinics with a history of limited or no involvement in quality improvement efforts were considered “less engaged.”

Project costs for development, implementation, and financial incentives were tracked. The costs of development include physician, project staff, health plan representative, and FPA staff time. This includes FPA information technology staff effort to create and integrate the EMR fax referral form. Implementation costs cover effort and materials to provide information packages to all clinics and also feedback efforts to intervention clinics (eg, kickoff meeting, clinic visits, e-mail reports). Hourly costs are estimated based on current annual salaries for project, health plan partner, and FPA staff. Performance incentives were tracked at the clinic level.

The primary unit of analysis was the clinic (total n = 49), and all statistical testing was performed at the clinic level. Comparison of the primary outcome (ie, rate of referral) was performed using a 2-sample t test with 47 df. We examined predictors of referral using 1-way and 2-way analyses. All analyses were performed using Stata statistical software package (version 9.0; StataCorp, College Station, Texas).

For the cost analysis, fixed costs for structural items that affected both usual care and intervention sites (eg, modifying the EMR) were divided equally between these 2 groups. Variable costs related to the volume of fax referrals (eg, referral triage fees, staff time to address physician questions about the referral system) were allocated to the usual care or intervention group clinics in proportion to the number of referrals submitted. Costs related to the pay-for-performance program (eg, monthly feedback, financial incentives) were assigned only to the intervention group.

All data presented herein were collected as part of quality improvement efforts of the participating organizations. The information presented in this article contains clinic level counts. No individual level data or personal health information was included in the evaluation of this project. This project was reviewed by the University of Minnesota's institutional review board (IRB) and determined to be exempt from IRB approval under federal guidelines 45 CFR §46.102 (f) for existing data.

The characteristics of clinics participating in this project are shown in Table 1. There were no significant differences between the usual care and intervention clinics in any of these characteristics (see Table 1 for P values).

During the project, there were 1483 referrals received from intervention clinics compared with 441 referrals received from usual care clinics. Intervention clinics referred a mean of 11.4% (95% CI, 8.0%-14.9%) of their smokers compared with 4.2% (95% CI, 1.5%-6.9%) of smokers visiting usual care clinics (t₄₇ = 3.45; P = .001).

The distribution of clinics in terms of the percentage of smokers referred is shown in Figure 1. Among usual care clinics, 15 of 25 clinics referred less than 2% of their smokers and only 4 of 25 clinics referred more than 10% of their smokers. This pattern is nearly reversed in the intervention clinics: 12 of 24 clinics referred more than 10% of their smokers (χ²₂ = 12.58; P = .002).

There was no association between clinic specialty type, the number of physicians in the clinic, or the use of EMRs (vs paper records) in the clinic and the percentage of smokers referred to quitline services. There was a significant association between the clinic's history of engagement with quality improvement activities and the percentage of smokers referred. Clinics with a history of being very engaged referred on average 14.5% of their smokers compared with 6.8% referred by “engaged” clinics and 5.5% referred by “less engaged” clinics (analysis of variance [ANOVA] F₄₈ = 4.52; P = .02). A 2-way ANOVA including intervention group, quality improvement history, number of physicians, type of clinic, and clinic prevalence of smoking was consistent with results of bivariate analysis in demonstrating that only the intervention group and the clinic's quality improvement history were independent predictors of the referral rate.

We performed a secondary analysis of referral rates with separate comparisons for groups of clinics that had a history of being very engaged, engaged, or less engaged (Figure 2). Among the clinics with a history of being very engaged with quality improvement activities (n = 9), there was no apparent difference in the referral rate between the intervention (15.1%) and usual care clinics (14.1%; t₇ = 0.20; P = .85). In contrast, among clinics considered to be only engaged (n = 22; intervention group, 10.1%, vs usual care group, 3.0%; t₂₀ = 3.76; P = .001) or less engaged (n = 18; intervention group, 10.1%, vs usual care group, 1.1%; t₁₆ = 2.52; P = .02), the rate of referral was significantly higher in intervention clinics compared with usual care clinics.

Receipt of a fax referral resulted in a proactive call from the participating quitline to initiate telephone counseling. The mean contact rate (ie, a counselor spoke with the patient) per clinic for these outbound calls was 60.2% (95% CI, 49.7%-70.7%). Of individuals who were contacted, the mean rate of quitline enrollment was 49.4% (95% CI, 42.8%-55.9%). This corresponds to a mean enrollment per clinic of 27.0% (95% CI, 21.3%-32.8%) of all referrals. There was no difference in the clinic means for the rates of initial contact (mean for intervention clinics, 58.2% [95% CI, 47.2%-69.3%] vs mean for usual care clinics, 62.3% [95% CI, 43.8%-81.7%]; t₄₄ = 0.386; P = .70) or eventual enrollment (mean for intervention clinics, 25.9% [95% CI, 19.5%-32.3%] vs mean for usual care clinics, 28.3% [95% CI, 17.9%-38.6%]; t₄₄ = 0.437; P = .68) between intervention and usual care sites. The overall percentage of smokers who were referred and enrolled in quitline services was higher in intervention clinics (3.0%; 95% CI, 2.2%-3.8%) compared with control clinics (1.3%; 95% CI, 0.4%-2.1%; t₄₇ = 2.93; P = .005).

Project costs and allocation of costs to usual care and intervention clinics are shown in Table 2. The greatest single cost item was $70 475 for financial incentives to clinics in the intervention group. This cost was the result of 11 of 24 intervention clinics receiving $5000 for referring at least 50 smokers and $25 per referral paid for 619 additional referrals from the clinics that met the initial threshold of 50 referrals.

The total cost of the project for usual care clinics was $8937, the cost per referral was $20, and the cost per enrollee was $72. Overall costs ($95 733), cost per referral ($65), and cost per enrollee ($232) were higher in intervention clinics. Intervention costs were greater than usual care costs by $86 796. In return for these costs, intervention clinics provided 1042 additional referrals that resulted in 289 additional enrollees. Therefore, the marginal cost for the intervention clinics was $83 per additional referral and $300 per additional enrollee.

This randomized controlled study demonstrates that a pay-for-performance program substantially increases the rate of clinician referral to quitline services. It is important to point out that free quitline services, a necessary ingredient in this program, are already broadly available in the United States, Canada, and a growing number of other countries.^11,15 In addition to the availability of a quitline, collaboration between Minnesota health plans likely contributed to the success of this project by addressing issues that may have limited the effectiveness of incentive strategies in prior studies.^22- 23 In particular, health plan collaboration simplified the referral process for clinics and also rewarded clinics for referring patients who smoked regardless of their insurance coverage. This allowed clinics to target their entire population of smokers as opposed to focusing on members of a single health plan.

In intervention clinics, approximately 1 in 10 smokers were referred to quitline services during the study period. This rate of active referral is higher than that reported in the feasibility study by Bentz et al¹⁶ (in which 3.2% of smokers were referred) that did not employ financial incentives. This 10% rate of referral is particularly impressive if one considers that at any point in time only 20% to 30% of smokers are expected to be considering quitting in the next 30 days (ie, will be eligible for referral).^34- 35 The absolute number of referrals received from intervention clinics is also impressive compared with those of other programs. For example, the number of referrals from the 24 intervention clinics is comparable with that reported for the entire state of Wisconsin during the first year of operation of their statewide fax referral program.¹⁷

An interesting finding is that this pay-for-performance program resulted in the greatest benefit for clinics with a history of being less engaged in quality improvement activities. A history of consistent engagement with quality improvement activities was a strong predictor of referral, and this may have limited the potential room for improvement in response to the offer of financial incentives. Other studies^36- 37 have noted a similar pattern with financial incentives leading to the greatest improvement among sites with the lowest baseline performance. This pattern is consistent with Rogers’³⁸ theory of diffusion of innovation if one considers clinics with a history of being very engaged in quality improvement to represent “early adopters.” In this context, our findings suggest that pay-for-performance programs may operate by speeding diffusion of innovation to the majority of clinics that adopted quality improvement programs later.

Contacting and enrolling individuals who were referred to quitlines was a major challenge. Rates of contact and enrollment observed in our study are lower than those reported in clinic referral studies by Roski et al²⁹ (80% contact, 83% subsequent enrollment) and Bentz et al¹⁶ (60% contact, 90% subsequent enrollment). In the study by Roski et al,²⁹ outreach calls and telephone counseling were performed by clinic staff (as opposed to a state or health plan quitline), and this may have contributed to higher contact and enrollment rates. In the study by Bentz et al,¹⁶ outreach calls were performed by a state quitline, and the initial contact rates were similar to those observed in our study. Higher enrollment among those reached in the study by Bentz et al¹⁶ may be related to their offer of brief, single-session counseling. It is important to point out that rates of contact and enrollment in this project did not differ between usual care and intervention clinics. This means that the pay-for-performance program did not result in a higher rate of inappropriate referrals from intervention clinics. Even with the challenges experienced in this study in terms of contact and enrollment, the overall reach in intervention clinics (ie, 3% of smokers enrolled) compares favorably with estimates of population participation in quitline services nationally (ie, 0.6%-1.7% of smokers are enrolled per year).¹¹

The costs per referral and per enrollee in usual care clinics are similar to those reported by Bentz et al¹⁶ in their feasibility study. The marginal costs per additional individual referred and enrolled from intervention clinics are comparable with the estimated cost per enrollee associated with statewide media campaigns promoting telephone counseling (ie, approximately $100-$200 per caller).^16,39- 40 In considering these costs, it is important to keep in mind the well-recognized health and economic benefits of tobacco cessation.^2,41 Solberg et al⁴² estimated that providing evidence-based treatments in primary care results in a cost savings of over $500 per smoker treated.

Increasing enrollment after referral would further increase the cost-effectiveness of the pay-for-performance intervention. Possible strategies to increase contact and enrollment rates include the mailing of prenotification letters prior to the outreach calls and changes in the number or timing of outreach attempts. Additional strategies under consideration include involvement of clinic staff in the initial telephone outreach or direct connection of smokers to quitline services as part of the office visit.

There are several limitations to consider when interpreting the results of this study. First, financial incentives were offered in the context of system changes that enabled easy referral. Incentives may or may not have had similar influence without these supportive systems. In this regard, we also note that the EMR modifications undertaken in our study to allow electronic fax referral were simple and did not include automated prompts or reminders. Recent work by Bentz et al⁴³ suggests that this approach may be beneficial. The combined influence of a pay-for-performance program paired with more sophisticated EMR modifications awaits future study. Second, in our study, the incentives were administered along with goal-setting and performance feedback. It is therefore not possible to determine the separate effects of these different components of the intervention. Future studies might consider factorial or nested designs, perhaps testing personalized feedback to individual physicians, to tease apart the independent effects of incentives and feedback. The linkage of incentives with feedback in this study also prevented the blinding of clinic leadership to their clinic's group assignment. This creates the possibility that clinic leadership may have suppressed referrals, waiting for some future time when the clinic might have become eligible for pay-for-performance incentives. Although possible, the similarity in the referral rate for control clinics in this study and the rate reported in the feasibility study by Bentz et al¹⁶ argues against this as a meaningful explanation of the results reported herein. Third, although the incentive program described herein demonstrated positive effects, there are unresolved issues regarding the optimal magnitude, duration, and timing of incentive payments.²² Future studies comparing different incentive structures are needed to address these questions. Finally, the outcome for this study focused on the percentage of smokers referred and enrolled. Although this is reasonable given the strong evidence supporting the efficacy and effectiveness of telephone counseling, it is important to acknowledge that we do not have information regarding use of other assistance or cessation rates among the population of smokers who received care in participating clinics. For future studies, investigators might consider identifying and tracking a cohort of smokers to determine how interventions such as the one described herein influence overall use of assistance and cessation rates in the population.

Despite these limitations, this study demonstrates that implementing a pay-for-performance program substantially increases clinician referral of smokers to a state tobacco quitline. Based on findings of this study, BCBSM is exploring statewide expansion of pay-for-performance incentives for quitline referral. Quitlines are widely available, and application of pay-for-performance strategies to encourage health care provider referral should be strongly considered by heath care organizations seeking to reduce the health and economic burden of tobacco-related disease.

Correspondence: Lawrence C. An, MD, Department of Internal Medicine, University of Minnesota, Mayo Building, Mail Code 741, 420 Delaware St SE, Minneapolis, MN 55455 (lcan@umn.edu).

Accepted for Publication: January 28, 2008.

Author Contributions: Dr An had full access to all of the data in the study, takes responsibility for the integrity of the data and the accuracy of the data analysis, and made final decisions regarding the manuscript content. Study concept and design: An, Bluhm, Foldes, Klatt, Nersesian, Larson, and Manley. Acquisition of data: An, Nersesian, Larson, and Manley. Analysis and interpretation of data: An, Bluhm, Foldes, Alesci, Klatt, Center, Nersesian, and Ahluwalia. Drafting of the manuscript: An, Alesci, Klatt, and Center. Critical revision of the manuscript for important intellectual content: An, Bluhm, Foldes, Alesci, Klatt, Center, Nersesian, Larson, Ahluwalia, and Manley. Statistical analysis: Center. Obtained funding: Foldes and Manley. Administrative, technical, and material support: An, Bluhm, Foldes, Alesci, Klatt, Nersesian, and Larson. Study supervision: An, Bluhm, Foldes, Klatt, Center, Nersesian, Ahluwalia, and Manley.

Financial Disclosure: None reported.

Funding/Support: This study was funded by the BCBSM Center for Prevention.

Role of the Sponsor: Staff at the BCBSM Center for Prevention participated in the design and conduct of this study and reviewed the manuscript but did not participate in the analysis or interpretation of the data.

Additional Contributions: Rhonda Evan, BA (BCBSM Center for Prevention), the Minnesota Call it Quits Helpline Collaborative (Clearway Minnesota, Minnesota Health Plans), and FPA clinics provided invaluable assistance.