Evaluation of Consumer Medication Information Dispensed in Retail Pharmacies FREE

Communication with patients regarding proper use of prescribed medications relies heavily on written information. Some sources of information, including the content on prescription container labels and medication guides distributed with drugs that have serious safety concerns, are regulated by the Food and Drug Administration (FDA). However, the content and format of the most pervasive and oftentimes sole form of medication information accompanying a dispensed medication is not subject to FDA regulation. Consumer medication information (CMI) leaflets provided by pharmacies as part of the medication dispensing process are not under the regulatory authority of the FDA.

In 1995, the FDA proposed a regulation to set specific goals regarding the distribution and quality of consumer medication information (60 FR 44182; August 24, 1995). Before the regulation could go into effect, Public Law 104-180 was enacted.¹ Although the law adopted the goals of the proposed rule, it prohibited the FDA from taking regulatory steps specifying uniform content or format under the assumption that private sector initiatives were able to meet the goals. The FDA was charged to evaluate progress toward the goals, which defined that by 2006, at least 95% of dispensed prescription drugs had to be accompanied by useful CMI. An initial study² finished in 2001 found serious deficits in the content and format of CMI distributed with prescription medications. On recommendation by the FDA Drug Safety and Risk Management Advisory Committee, which expressed concern about the lack of progress toward the 2006 goals,³ the FDA developed a guidance document for the private sector that describes criteria for content and formatting of CMI.⁴ The research reported herein aimed to evaluate the percentage of dispensed medications accompanied by any written CMI and the percentage of CMI that adhered to the criteria defined by the FDA guidance document for content and formatting.

This study evaluated CMI ascertained from a national sample of retail pharmacies in the United States.

Pharmacies were selected from a national electronic list of 55 513 retail pharmacies certified by the National Council for Prescription Drug Programs, which excluded pharmacies identified as hospital, clinic, long-term care, mail order, intravenous infusion, dispensing physicians, Indian Health Service, Veterans Administration Hospital, or other government/federal setting. The list also excluded pharmacies located in Alaska, Hawaii, Puerto Rico, the US possessions, Ohio, Oregon, or Georgia, the latter 3 of which prohibit the filling of prescriptions for research purposes. Each 134th record in a random order of eligible pharmacy outlets was selected by the database vendor to arrive at a random sample of 420 pharmacies. This number reflected the necessary sample size plus a 20% dropout rate to determine the percentage of pharmacies dispensing any written information, with 95% confidence intervals (CIs) no larger than ±5% under the worst-case scenario assumption that only 50% of pharmacies would provide CMI.

The CMI was obtained by professional shoppers recruited by a national customer experience measurement agency and trained by the University of Florida to fill prescriptions written by FDA-recruited physicians. Shoppers were briefed on the role of a person recently diagnosed as having diabetes and hypertension and had to pass an online examination to evaluate preparedness to answer standard questions. Training materials included a videotape of 3 scenarios in which actors presented prescriptions to pharmacists and responded to questions according to the scripted examples of anticipated dialogues between patients and pharmacy staff. After the pharmacy visit, shoppers submitted the prescription containers and any written materials provided along with notes about verbal counseling and, wherever applicable, reasons why prescriptions were not filled. All pharmacy visits were conducted between January 28, 2008, and March 31, 2008.

Eight standards outlined by the FDA 2006 Guidance Document on Useful Written Consumer Medication Information⁴ were used to define explicit criteria that comprehensively expressed the “usefulness of CMI” for 2 widely used medications: lisinopril and metformin (Table 1).

The Criteria Development Expert Panel consists of 4 clinical experts (1 internist, 1 endocrinologist, 1 drug information specialist, and 1 community pharmacist) that was convened to develop the initial set of evaluation criteria. To select clinical content (standards 1-6), we developed a master drug information repository from the following sources: the FDA-approved labeling,^5- 6 Clinical Pharmacology Online,⁷ Micromedex,⁸Drug Facts and Comparisons,⁹ the American Hospital Formulary System,¹⁰ and Lexi-Comp.¹¹ Information was organized in a spreadsheet that allowed comparisons across references and was supplemented with directed searches for primary literature. In addition, panelists considered consumer information made available in the American Hospital Formulary System¹² and Clinical Pharmacology.¹³ Each information source was the most current version available in October 2007. Finally, the panel reviewed examples of CMI obtained from local pharmacies for wording and formatting and to test the draft set of criteria.

Each item of content identified by the development panel as critical for useful CMI was phrased as a single criterion. Some controversy arose about the inclusion of off-label indications, which is considered inappropriate practice in the FDA guidance document. Both study drugs, lisinopril and metformin, have evidence of effectiveness and are frequently used for certain off-label indications. Thus, consumer understanding that valid indications exist in addition to those in the FDA-approved labeling may help avoid confusion. A similar controversy surfaced about use in young children, which is not approved but is common medical practice. The panel decided to collect information on these 2 types of off-label use for descriptive purposes but to suspend its inclusion in the aggregate evaluative scores.

For standards 7 and 8, the development panel defined attributes of format and accuracy according to the FDA guidance document. Standard 7 specified that the information was scientifically accurate, unbiased, and up-to-date. To examine possible information overload, the word count of the CMI was also obtained but was excluded from the quality score for reasons of consistency with the FDA guidance document. The word count included only the text that provided medication information. Additional text on the leaflets, such as advertisements, general information about the disease state, and coupons, was excluded. To determine word count and reading level, each leaflet was scanned into a PDF file and then converted to a word document (Microsoft Word, Microsoft Corp, Redmond, Washington). Reading difficulty was determined using the Flesch-Kincaid Grade Level Index as follows¹⁴: (0.39 × ASL) + (11.8 × ASW) − 15.59, where ASL indicates average sentence length (number of words divided by number of sentences) and ASW, average number of syllables per word (number of syllables divided by number of words). Reading levels higher than eighth grade were considered inappropriate for meeting the criterion.

Space between lines of text and amount of white space around text were measured using calipers. Font size was determined using an “E-scale” transparent-type gauge and specifier set (AccuSpec II; The C-Thru Ruler Co, Bloomfield, Connecticut) using a template with a capital “E” as the standard to which the same capital letter in each leaflet was compared. These assessments were limited to the main body of the text that provided medication information. Advertisements for other products, store coupons, and Health Insurance Portability and Accountability Act statements, which were often included on the leaflets, were not considered.

The final draft evaluation forms were forwarded to a national panel of 8 experts (Evaluation Expert Panel), including physicians and clinical pharmacists, drug information specialists, and pharmacy educators. This panel was asked to review the FDA guidance document and to apply the draft criteria to a sample of 40 CMI leaflets. To determine interrater reliability, each CMI leaflet was evaluated by 2 panelists independently. Inconsistencies between raters were addressed as follows: criteria that were explicit but that raters had interpreted differently were reworded or examples were shared for clarification. A few criteria that required implicit judgment and resulted in a high degree of subjectivity were deleted from the final assessment form.

The final number of criteria included in each overall score was 77 for lisinopril (eTable 1) and 78 for metformin (eTable 2). The 8 evaluation panelists scored all the criteria except the last 11 in standard 8, which did not require judgment and were assessed by research staff. For each criterion in standards 1 to 6, evaluation panel raters indicated whether the information was present. Standards 7 and 8 asked experts (Evaluation Expert Panel) or research staff to assess whether the criteria were met. Twenty percent of CMI leaflets were assessed independently by 2 raters to evaluate interrater reliability. The percentage agreement between raters for the final set of criteria for lisinopril ranged from 72.0% to 100%, with a mean (SD) of 92.5% (6.6%), and for metformin was 80.9% to 100%, with a mean (SD) of 93.4% (5.3%).

Scores were reported as an overall aggregate score across all standards and criteria, for each individual standard (1-8), and for each individual criterion. Scores were summarized as means with 95% CIs. Reported univariate results for the relationship between independent vs chain pharmacies and CMI scores were compared using a 2-sample t test because data met the requirements for normality. Bar graphs to depict mean scores across standards and scatterplots to illustrate word count and content quality scores were constructed using the graphic tool in Microsoft Excel 2007 (Microsoft Corp). Data entry and analyses were performed using Microsoft Access 2007 (Microsoft Corp), and inferential statistics were computed using a software program (SPSS version 16.0; SPSS Inc, Chicago, Illinois).

Fifty-five of the 420 pharmacies sampled were excluded because shoppers were asked for identification or because the store was no longer in business. Shoppers filled prescriptions for lisinopril at 365 pharmacies and for metformin at 364 pharmacies in 41 states. Twenty-two pharmacies (6%) did not provide any written information beyond directions on the medication containers. The remaining 94% of pharmacies (95% CI, 91.5%-96.4%) provided CMI for lisinopril (n = 343) and metformin (n = 342). No pharmacy provided the official package insert. Leaflets ranged from 33 to 2482 words. For CMI with information about the publisher (57%), First Databank (33%) and Wolters Kluwer Health Inc (24%) were the most common.

Eleven prescriptions for lisinopril (3.0%; 95% CI, 1.5%-5.3%) and 1 for metformin (0.3%; 0.1%-1.5%) were accompanied by CMI that met 80% or more of all usefulness criteria (Table 2). Fourteen percent (95% CI, 10.6%-18.0%) of lisinopril and 16% (12.1%-19.8%) of metformin CMI leaflets had considerably low levels of quality, with scores of less than 40%.

Figure 1 shows the distribution of panelists' ratings for each of the 8 FDA standards. Scores varied across the standards, with the highest means obtained for standard 7 (scientific accuracy: 97.3% [95% CI, 95.8%-98.7%] for lisinopril and 97.4% [95.8%-98.9%] for metformin) and the lowest scores for standards 3 (directions for use and monitoring: 53.4% [51.4%-56.5%] and 45.6% [43.7%-47.6%]) and 8 (comprehensibility/legibility: 43.8% [42.6%-44.9%] and 42.6% [41.1%-43.7%]). Scores for each standard were similar between lisinopril and metformin, with the largest discrepancies in standards 3 and 5.

eTables 1 and 2 provide detailed information about each criterion. Most leaflets for lisinopril and metformin included generic names and indications for use. Brand names and physical descriptions of the medication were each provided in less than half of the CMI leaflets. The possibility of off-label use was mentioned on 84% of leaflets for lisinopril but on only 25% for metformin (excluded from summated scores). Specific off-label indications were provided on 22% of leaflets for lisinopril and on 12% for metformin, mainly for prevention of diabetic nephropathy and polycystic ovarian syndrome.

Ten percent of leaflets did not mention allergic reactions to angiotensin-converting enzyme inhibitors as a contraindication for use (standard 2). For metformin, appropriate listing of contraindications ranged from 40% for radiographic contrast agents to 89% for known hypersensitivity or allergic reaction. Usual dosing information was included in slightly more than one-third of leaflets for both medications, and actual personal dosing instructions were appended to the leaflet for 60% of leaflets for both medications (standard 3). Although slightly more than 70% of leaflets mentioned that monitoring was needed, detail on monitoring parameters or their frequency was rarely mentioned.

For metformin, 88% to 90% of the leaflets identified lactic acidosis, alcohol use, and pregnancy as precautions. Only 69% of metformin leaflets mentioned drug-drug interactions as a possible concern. Bone marrow disease was mentioned on only approximately 41% of lisinopril leaflets. Information on use in children, specifically, the lack of approval, was mentioned by less than half of the leaflets. At least 90% of the leaflets identified all of the serious adverse effects for lisinopril and metformin (standard 5). However, only 3% of lisinopril leaflets and 2% of metformin leaflets advised consumers to discontinue taking the medication immediately, although most advised contacting physicians when a serious adverse effect occurred.

Less than one-third of the leaflets used a font size of 10 points or larger (range, 5-12 points) (standard 8). Only 15% of leaflets for either medication met the criterion for space between lines of at least 2.2 mm, and only 7% used bullets when listing key points. Boldfaced text was rarely used for emphasis (6%), and less than 3% of CMI used boldfaced text in or boxes around black box warning information. Most common for providing emphasis was the use of all caps, which increases reading difficulty.¹⁵ Only 10% of lisinopril and 6% of metformin leaflets were written at or below the eighth-grade reading level, with a mean Flesch-Kincaid Grade Level Index score of 9.40 (95% CI, 9.26-9.54) for lisinopril and 9.94 (9.8-10.08) for metformin.

Surveyed pharmacies included 87 independent pharmacies, 252 chain outlets, and 4 franchise stores. All instances where prescriptions were dispensed without any CMI occurred in independent pharmacies. Chain pharmacies dispensed longer CMI leaflets, which met a larger percentage of the expert-required content (standards 1-6), with a mean difference of 22.1% (95% CI, 15.8%-28.4%) for lisinopril and 21.1% (95% CI, 14.9%-27.3%) for metformin (Table 3).

Large disparities in quality and length were found when different leaflets from the same publishers were compared. For example, examination of 2 metformin leaflets, both from chain outlets, indicated that First Databank was the publisher for both, with 2008 as the date of publication, yet 1 leaflet had 760 words and a 30% score for content and the other had 2457 words and an 88% score for content. In the shorter leaflet, the “Side Effects” section began with the statement “See also Warning Section,” but the warning section had been eliminated in the abbreviated CMI leaflet.

Although longer leaflets had higher levels of adherence to content criteria, conciseness varied substantially among CMI leaflets with higher content scores (Figure 2). Specifically, for the 92 lisinopril leaflets that met more than 80% of the content quality criteria (standards 1-6), the mean word count was 1523, with a minimum of 1112 and a maximum of 2106 words. Respective values for the 47 metformin leaflets were 1918, 1462, and 2482.

Excessive text resulted oftentimes in poor formatting because font size and line spacing was altered to accommodate page limits. Figure 3 depicts a leaflet with excellent format yet a low content score, and Figure 4 illustrates a poorly formatted leaflet with adequate content quality.

Although white space around text is seen as a positive feature because it enhances readability, examination of CMI leaflets found that free space was often used for information not pertinent to the medication. Extraneous information included advertisements, coupons, bible quotes, information about the general disease state, and Health Insurance Portability and Accountability Act regulations (Figure 5).

Shrank et al¹⁶ summarized concerns about the quality of written prescription information and its effect on comprehensibility and concluded that the “current patchwork of bad communication and excessive promotion . . . [should be replaced] . . . with a responsible national system of balanced, evidence-based, and user-friendly drug information.”¹⁷ The present data ascertained from a nationwide sample of retail pharmacies confirm this notion and underscore the concern that private sector initiatives to provide quality written drug information have failed to meet FDA guidance criteria for useful written information.

This study found that most pharmacies provided computer-generated CMI. However, the length and format and the presence of critical content varied considerably. Many leaflets failed to meet the minimum requirements, such as provision of a complete list of absolute contraindications, and more than half lacked specific directions that would allow patients to manage problems. Because CMI was the sole written information dispensed, some patients had no information about the risk of lactic acidosis associated with metformin or related warning signs or action steps.

The high reading level required to comprehend the presented information and the inadequate formatting suggest additional shortcomings. Whereas the severity of inappropriate reading levels for patient comprehension seems evident, formatting does have similar effects. Effects on comprehension were recently illustrated in a randomized trial of direct-to-consumer advertisements where fine print was replaced by a drug facts box, resulting in significantly better understanding of drug risk and benefit.¹⁸

Consumer medication information is typically generated electronically as part of the medication dispensing process. The CMI content is defined by a few private vendors, and the CMI formatting is determined by pharmacies or their software vendors. As a result, we found CMI leaflets from the same publisher with the same date of publication but with different content and appearance. The formatting of CMI was further compromised by extensive use of the leaflet for promotional messages. Considering the sheer volume of information in addition to the formatting of CMI, it is not surprising if consumers treat CMI as they would manuals provided with electronic equipment: they defer to the “quick start” and hope that they never need to consult the remainder.

Although the longer the leaflet, the more the content criteria were met, the efficiency and conciseness with which content is presented is an important issue given concerns about overload so that leaflets are not read and warnings not retained. In this context, content quality in this study was defined by the presence of information, resulting in better scores for high-volume leaflets. However, leaflets that “cover all the bases” to protect against potential litigation may fail to meet their primary purpose in instructing patients on how to derive the most benefit from therapy. Effects of information overload on comprehension and retention and respective integration in quality standards for useful CMI should be addressed in future research.

In examining standard drug monographs and package information, it was unclear what criteria were used by publishers to select information for consumers. It is disputable whether any medication that could potentially (but not necessarily practically) interact with the prescribed medication should be listed or whether restriction to contraindicated medications might suffice. Considering the purpose of the package insert as a legal document to limit the sponsor's promotional activities and to protect from liability vs its use for patient education in CMI, a glaring disconnect becomes apparent. For example, although the drug indication is validated through randomized clinical trials, evidence that guides precautionary statements is often not validated, and the clinical relevance of such warnings is questionable or unknown. Such information is relevant in a legal document but is potentially useless in patient education. On the other hand, common off-label uses that may not be included in the package insert would be useful for many patients. For example, a woman who received metformin for polycystic ovarian syndrome may be confused by a leaflet that lists only diabetes mellitus as an indication. The issue of validating evidence that has not undergone stringent FDA review and approval certainly raises other concerns that must be addressed in any comprehensive consumer information.¹⁹

These broader challenges in information selection and the demonstrated shortcomings identified in the study at hand suggest that an alternative approach in developing useful CMI is needed. This should include explicit guidance and regulatory authority through the FDA. Perhaps sponsor-initiated and FDA-guided models as established with package inserts but with distinct differences in content, reading level, and formatting, present a workable solution. The ability to generate more customized information (eg, medications that mention pregnancy precautions dispensed only for prescriptions for women) could help provide more efficient CMI. In addition, a “tiered” level of information that could meet the needs of those with limited literacy along with access to more extensive information that can be requested or accessed online could perhaps overcome the difficulties of a “one size fits all” approach to CMI. Finally, to ensure effective communication, enhanced research efforts regarding content selection and presentation and their effect on consumer comprehension and retention are needed.

Study limitations include the inability to generalize to excluded settings, such as mail order outlets, and the perhaps limited definition of quality according to the presence of content (as opposed to consideration of overload or conciseness). The weight or importance of each included quality criterion remains undefined and should be the focus of further research, especially in light of concerns about information overload and the effect of formatting on patient comprehension.

In conclusion, although CMI distribution through pharmacies seems to be effective, the content, format, reading level, and excessive length are disconcerting. Private sector initiatives to ensure the provision of useful CMI to patients have failed to meet the standards for useful, readable information. Further research needs to address the quantity, presentation, and format of CMI that will result in adequate patient comprehension and, ultimately, appropriate actions to improve patient safety. It is, furthermore, unclear whether content in CMI is selected according to the greatest applicability to patient concerns and their ability to manage drug therapy and whether the official labeling should be the only source to guide content in CMI. The usefulness of CMI ultimately depends on meeting the needs of patients for information that facilitates the understanding and management of their therapies.

Correspondence: Almut G. Winterstein, PhD, Pharmaceutical Outcomes and Policy, College of Pharmacy, College of Public Health and Health Professions, University of Florida, Box 100496, Gainesville, FL 32610 (almut@cop.ufl.edu).

Accepted for Publication: January 18, 2010.

Author Contributions:Study concept and design: Winterstein and Kimberlin. Acquisition of data: Winterstein, Linden, Fernandez, and Kimberlin. Analysis and interpretation of data: Winterstein, Linden, Lee, Fernandez, and Kimberlin. Drafting of the manuscript: Winterstein, Lee, Fernandez, and Kimberlin. Critical revision of the manuscript for important intellectual content: Winterstein, Linden, and Kimberlin. Statistical analysis: Winterstein, Linden, and Kimberlin. Obtained funding: Kimberlin. Administrative, technical, and material support: Winterstein, Linden, Lee, Fernandez, and Kimberlin. Study supervision: Winterstein and Kimberlin.

Financial Disclosure: None reported.

Disclaimer: The views expressed in written conference materials or publications and by speakers and moderators at US Department of Health and Human Service–sponsored conferences do not necessarily reflect the official policies of the US Department of Health and Human Service; nor does mention of trade names, commercial practices, or organizations imply endorsement by the US government.

Previous Presentation: This study was presented at the FDA/Center for Drug Evaluation and Research Risk Communication Advisory Committee Meeting; February 26, 2009; Washington, DC. The final report is available at the FDA Web site (http://www.fda.gov/downloads/AboutFDA/CentersOffices/CDER/ReportsBudgets/UMC163783.pdf).

Online-Only Material: eTables 1 and 2 are available at http://www.archinternmed.com.

Additional Contributions: Jodi Duckhorn, MA, at the FDA, Center for Drug Evaluation and Research, provided guidance and excellent project management.

Expert Panel Members: Criteria Development Expert Panel:University of Florida: Timothy J. Grainger Rousseau, RPh, PhD, College of Pharmacy; Randy C. Hatton, PharmD, FCCP, BCPS, Drug Information and Pharmacy Resource Center, Shands Jacksonville, M. Cecilia Lansang, MD, MPH, Endocrinology, Diabetes & Metabolism, and Eric I. Rosenberg, MD, MPH, Department of Medicine, College of Medicine. Evaluation Expert Panel: Bernadette S. Belgado, PharmD, Drug Information Center, Shands Jacksonville. University of Kentucky, Lexington: Heidi Anderson Harper, PhD, College of Pharmacy. University of Maryland, Baltimore: Robert S. Beardsley, RPh, PhD, Pharmaceutical Health Services Research, School of Pharmacy. University of Minnesota, Minneapolis: Michael C. Brown, PharmD, College of Pharmacy. Exempla Lutheran Medical Center, Wheat Ridge, Colorado: Cristin Sarmiento, PharmD. M. D. Anderson Cancer Center, Houston, Texas: Wendy D. Smith, PharmD, BCPS. University of North Carolina, Chapel Hill: Betsy L. Sleath, PhD, School of Pharmacy. Purdue University, West Lafayette, Indiana: Joseph Thomas III, PhD, School of Pharmacy and Pharmaceutical Sciences.