Physicians' Understanding of the Regulation of Dietary Supplements FREE

The Dietary Supplement Health and Education Act (DSHEA) of 1994 dramatically changed the way in which vitamins, herbs, and other supplements are regulated.¹ Under DSHEA, supplements are not required to have Food and Drug Administration (FDA) approval before their production or distribution, and no safety or efficacy studies are required before they are marketed. Products are assumed to be safe unless proved otherwise by the FDA through postmarket surveillance.² The regulation of dietary supplements is in stark contrast to the regulation of drugs, which require premarket safety and efficacy review by the FDA. The DSHEA did give the FDA authority to establish Good Manufacturing Practices for supplements, similar to those for drugs, in an effort to ensure the quality of products that are available to the public. To date, however, no such regulations have been enacted, and cases of dietary supplement adulteration have been reported.³

As a result of the virtual deregulation of the dietary supplement industry, sales of these over-the-counter products increased substantially. Millions of Americans have spent billions of dollars on supplements in an effort to foster and maintain their health.⁴ Yet, the public seems unaware of the lack of governmental oversight of dietary supplements and may be ingesting these products with a false sense of security regarding their efficacy and safety.⁵ Physicians therefore have a responsibility to educate their patients regarding the regulation of supplements so that an informed decision about their use may be made. Also, physicians need to be cognizant of the potential for adverse events due to supplement use and of the process to report such unfortunate occurrences. To date, little is known about physicians' knowledge of dietary supplement regulation or adverse event reporting. Many medical schools have incorporated complementary and alternative medicine teaching into their curricula, but no standardization of content exists.^6- 8 We hypothesized that internal medicine postgraduate physicians' knowledge of dietary supplement regulation and adverse event reporting was poor, and could be improved by an online, case-based, interactive curriculum.

A didactic module on dietary supplement regulation was developed using a 6-step approach to curriculum development.⁹ The following 5 content objectives of the pretest and posttest questions (Q1-Q5) were identified for study after discussion with experts in the field:

Ten questions (5 pretest and 5 posttest) of clinical cases were written that described patients presenting with inquiries regarding dietary supplements that they were taking or planning to take. Multiple answer choices were provided, requesting the learner to select the 1 most appropriate answer. The face validity and content validity of the questions were obtained by having the questions reviewed by 5 experts in complementary and alternative medicine, dietary supplement regulation, and/or medical education; then, the questions were revised until a consensus was reached that the questions were clear and could be answered from the choices offered. We also pilot tested the questions in random fashion on 10 medical residents and found no significant difference between correct answers for the pretest and posttest questions (P = .80). Registered users began the module by completing the pretest. They were then able to access the didactic section, which consisted of a case-based interactive curriculum on supplement regulation and adverse event reporting, consisting of case descriptions with multiple-choice questions and answers. The multiple-choice questions were followed by descriptions of the correct and incorrect responses as well as by further information presented in text and tabular format. The posttest questions could not be accessed unless the didactic section had been completed. For the pretest and posttest questions, learners were informed if their answer choice was correct, and if incorrect, which answer choice was correct. Each pretest and posttest question was based on 1 specific content area and objective.

The dietary supplement module was used by 15 internal medicine residency training programs throughout the United States. Study subjects were physicians at each participating residency training program (internal medicine house staff and faculty) who had registered, been approved to use the curriculum Web site (http://www.hopkinsilc.org), and completed the module voluntarily. Residency program directors typically chose which modules from the entire curriculum that they wanted their residents to complete. Participating residency training programs included primary affiliates of medical schools and several community hospitals.¹⁰ The institutional review board of The Johns Hopkins University School of Medicine, Baltimore, Md, granted exemption to this project and allowed the study to proceed without requiring participants to provide specific informed consent. All registrants of the Web site were informed that their answers might be used for educational research but that their identities would be removed.

Performance data were tabulated by the Web site from March 1, 2006, through June 30, 2006. Results of partially completed modules were not included. Responses to the case scenario and management questions were tabulated electronically by the Web site and then analyzed based on year of training, attending physician status, and residency training program. Statistical analyses were performed with Stata software, version 8.2 (Stata Corp, College Station, Tex). Comparisons between pretest and posttest scores and comparisons of pretest scores at different training levels were performed using the χ² test. All P values were 2-sided, and P<.05 was considered significant.

A total of 1541 physicians at 15 residency training programs had access to the module, which was completed in its entirety by 335 physicians, for a participation rate of 22%. Participation rates varied among residency training programs. To test for selection bias among physicians who completed the study, the performance of the 3 residency training programs with the highest participation rates was compared with that of the 3 residency programs with the lowest participation rates. There was no difference in performance shown with this analysis (P = .12). Of the 335 physicians who completed the module, 69 (20.6%) were in postgraduate year (PGY)-1, 114 (34.0%) were in PGY-2, 118 (35.2%) were in PGY-3, 33 (9.9%) were attending physicians, and 1 (0.3%) did not state his or her level of training.

Baseline knowledge of dietary supplement regulatory issues was low (Figure). The average pretest score was 58.8%, suggesting that physicians may need further education on such issues. More than one third (37%) of physicians were unaware that dietary supplements did not require FDA approval before being sold (Q1). Similar results were seen for the other questions regarding supplement regulation (Q2-Q4). Even fewer correct responses were seen in regard to adverse event reporting (Q5). Nearly two thirds (60%) of physicians were unclear on how to proceed in the event of a serious adverse event potentially due to use of a supplement. After completion of the curriculum, scores improved significantly (P<.001) in all question/content areas (Figure). After completing the module, the average posttest score rose to 91.1%.

The ability to answer questions about supplement regulation and adverse event reporting appropriately did not increase by level of residency training. However, the scores for attending physician baseline knowledge regarding the lack of an FDA requirement for dietary supplement approval (84.8% vs 61.1%; P = .007) and the lack of an efficacy requirement for dietary supplement approval (87.9% vs 68.4%; P = .002) were significantly higher than resident scores. Attending pretest scores on other concepts did not differ significantly from postgraduate trainee scores.

We demonstrated that resident and attending physician knowledge of key concepts of dietary supplemental regulation is poor and that it can be improved by an interactive curriculum. Our study adds to the finding of previously published surveys that reported that physicians desire more training in complementary and alternative medicine in general.¹¹ We found that knowledge on all concepts did not vary between PGY-1 and PGY-3 residents, suggesting that training on these concepts is not occurring. Attending physicians have better baseline knowledge on some, but not all, aspects of dietary supplement regulation. Knowledge of governmental regulation has been proposed as a core competency for medical school curricula in integrative medicine.¹² Our study suggests that physicians have not achieved this level of competency and are not equipped to fulfill their educational duties adequately on this subject. The online interactive curriculum that we developed may begin to bridge this gap in knowledge.

A number of reports of adverse events and supplement-drug interactions have appeared over recent years.^13- 16 The FDA MedWatch program has been set up to monitor the safety of drugs, devices, biologicals, and dietary supplements. Adverse event reporting for dietary supplements has historically relied heavily on patients and practitioners, since manufacturers have only recently been mandated to report serious side effects. Our study suggests that physicians are unaware of the appropriate mechanism for reporting adverse events. This finding is consistent with a report from the Department of Health and Human Services, Office of the Inspector General, which noted that very few dietary supplement adverse events actually get reported, and most of those seem to be generated by consumers.¹⁷ The report recommended that the FDA expand its outreach to health professionals in regard to education about its MedWatch system. Internet curricula such as ours may serve as a valuable adjunct to other governmental outreach programs.

There are several limitations to our study. First, it is possible that physicians who completed the module did so only because they believed that their knowledge regarding supplement regulation was poor. However, scores at residency training programs with low use were not lower than those at residency training programs with high use, making selection bias based on poor knowledge unlikely. Second, test questions may not fully evaluate the extent of knowledge even though they were validated by experts. Finally, the outcome measure for our curriculum was short-term retention. It is not known whether this improvement in knowledge will be sustained. However, the flexibility of an online program allows for periodic repetition to further bolster long-term retention.

The DSHEA has come under a great deal of scrutiny as a result of highly publicized cases of adverse events.^18- 19 Despite this, patients continue to spend billions of dollars on dietary supplements, frequently not realizing that these products have not been evaluated by the government before being marketed. Physicians have a responsibility to openly discuss supplement uses, efficacy, and safety as well as the potential for adverse events.²⁰ Part of this discussion should include information regarding regulatory issues. This study demonstrates that physicians currently do not have adequate knowledge to engage in such discussions with patients. An online curriculum can serve to improve knowledge and to enhance patient-physician communication so that patients can make truly informed choices.

Correspondence: Bimal H. Ashar, MD, MBA, 601 N Caroline St, Suite 7143, Baltimore, MD 21287-0941 (bashar@jhmi.edu).

Accepted for Publication: January 3, 2007.

Author Contributions:Study concept and design: Ashar and Sisson. Acquisition of data: Ashar and Sisson. Analysis and interpretation of data: Ashar, Rice, and Sisson. Drafting of the manuscript: Ashar. Critical revision of the manuscript for important intellectual content: Ashar, Rice, and Sisson. Statistical analysis: Rice. Administrative, technical, and material support: Ashar. Study supervision: Sisson.

Financial Disclosure: Dr Sisson is paid a supplemental salary as an editor of the curriculum contained on the Johns Hopkins Web site.

Additional Information: Individuals interested in accessing the study curriculum can do so free of charge by logging onto http://www.hopkinsilc.org, clicking on “first-time user,” selecting “Internal Medicine Curriculum,” and then selecting “Demonstration Group” as their user group.

Acknowledgment: We thank Bennett Lee, MD, MPH, Redonda Miller, MD, MBA, Brent Petty, MD, and Jon Tilburt, MD, MPH, for their assistance in pretest and posttest question development.