0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letter |

Website Ratings of Physicians and Their Quality of Care FREE

Bradley M. Gray, PhD1; Jonathan L. Vandergrift, MS1; Guodong (Gordon) Gao, PhD2; Jeffrey S. McCullough, PhD3; Rebecca S. Lipner, PhD1
[+] Author Affiliations
1American Board of Internal Medicine, Philadelphia, Pennsylvania
2Robert H. Smith School of Business, University of Maryland, College Park
3Division of Health Policy and Management, University of Minnesota, Minneapolis
JAMA Intern Med. 2015;175(2):291-293. doi:10.1001/jamainternmed.2014.6291.
Text Size: A A A
Published online

One-third of consumers in the United States who consulted physician website ratings reported selecting and/or avoiding physicians because of these ratings.1 However, little is known about the validity of these ratings. Available studies have focused mostly on hospital website ratings or non-US website ratings.2,3 We partially address this gap by measuring the association between US physician website ratings and traditional quality measures (QMs) of clinical and patient experience.

We used a sample of 1299 physicians who completed an American Board of Internal Medicine Hypertension or Diabetes Practice Improvement Module between July 1, 2011, and November 30, 2012. Quality measures were drawn from about 25 Practice Improvement Module medical record abstractions and patient survey responses (59% response rate per physician). From medical record abstractions, we computed overall, intermediate outcome, and clinical process-of-care composites based on an expert panel’s assessment of quality.4 We also computed 2 QMs each for clinical and patience experience.

Website physician rating measures were drawn from Internet searches in which each physician’s name, specialty, and city were entered into the Google search engine. We extracted information from 8 free publicly available leading health-based websites. This information included physician rating, the number of patient ratings per physician, and search ranking. We normalized physician website ratings by dividing each rating by the website’s maximum score (eg, 4 of 5 stars equals 80%).

Physician-level QMs were regressed on website ratings. Regressions were estimated separately for each QM and controlled for physician, patient, and website characteristics. We estimated a model using all 8 websites and another model limited to the website with the highest Google search result as it may be more representative of what consumers view.

The project was reviewed by the University of Maryland College Park institutional review board and was determined to be exempt from institutional review board review. All data applied in this analysis were analyzed anonymously. Physicians who enroll in an American Board of Internal Medicine certification program enter into a business associates agreement that permits the American Board of Internal Medicine use of their de-identified data at an aggregate level for research purposes (www.abim.org/privacy.aspx). After linking publicly available rating data from the Internet, we were blinded to the physicians’ identities and the data were de-identified and were viewed and analyzed in aggregate. Patient survey data did not include any protected health information or other identifiers and were collected by physicians participating in the American Board of Internal Medicine’s maintenance of certification program; therefore, informed consent could not be collected. Furthermore, no names of patients were recorded.

Physician website ratings existed for 61.0% of physicians, with 5.6 patient ratings per physician and a mean normalized rating of 81.6% (Table 1).

Table Graphic Jump LocationTable 1.  Physician Website Ratings From Health Websites Identified by Google Searchesa

The associations between physician website ratings and clinical QMs were small and statistically insignificant (≤0.3 percentage point change associated with a 20 percentage point rating change; P > .41) (Table 2). For patient experience QMs, associations were also small but were statistically significant (≤1.7 percentage point change associated with a 20 percentage point rating change; P < .05). For example, regression results indicate that a physician with a website rating of just 1 of 5 stars had 79% of their patients rate overall quality of their care as very good or excellent vs 82% for a physician who had a perfect 5 out of 5 rating. Overall, results were similar across website samples.

Table Graphic Jump LocationTable 2.  Association Between Quality Measures and a 20-Percentage-Point Change in Normalized Physician Website Ratingsa

We found no evidence that physician website ratings were associated with clinical QMs. We did find a statistically significant, but small, association between physician website ratings and 2 Practice Improvement Module measures of patient experience. Overall, the weak associations we found could have resulted from the low number of website ratings per physician or because patients whose ratings are reported on websites are not typical of the overall population of patients treated by the physicians in our sample. Alternatively, weak associations may have been due to inherent limitations of our Practice Improvement Module data related to their content, sample structure, and chronic condition focus. Finally, we did not examine associations with patients’ narratives on the website.

Notably, a study of ratings for websites in the United Kingdom reported stronger associations with practice-level QMs than we found.2 More generally, patient experience measures, properly constructed and administered, have been shown to be associated with clinical outcomes.5 Regardless of whether US physician website ratings are related to the QMs we examined, these measures of patient experience may capture other important aspects of quality of care. Furthermore, the high prevalence and use of physician website ratings by US consumers indicates that either there might be a benefit not reflected in our QMs or consumers have no other easily accessible source of information.1,6 Overall, our study provides valuable information to consumers considering the usefulness of physician website ratings.

Corresponding Author: Bradley M. Gray, PhD, American Board of Internal Medicine, 510 Walnut St, Ste 1700, Philadelphia, PA 19106 (bradleygraypub2014@gmail.com).

Published Online: December 1, 2014. doi:10.1001/jamainternmed.2014.6291.

Author Contributions: Dr Gray had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Gray, Gao, McCullough, Lipner.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Gray, Vandergrift, McCullough, Lipner.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Gray, Vandergrift, McCullough, Lipner.

Administrative, technical, or material support: Vandergrift, Gao, Lipner.

Study supervision: Gray, Lipner.

Conflict of Interest Disclosures: Drs Gray and Lipner and Mr Vandergrift are paid employees at the American Board of Internal Medicine.

Funding/Support: The American Board of Internal Medicine provided financial and material support for this study. This study was partially supported by National Science Foundation Career Award 1254021 (Dr Gao).

Role of the Funder/Sponsor: The American Board of Internal Medicine had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Hanauer  DA, Zheng  K, Singer  DC, Gebremariam  A, Davis  MM.  Public awareness, perception, and use of online physician rating sites. JAMA. 2014;311(7):734-735.
PubMed   |  Link to Article
Greaves  F, Pape  UJ, Lee  H,  et al.  Patients’ ratings of family physician practices on the Internet: usage and associations with conventional measures of quality in the English National Health Service. J Med Internet Res. 2012;14(5):e146.
PubMed   |  Link to Article
Bardach  NS, Asteria-Peñaloza  R, Boscardin  WJ, Dudley  RA.  The relationship between commercial website ratings and traditional hospital performance measures in the USA. BMJ Qual Saf. 2013;22(3):194-202.
PubMed   |  Link to Article
Hess  BJ, Weng  W, Lynn  LA, Holmboe  ES, Lipner  RS.  Setting a fair performance standard for physicians’ quality of patient care. J Gen Intern Med. 2011;26(5):467-473.
PubMed   |  Link to Article
Anhang Price  R, Elliott  MN, Zaslavsky  AM,  et al.  Examining the role of patient experience surveys in measuring health care quality.  Med Care Res Rev. 2014;71(5):522-554.
PubMed   |  Link to Article
Gao  GG, McCullough  JS, Agarwal  R, Jha  AK.  A changing landscape of physician quality reporting: analysis of patients’ online ratings of their physicians over a 5-year period. J Med Internet Res. 2012;14(1):e38.
PubMed   |  Link to Article

Figures

Tables

Table Graphic Jump LocationTable 1.  Physician Website Ratings From Health Websites Identified by Google Searchesa
Table Graphic Jump LocationTable 2.  Association Between Quality Measures and a 20-Percentage-Point Change in Normalized Physician Website Ratingsa

References

Hanauer  DA, Zheng  K, Singer  DC, Gebremariam  A, Davis  MM.  Public awareness, perception, and use of online physician rating sites. JAMA. 2014;311(7):734-735.
PubMed   |  Link to Article
Greaves  F, Pape  UJ, Lee  H,  et al.  Patients’ ratings of family physician practices on the Internet: usage and associations with conventional measures of quality in the English National Health Service. J Med Internet Res. 2012;14(5):e146.
PubMed   |  Link to Article
Bardach  NS, Asteria-Peñaloza  R, Boscardin  WJ, Dudley  RA.  The relationship between commercial website ratings and traditional hospital performance measures in the USA. BMJ Qual Saf. 2013;22(3):194-202.
PubMed   |  Link to Article
Hess  BJ, Weng  W, Lynn  LA, Holmboe  ES, Lipner  RS.  Setting a fair performance standard for physicians’ quality of patient care. J Gen Intern Med. 2011;26(5):467-473.
PubMed   |  Link to Article
Anhang Price  R, Elliott  MN, Zaslavsky  AM,  et al.  Examining the role of patient experience surveys in measuring health care quality.  Med Care Res Rev. 2014;71(5):522-554.
PubMed   |  Link to Article
Gao  GG, McCullough  JS, Agarwal  R, Jha  AK.  A changing landscape of physician quality reporting: analysis of patients’ online ratings of their physicians over a 5-year period. J Med Internet Res. 2012;14(1):e38.
PubMed   |  Link to Article

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment
ABIM scam
Posted on December 4, 2014
george fernandez
none
Conflict of Interest: None Declared
expected of the ABIM.another biased, pro-ABIM article, in order to continue their monopoly on the medical \"quality\" business.ABIM, the same organization that in exchange for co-operation from the medical establishment, for over 25 years (since 1991), granted some drs \"for-life\" BC status. so a cardiologist trained in MI treatment in the 70's-80's of bed rest and morphine, was presented to the general public as being qualified and representing the Board's \"the gold standard\". a lie, deception and a fraud.reason for which all physicians should revolt against this self serving multi-billion dollar business enterprise and corporation that has done nothing to improve quality or patient outcome. US: highest healthcare cost in the world, and 46 in healthcare outcomes and quality, despite increased number of board certifications. ABIM must be made to justify their unsubstantiated claims and investigated.
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

2,686 Views
2 Citations
×

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
Jobs
JAMAevidence.com

Care at the Close of Life: Evidence and Experience
What Should the Physician Expect of the Hospital-Based Palliative Care Service?