0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letters |

Associations Between Web-Based Patient Ratings and Objective Measures of Hospital Quality FREE

Felix Greaves, BMBCh; Utz J. Pape, PhD; Dominic King, MBChB; Ara Darzi, MD; Azeem Majeed, MD; Robert M. Wachter, MD; Christopher Millett, PhD
[+] Author Affiliations

Author Affiliations: Departments of Primary Care and Public Health (Drs Greaves, Pape, Majeed, and Millett) and Surgery and Cancer (Drs King and Darzi), Imperial College London, London, England; and Division of Hospital Medicine, University of California, San Francisco (Dr Wachter).


Arch Intern Med. 2012;172(5):435-436. doi:10.1001/archinternmed.2011.1675.
Text Size: A A A
Published online

Patients are increasingly using the Internet to rate their experience of health care.1 Although controversial, particularly among health care professionals,2 Web-based hospital or physician rating sites represent a potentially important development in public reporting.3 Patients rarely use conventional publicly reported metrics when choosing their health care provider and often find these difficult to understand.4

In 2008, the English National Health Service (NHS) established a Web site called NHS Choices that allows patients to rate their experiences with health care providers. Patients can rate several dimensions of quality on simple scales and leave comments on particular aspects of care in free text. As of October 2011, Medicare has not chosen to include patient comments and ratings, despite recent calls for their introduction.5

Although some prior research has shown an association between results on formal patient satisfaction surveys and clinical measures of quality,6 to our knowledge, no previous study has examined the relationship between unsolicited ratings by patients and objective measures of quality. The degree to which such ratings are associated with objective clinical outcomes and conventional patient surveys may well determine their place in the landscape of transparency measures. In addition, decisions by policy makers and payers to support such Web sites might well hinge on whether such associations exist. This study examines hospital-level associations between Web-based patient ratings on the NHS Choices Web site and objective measures of quality.

We performed a cross-sectional observational study of all (n = 166) NHS acute hospital trusts in England, using data from 10 274 patient Web-based ratings posted on the NHS Choices Web site from January 1, 2009, to December 31, 2010, obtained from the English Department of Health. Hospital trust–level indicators of clinical outcomes (5 measures) and health care-acquired infections (2 measures) were compared with Web-based patient ratings using the Spearman rank correlation coefficient. When comparing patient ratings with infection rates, we used all 166 acute care hospital trusts in England. When comparing ratings with other clinical outcomes, a reduced list of 146 hospital trusts was used that excluded trusts that only care for children or specific specialties (eg, oncology). Data on clinical outcomes used were for financial year 2009/2010, and obtained from Dr Foster (a health intelligence company) and NHS Comparators (the NHS's comparative health system performance service). We obtained data on methicillin-resistant Staphylococcus aureus (MRSA) bacteremia and Clostridium difficile infection rates from the Health Protection Agency compulsory national surveillance system.

The mean and median number of ratings per hospital was 62 and 46, respectively (range, 1-290). Of those who offered a view about recommending the hospital (9349 ratings), 68.0% (n = 6354) would recommend the hospital to a friend. Positive recommendations of hospitals were significantly associated with lower hospital standardized mortality ratios (Spearman ρ = −0.20; P = .01), lower mortality from high-risk conditions (ρ = −0.23; P = .01), and lower readmission rates (ρ = −0.31; P < .001) (Table). Positive recommendations were not associated with mortality rates among surgical inpatients with serious treatable complications (ρ = 0.00; P = .99) or mortality from low-risk conditions (ρ = 0.03; P = .70). Mean and median ratings of hospital cleanliness were 3.6 (range, 2.6-5.0) on a scale from 1 (dirty) to 5 (exceptionally clean). Better ratings of hospital cleanliness were associated with lower MRSA (ρ = −0.30, P < .001) and C difficile (ρ = −0.16; P = .04) infection rates.

Table Graphic Jump LocationTable. Hospital Trust–Level Associations Between Web-Based Patient Ratings and Clinical Outcomes

Our results demonstrate a relationship between patients' Web site ratings of hospitals and some objective measures of clinical quality, including mortality and infection rates.

Arguments in favor of the use of patient Web site reporting include that physicians can be poor judges of their patients' experience, that feedback changes physicians' performance, and people will inevitably use the Internet to voice opinions, so why not capture this information in a useful form.3 Arguments against using this data include the selection bias by those leaving reviews, the lack of meaningful data on technical quality of health care, and straining of physician-patient relationships.2,5 Although our results do not counter all of these arguments against, they suggest that discretionary patient ratings, obtained through a Web site, may be a more useful tool than previously considered for both patients and health care workers. If patients are making choices based on this information, they can be reassured that the ratings are not entirely misleading and may be providing relevant information about health care quality. In his book The Wisdom of Crowds, James Surowiecki7 argues that a diverse collection of “independently deciding individuals” is likely to make better predictions and decisions than single individuals or even experts. At least to an extent, the self-selecting crowd of patients appears to be wise.

The use of Web-based patient ratings has become common in other industries such as hotels and restaurants, and consumers value these rankings in making choices. We believe that the information provided by these Web sites, although flawed, represents a potentially important development in the measurement of health care quality.

Correspondence: Dr Greaves, Department of Primary Care and Public Health, Reynolds Building, Charing Cross Campus, Imperial College London, London W6 8RP, England (felix.greaves08@imperial.ac.uk).

Published Online: February 13, 2012. doi:10.1001/archinternmed.2011.1675

Author Contributions: All authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Greaves and Millett. Acquisition of data: Greaves. Analysis and interpretation of data: Greaves, Pape, King, Darzi, Majeed, Wachter, and Millett. Drafting of the manuscript: Greaves, King, and Millett. Critical revision of the manuscript for important intellectual content: Pape, Darzi, Majeed, and Wachter. Statistical analysis: Greaves and Pape. Obtained funding: Majeed. Administrative, technical, and material support: King, Darzi, and Millett. Study supervision: Darzi, Majeed, and Millett.

Financial Disclosure: Dr Wachter reports having an equity interest and/or serving on paid advisory boards for PatientSafe Solutions and CRISI; receiving support for helping to lead a leadership training program for IPC–The Hospitalist Company; receiving honoraria from the American Board of Internal Medicine for serving on its board of directors and executive committee; receiving honoraria for many speeches on patient safety and quality; receiving support from John Wiley and Sons for writing a blog; and receiving funding under a contract from the Agency for Healthcare Research and Quality for editing 2 patient-safety Web sites and royalties from publishers from 2 books on patient safety.

Funding/Support: Dr Wachter is a 2011 recipient of a Fulbright Award for study in the United Kingdom. Dr Millett is funded by the Higher Education Funding Council for England and the National Institute for Health Research.

Role of the Sponsors: The funding sources had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript.

Additional Information: Dr Darzi was Parliamentary Under-Secretary of State (Lords) in the United Kingdom Department of Health from 2007 to 2009.

Additional Contributions: We thank the team at NHS Choices, and John Robinson in particular, for providing access to their data. The Department of Primary Care & Public Health at Imperial College is grateful for support from the National Institute for Health Research Biomedical Research Centre Funding scheme, the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care scheme, and the Imperial Centre for Patient Safety and Service Quality.

Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients' evaluations of health care providers in the era of social networking: an analysis of physician-rating websites.  J Gen Intern Med. 2010;25(9):942-946
PubMed   |  Link to Article
McCartney M. Will doctor rating sites improve the quality of care? no.  BMJ. 2009;338:b1033
PubMed   |  Link to Article
Bacon N. Will doctor rating sites improve standards of care? yes.  BMJ. 2009;338:b1030
PubMed   |  Link to Article
Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care.  Ann Intern Med. 2008;148(2):111-123
PubMed
Lagu T, Lindenauer PK. Putting the public back in public reporting of health care quality.  JAMA. 2010;304(15):1711-1712
PubMed   |  Link to Article
Isaac T, Zaslavsky AM, Cleary PD, Landon BE. The relationship between patients' perception of care and measures of hospital quality and safety.  Health Serv Res. 2010;45(4):1024-1040
PubMed   |  Link to Article
Surowiecki J. The Wisdom of Crowds. New York, NY: Doubleday; 2004

Figures

Tables

Table Graphic Jump LocationTable. Hospital Trust–Level Associations Between Web-Based Patient Ratings and Clinical Outcomes

References

Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients' evaluations of health care providers in the era of social networking: an analysis of physician-rating websites.  J Gen Intern Med. 2010;25(9):942-946
PubMed   |  Link to Article
McCartney M. Will doctor rating sites improve the quality of care? no.  BMJ. 2009;338:b1033
PubMed   |  Link to Article
Bacon N. Will doctor rating sites improve standards of care? yes.  BMJ. 2009;338:b1030
PubMed   |  Link to Article
Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care.  Ann Intern Med. 2008;148(2):111-123
PubMed
Lagu T, Lindenauer PK. Putting the public back in public reporting of health care quality.  JAMA. 2010;304(15):1711-1712
PubMed   |  Link to Article
Isaac T, Zaslavsky AM, Cleary PD, Landon BE. The relationship between patients' perception of care and measures of hospital quality and safety.  Health Serv Res. 2010;45(4):1024-1040
PubMed   |  Link to Article
Surowiecki J. The Wisdom of Crowds. New York, NY: Doubleday; 2004

Correspondence

CME
Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.
Submit a Comment

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 19

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
JAMAevidence.com