Associations Between Web-Based Patient Ratings and Objective Measures of Hospital Quality FREE

Patients are increasingly using the Internet to rate their experience of health care.¹ Although controversial, particularly among health care professionals,² Web-based hospital or physician rating sites represent a potentially important development in public reporting.³ Patients rarely use conventional publicly reported metrics when choosing their health care provider and often find these difficult to understand.⁴

In 2008, the English National Health Service (NHS) established a Web site called NHS Choices that allows patients to rate their experiences with health care providers. Patients can rate several dimensions of quality on simple scales and leave comments on particular aspects of care in free text. As of October 2011, Medicare has not chosen to include patient comments and ratings, despite recent calls for their introduction.⁵

Although some prior research has shown an association between results on formal patient satisfaction surveys and clinical measures of quality,⁶ to our knowledge, no previous study has examined the relationship between unsolicited ratings by patients and objective measures of quality. The degree to which such ratings are associated with objective clinical outcomes and conventional patient surveys may well determine their place in the landscape of transparency measures. In addition, decisions by policy makers and payers to support such Web sites might well hinge on whether such associations exist. This study examines hospital-level associations between Web-based patient ratings on the NHS Choices Web site and objective measures of quality.

We performed a cross-sectional observational study of all (n = 166) NHS acute hospital trusts in England, using data from 10 274 patient Web-based ratings posted on the NHS Choices Web site from January 1, 2009, to December 31, 2010, obtained from the English Department of Health. Hospital trust–level indicators of clinical outcomes (5 measures) and health care-acquired infections (2 measures) were compared with Web-based patient ratings using the Spearman rank correlation coefficient. When comparing patient ratings with infection rates, we used all 166 acute care hospital trusts in England. When comparing ratings with other clinical outcomes, a reduced list of 146 hospital trusts was used that excluded trusts that only care for children or specific specialties (eg, oncology). Data on clinical outcomes used were for financial year 2009/2010, and obtained from Dr Foster (a health intelligence company) and NHS Comparators (the NHS's comparative health system performance service). We obtained data on methicillin-resistant Staphylococcus aureus (MRSA) bacteremia and Clostridium difficile infection rates from the Health Protection Agency compulsory national surveillance system.

The mean and median number of ratings per hospital was 62 and 46, respectively (range, 1-290). Of those who offered a view about recommending the hospital (9349 ratings), 68.0% (n = 6354) would recommend the hospital to a friend. Positive recommendations of hospitals were significantly associated with lower hospital standardized mortality ratios (Spearman ρ = −0.20; P = .01), lower mortality from high-risk conditions (ρ = −0.23; P = .01), and lower readmission rates (ρ = −0.31; P < .001) (Table). Positive recommendations were not associated with mortality rates among surgical inpatients with serious treatable complications (ρ = 0.00; P = .99) or mortality from low-risk conditions (ρ = 0.03; P = .70). Mean and median ratings of hospital cleanliness were 3.6 (range, 2.6-5.0) on a scale from 1 (dirty) to 5 (exceptionally clean). Better ratings of hospital cleanliness were associated with lower MRSA (ρ = −0.30, P < .001) and C difficile (ρ = −0.16; P = .04) infection rates.

Our results demonstrate a relationship between patients' Web site ratings of hospitals and some objective measures of clinical quality, including mortality and infection rates.

Arguments in favor of the use of patient Web site reporting include that physicians can be poor judges of their patients' experience, that feedback changes physicians' performance, and people will inevitably use the Internet to voice opinions, so why not capture this information in a useful form.³ Arguments against using this data include the selection bias by those leaving reviews, the lack of meaningful data on technical quality of health care, and straining of physician-patient relationships.^2,5 Although our results do not counter all of these arguments against, they suggest that discretionary patient ratings, obtained through a Web site, may be a more useful tool than previously considered for both patients and health care workers. If patients are making choices based on this information, they can be reassured that the ratings are not entirely misleading and may be providing relevant information about health care quality. In his book The Wisdom of Crowds, James Surowiecki⁷ argues that a diverse collection of “independently deciding individuals” is likely to make better predictions and decisions than single individuals or even experts. At least to an extent, the self-selecting crowd of patients appears to be wise.

The use of Web-based patient ratings has become common in other industries such as hotels and restaurants, and consumers value these rankings in making choices. We believe that the information provided by these Web sites, although flawed, represents a potentially important development in the measurement of health care quality.