Online physician reviews: The good, the bad, and the ugly

Editor’s note: The subject of this article, which outlines the pros and cons of using patient rankings to influence quality assessments and reimbursement of surgeons, will be the subject of this year’s Resident and Associate Society of the American College of Surgeons (RAS-ACS) Symposium held on Sunday, October 6, during the College’s 2013 Clinical Congress in Washington, DC. This article was sponsored by the Issues Committee of the RAS-ACS.

In an era of increasing access to electronic information, online reviews have become a modernized viva voce. Websites such as Yelp, Angie’s List, and TripAdvisor offer information and reviews on nearly every aspect of life, including finding the perfect Italian restaurant, locating au pair services, booking a vacation, and perhaps of greatest concern to surgeons and residents, selecting a reliable health care provider. Physician review websites (PRWs) permit patients and third-party reviewers to grade both physicians and hospitals in popular online forums. Examples of PRWs include Healthgrades.com, Vitals.com, RateMDs.com, and a host of smaller, less-organized websites.

Although traffic on physician and health care-centered websites has yet to equal that of the hospitality and service industry, recent data suggest that the popularity of PRWs is rising steadily. In 2011, a survey of health care consumers found that 28 percent of respondents had searched for information about physician quality online, up from 24 percent in 2010 (n=4,000).¹ Not unexpectedly, this percentage was higher (34 percent) for younger patients identified as Generation Y or as the Millennial Generation.¹ The growing demand for PRWs is illustrated by the sharp increase in the number of physicians rated. Even more impressively, the number of reviews on RateMDs.com, one of the first PRWs, has grown from 2,475 reviews in 2005 to 112,024 in 2010.²

Although substantial evidence regarding the widespread use of PRWs is available, it is unclear how this consumer-derived information is obtained and what influence the data have on patient decision making. Over the past decade, PRWs have been heavily criticized, with many industry experts questioning the sometimes cryptic or confidential methodology used to develop consumer-driven physician and hospital evaluations.

The role of PRWs for physicians is similarly ill-defined. Could positive reviews increase a physician’s patient base? Could negative reviews seriously damage a surgeon’s reputation? Will patient reviews affect physician reimbursements? In a recent proposal, it was suggested that New York City’s public hospital ratings and other measures of their care should be reflected in the health care professionals’ earnings. Given the increasing visibility of PRWs, surgeons need to have a clearer understanding of why patients are turning to online review tools and how they are using them to make health care decisions.

Who is being rated?

Unfortunately, most information on PRW sites is derived from other databases (for example, Healthgrades.com, Vitals.com, Doctor.com), and may be inaccurate or outdated. For example, one of the authors of this article, a fourth-year urology resident, is listed on most PRWs as a general surgeon who could be contacted for bariatric procedures. Other studies, including one evaluating 250 physicians and another surveying 500 physicians, indicate that up to 80 percent of physicians are reviewed on at least one PRW.^3,4 There is some variation among the specialties, with obstetrician/gynecologists rated most frequently, while physicians with less direct patient contact (such as radiologists, pathologists, and anesthesiologists) are rated least frequently.²

What is being rated?

Most PRWs provide users with basic information about the physician, such as specialty, years in practice, education, board certification, and liability claims. Normally, this information is available for free or as part of a “freemium” model, where users have unlimited access to basic information but must purchase a subscription for more in-depth material. PRWs typically allow users to enter numerical ratings for various dimensions of care, and some allow free narrative-form reviews. Dimensions of care vary among PRWs. One study identified 35 different dimensions evaluated by 10 popular PRWs and then organized those areas into five major categories:⁵

• Overall rating
• Communication skills (explanation of medical care/treatment, follow-up, attentiveness, listening skills, and bedside manner)
• Access (availability of appointments, ease of scheduling, punctuality)
• Facilities (office cleanliness, lab services, waiting room accommodations)
• Staff (courtesy, friendliness, professionalism)

Although these categories capture some relevant information, they are lacking with respect to clinical relevancy. In a survey of 660 healthy patient volunteers, de Groot and colleagues determined physician expertise, wait time for outpatient appointment, and wait time for surgery to be the most important factors cited by patients.⁶ A study of patients in the United Kingdom demonstrated a poor correlation between perceived and actual quality of care measured with objective parameters, such as hypertension control and appropriate vaccinations.⁷ In a separate study, perceived quality of care again did not correlate well with objective quality markers.⁸

Popular PRWs

Exact and validated information regarding PRWs is limited, and readers are encouraged to visit these websites directly to learn more about the specific content they offer. Following are capsule summaries of three of the most popular ratings websites:

Healthgrades.com. This website is a for-profit site that allows patients to anonymously complete surveys about physicians after registering with a valid e-mail address or phone number. Patients are unable to provide written reviews on this website. The company encourages physicians to register and establish a profile. The website also publishes national hospital rankings based on Medicare data and acts as a “consulting firm” for some hospital systems, working with hospital systems to interpret and alter their rankings. There has been some debate as to whether rankings by Healthgrades.com truly captures all aspects of the quality of medical care, including outcomes, whether relevant screens/tests were ordered, and accuracy of diagnoses rendered.⁹

Vitals.com. This online review website provides free information on physician’s educational backgrounds and liability claims using state-based public and private insurance data sources. Patients can anonymously rate physicians and post comments. The website encourages physicians to create a profile and to participate in the review process. The emphasis of Vitals.com is largely on patient reviews of individual physicians rather than hospital rankings and consulting.

RateMDs.com. This Web-based tool allows anyone to anonymously post comments about a physician. There is minimal oversight, and postings and categories can be rather aggressive (for example, a physician “Wall of Shame”). Physicians may register and respond to comments.

An increasing number of websites blend reviews along with the company’s marketing and business strategies. These multifaceted sites include www.vimo.com, an online insurance comparison shopping website, which publishes anonymous physician comments and ratings and provides health insurance quotes, or www.zocdoc.com, which offers patient reviews and the option to book appointments with participating physicians.

The good

It is encouraging to those of us in the medical professions to see that physicians are often reviewed favorably. In a 2011 study of 4,999 individual online ratings, the average rating was 77/100 on a 100-point scale, 3.8 on a 5-point scale, and 3.1 on a 4-point scale.⁵ Using a standardized scale, 500 physicians examined across 10 PRWs were shown to have 86 percent positive ratings.⁴ Another study found that 50 percent of physicians reviewed received a perfect 5/5 rating, and that only 12 percent were below 2 percent.² Patient narratives typically have been positive.^4,10,11

Another benefit of online ratings is that they may be a good measure of patient satisfaction. In a British study published in 2012, the authors found a correlation between unsolicited Web-based patient ratings and conventional paper surveys, particularly with respect to questions related to “patient experience,” and concluded that Web-based patient ratings may be a useful tool for health care providers.¹² A German study compared 13 dimensions of health care with a systematic review of scientific articles that used standardized instruments to measure patient experience and satisfaction.¹³ This study revealed that no PRW captured all 13 dimensions but that the top English-language sites captured five to six dimensions.¹³

Finally, although the data on the correlation between physician rankings and quality of care are limited, some evidence indicates that positive patient-generated reviews of hospitals may be associated with better clinical quality. Using data from the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey, Jha and colleagues found that hospitals with a high-level of patient satisfaction provided clinical care that was higher in quality. Hospitals in the top quartile of HCAHPS ratings provided better care for acute myocardial infarction and for pneumonia than hospitals in the bottom quartile.¹⁴ Again, it is unclear whether a similar association exists for individual physicians and PRWs.

The bad

Although PRWs have some favorable characteristics, they also have their deficiencies. In particular, many patients may browse physician ratings and reviews, but very few actually take the time to write one. A Pew Research Center study showed that 37 percent of Internet users have rated products and services online, but only 3 to 4 percent have rated a physician or hospital.¹⁵ This low rate of participation results in few ratings/reviews per physician, which means these reports are less representative of the physician spectrum and more prone to volatility.^4,5 On RateMDs.com, the average number of ratings per physician is 3.2, and approximately 50 percent of all physicians had only one rating.^2,3 The implications of a single unfavorable rating on overall score can decrease the average score and make an otherwise high-performing physician appear mediocre.

Citing patient privacy concerns, PRWs do not verify the authenticity of the reviews; a valid e-mail address is often the only requirement for rating a physician. In their assessment of 33 PRWs, Lagu and colleagues found that 61 percent of these sites required patients to register with a valid e-mail address in order to leave a review, one site required credit card information, and one site even offered a $250 gift card certificate for reviewing multiple physicians. Without the ability to confirm whether ratings/reviews are actually issued by valid patients, the opportunity for abuse or fraud is clear, quite often at the hands of patients harboring a grudge or competing practices. On the other hand, office staff or specific physicians may write positive reviews to boost their online reputation.

Another concern centers on the accuracy of online reviews. Some physicians are inaccurately classified, while others may have moved their practices, and may not have their new information updated on PRWs. Physicians also are concerned that factors beyond their control (a grumpy receptionist or a parking problem, for example) may affect a physician’s composite score.

Response rates and ratings appear to be influenced by the patient population served. For example, there is evidence that physicians caring for patients from lower socioeconomic backgrounds may receive lower rankings.^16,17 These issues create a general concern that physicians may be inappropriately rewarded or punished for serving specific patient groups, leading to a deterioration of care for vulnerable patient populations.

The ugly

The anonymity of online reviews (and their potential for abuse) has resulted in several physician-filed lawsuits against PRWs. A Minnesota neurologist sued one of his patients for $50,000 after the patient’s son posted allegedly false remarks on a PRW.¹⁸ The neurologist then faced a strong online backlash with more than 30 (fabricated) negative reviews posted within two days. The case went before the Minnesota Supreme Court, which determined that the physician could not sue because the patient’s remarks were protected under the right to freedom of speech. Another case involved a plastic surgeon suing for $49,000 after a patient criticized the surgeon’s performance on several PRWs.¹⁹ In a recent New York Times interview, the founder of RateMDs.com stated that negative physician reviews have resulted in an average of at least one legal threat each week.²⁰ According to Eric Goldman, a law professor at Santa Clara (CA) University, physicians suing for online defamation rarely succeed.²¹

The future

Most physicians, patients, and policymakers would probably agree that transparency leads to improved quality of care. However, are PRWs the best means for achieving greater accountability? The Centers for Medicare & Medicaid Services (CMS) and the American College of Surgeons (ACS) have promoted reporting of quality measures through initiatives such as value-based purchasing and the National Surgical Quality Improvement Program (ACS NSQIP^®). Concerns raised about the validity of such measures are even more critical when an individual physician’s reputation is at risk.

There is an active move toward physician quality reporting. The Physician Quality Reporting System (PQRS), administered by CMS, collects data on 144 individual quality measures. Currently, CMS provides payment incentives to physicians who participate in the system in 2015, and it will penalize those who do not comply. CMS has started adopting patient satisfaction scores from its popular HCAHPS initiative, and hospitals with low scores are penalized.²²

As mentioned previously, whether patient experience translates to physician quality is unclear. Using volume as a proxy for quality, Segal and colleagues sampled 600 surgeons and compared numerical ratings between high-volume and low-volume surgeons who perform lumbar operations, total knee replacement, or bariatric procedures. Researchers found no statistical difference in the scores. With respect to narrative or text comments, they found that high-volume surgeons had more “glowing comments” to the ratio of total comments.²³ Greaves and colleagues evaluated 16,952 ratings on primary care physicians and found a weak association between clinical process and outcome measures.²⁴ Some studies have shown a correlation between the HCAHPS and clinical quality.²⁵ Additionally, hospitals with higher HCAHPS scores had few readmissions and lower inpatient mortality rates.²⁶ Whether these results, which focus on the patient’s hospital experience, can be generalized to individual physician patient satisfaction scores is yet to be determined.

Conclusion

The public’s increasing demand for online health care provider information, the emerging role of PRWs, and the influence of patient ratings and rankings on physician practice and reimbursement can no longer be ignored. Whether current PRWs are valid tools to assess physicians’ quality of care, or a platform for these online entities and their marketing initiatives, is debatable. However, it is clear that patients, insurers, for-profit organizations, and the government will continue to rank and rate surgeons. The consequences of these ratings will be increasingly profound, and it is imperative that surgeons become more proactive in understanding and participating in these new online resources.

---

Disclaimer
The views expressed in this article include those of Dr. Ward and do not necessarily reflect the official policy or position of the U.S. Department of the Navy, U.S. Department of Defense, or the U.S. government.

---

References

1. Keckley P, Coughlin S, Eselius L. 2011 Survey of Health Care Consumers in the United States: Key Findings, Strategic Implications. Washington, DC: Deloitte Center for Health Solutions; 2011.
2. Gao GG, McCullough JS, Agarwal R, Jha AK. A changing landscape of physician quality reporting: Analysis of patients’ online ratings of their physicians over a 5-year period. J Med Internet Res. 2012;14(1):e38.
3. Mostaghimi A, Crotty BH, Landon BE. The availability and nature of physician information on the internet. J Gen Intern Med. 2010; 25(11):1152-1156.
4. Ellimoottil C, Hart A, Greco K, Quek ML, Farooq A. Online reviews of 500 urologists. J Urol. 2013;189(6):2269-2273.
5. Kadry B, Chu LF, Kadry B, Gammas D, Macario A. Analysis of 4999 online physician ratings indicates that most patients give physicians a favorable rating. J Med Internet Res. 2011;13(4):e95.
6. de Groot IB, Otten W, Dijs-Elsinga J, Smeets HJ, Kievit J, Marang-van de Mheen PJ. Choosing between hospitals: The influence of the experiences of other patients. Med Decis Making. 2012;32(6):764-778.
7. Rao M, Clarke A, Sanderson C, Hammersley R. Patients’ own assessments of quality of primary care compared with objective records based measures of technical quality of care: Cross sectional study. BMJ. 2006;333(7557):19.
8. Chang JT, Hays RD, Shekelle PG, MacLean CH, Solomon DH, Reuben DB, Roth CP, Kamberg CJ, Adams J, Young RT, Wenger NS. Patients’ global ratings of their health care are not associated with the technical quality of their care. Ann Intern Med. 2006;144(9):665-672.
9. Osborne NH, Ghaferi AA, Nicholas LH, Dimick JB, Mph M. Evaluating popular media and internet-based hospital quality ratings for cancer surgery. Arch Surg. 2011;146(5):600-604.
10. Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients’ evaluations of health care providers in the era of social networking: An analysis of physician-rating websites. J Gen Intern Med. 2010;25(9):942-946.
11. Lopez A, Detz A, Ratanawongsa N, Sarkar U. What patients say about their doctors online: A qualitative content analysis. J Gen Intern Med. 2012;27(6):685-692.
12. Greaves F, Pape UJ, King D, Darzi A, Majeed A, Wachter RM, Millett C. Associations between Internet-based patient ratings and conventional surveys of patient experience in the English NHS: An observational study. BMJ Qual Saf. 2012;21(7):600-605.
13. Reimann S, Strech D. The representation of patient experience and satisfaction in physician rating sites. A criteria-based analysis of English- and German-language sites. BMC Health Serv Res. 2010;10:332.
14. Jha AK, Orav EJ, Zheng J, Epstein AM. Patients’ perception of hospital care in the United States. N Engl J Med. 2008;359(18):1921-1931.
15. Pew Research Center. Health online 2013. Available at: www.pewinternet.org/Reports/2013/Health-online.aspx. Accessed February 7, 2013.
16. Hong CS, Atlas SJ, Chang Y, Subramanian SV, Ashburner JM, Barry MJ, Grant RW. Relationship between patient panel characteristics and primary care physician clinical performance rankings. JAMA. 2010;304(10):1107-1113.
17. Higashi T, Wenger NS, Adams JL, Fung C, Roland M, McGlynn EA, Reeves D, Asch SM, Kerr EA, Shekelle PG. Relationship between number of medical conditions and quality of care. N Engl J Med. 2007;356(24):2496-2504.
18. Simons A. High court rules online posts didn’t defame doctor. January 30, 2013. Star Tribune. Available at: www.startribune.com/local/189028521.html. Accessed July 23, 2013.
19. Pacheco W. Doctor sues patient over Web comments. May 21, 2012. Orlando Sentinel. Available at: http://articles.orlandosentinel.com/2012-05-21/health/os-doctor-online-defamation-lawsuit-20120521_1_patient-comments-defamation-lawsuit. Accessed July 23, 2013.
20. Lieber R. The Web is awash in reviews, but not for doctors. March 9, 2012. The New York Times. Available at: www.nytimes.com/2012/03/10/your-money/why-the-web-lacks-authoritative-reviews-of-doctors.html?pagewanted=all&_r=0. Accessed July 3, 2013.
21. Jones A. Online reviews: When do they go from helpful to defamatory? Wall St J. Available at: http://blogs.wsj.com/law/2012/10/25/online-reviews-when-do-they-go-from-helpful-to-defamatory/. Accessed July 23, 2013.
22. Scalise D. Who’s rating you? Hospitals and Health Networks. 2001;75(12):36-40.
23. Segal J, Sacopulos M, Sheets V, Thurston I, Brooks K, Puccia R. Online doctor reviews: Do they track surgeon volume, a proxy for quality of care? J Med Internet Res. 2012;14(2):e50.
24. Greaves F, Pape UJ, Lee H, Smith DM, Darzi A, Majeed A, Millett C. Patients’ ratings of family physician practices on the internet: Usage and associations with conventional measures of quality in the English National Health Service. J Med Internet Res. 2012;14(5):e146.
25. Manary MP, Boulding W, Staelin R, Glickman SW. The patient experience and health outcomes. N Engl J Med. 2013;368(3):201-203.
26. Glickman SW, Boulding W, Manary M, Staelin R, Roe MT, Wolosin RJ, Ohman EM, Peterson ED, Schulman KA. Patient satisfaction and its relationship with clinical quality and inpatient mortality in acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2010;3(2):188-195.