Rankings and Reactions: CMS Launches Star Ratings for Hospitals
posted by David W. Johnson on September 8, 2016 - 9:59am
“The fault, dear Brutus, is not in our stars but in ourselves.” [1] ~William Shakespeare
On July 27th, the Centers for Medicare and Medicaid Services (CMS) released its first-ever quality ratings for hospitals. Within the week, U.S. News and World Report released its much-followed 27th annual hospital rankings. The results could not have differed more.
Of the 20 U.S. News 20 “Honor Roll” hospitals, only the Mayo Clinic received 5 stars. Four of their "Honor Roll" hospitals received only 2 stars. These contradictory results reflect differing rating methodologies and illustrate the consumer challenge of finding actionable quality data for making healthcare decisions.
Healthcare providers have long fought against external performance assessments. It’s a losing battle. Amid much teeth-gnashing, hospitals are adapting to market demands for transparency. Expect the CMS hospital ratings to gain currency and shape future demand for hospital services.
Giving Consumers the Hospital Ratings They Want
Desperate for guidance, consumers increasingly rely on third-party provider ratings and report cards to select hospitals and doctors. “About three-quarters of Americans with Internet access have searched for health or medical information online.”[2]
Despite strong public support for more accessible quality data, the hospital industry fought the release of CMS’ new “consumer-friendly” star ratings. In response to industry concerns, CMS delayed their release by three months and conducted significant stakeholder outreach. This did little to dampen industry ire.
Since their release, industry representatives have lampooned the CMS ratings, noting the poor performance of teaching and safety net hospitals. In a blog accompanying the new ratings, CMS Quality Director Kate Goodrich acknowledged industry concerns while heralding the benefits of transparency to American healthcare consumers,
"[CMS’ Hospital Compare website will] help millions of patients and their families learn about the quality of hospitals, compare facilities in their area side-by-side, and ask important questions about care quality when visiting a hospital or other health care provider."
Given the wide variation in hospital quality, outcomes and costs, external report cards can help consumers make better choices. Highlighting this point, Dr. Goodrich notes, “researchers found that hospitals with more stars on the Hospital Compare website have tended to have lower death and readmission rates.”
Likewise, a National Bureau of Economic Research study found that hospitals with higher quality ratings for specific medical conditions, including heart attacks, pneumonia and congestive heart failure, disproportionately gain market share. Market forces work. When given choices, consumers select higher-performing hospitals.
Public demand for better hospital ratings is accelerating. Multiple independent sources have emerged to meet this need; however, these third-party hospital ratings generate as much confusion as clarity.
The Confusing World of Hospital Ratings
"No hospital was rated as a high performer by all four national rating systems. Only 10 percent of the 844 hospitals rated as a high performer by one rating system were rated as a high performer by any of the other rating systems."
They attributed the lack of overlap to different rating methodologies, different areas of focus and different criteria weighting. Their unsurprising conclusions are few hospitals excel across all service dimensions and a lack of performance standardization retards hospital performance improvement.
The Affordable Care Act has addressed the lack of hospital quality, cost and performance transparency through more intense data collection (currently 113 metrics) and distribution. Issuing star ratings based on newly collected data is a logical next step.
CMS’ star ratings for individual hospitals emerge from a statistical analysis of 64 metrics across the following 7 categories: mortality; safety; readmissions; patient experience; effectiveness; timeliness; and imaging. The first 4 categories carry 22% weights and the last 3 carry 4% weights. CMS will update and publish its quality ratings quarterly. Individual hospital performance summaries, like this report for the Mayo Clinic, are clear, detailed and easily comparable.
While controversial, CMS’ star ratings are entirely data-driven. CMS took over two years to develop its rating methodology. It leaned heavily on a prestigious technical expert panel and invited widespread public input. Their analytics are statistically rigorous in selecting and assigning weights to metrics and performance categories. Their methodology employed clustering techniques to establish a bell-curve distribution among the almost 4,000 rated hospitals.
By contrast, U.S. News targets specialty care and relies heavily (27.5% weighting) on reputational surveys by board-certified specialists. U.S. News also weights 30-day Medicare survival rates (37.5%); patient safety (5%) and other care-related factors such as nurse-staffing ratios (30%). All 20 of U.S. News’ “Honor Roll” hospitals are academic medical centers. The U.S. News' methodology does not incorporate patient experience.
The following chart lists the U.S. News “Honor Roll” hospitals in rank order with their current CMS star ratings:
The disparity of star ratings among these “Honor Roll” hospitals is striking. While CMS’s rating methodology may not be perfect, it is data-driven and consistent. The inability of the nation’s most-recognized hospitals to all excel on CMS’ quality metrics testifies to unacceptable performance variation throughout American hospitals in outcomes, quality, costs and customer experience.
Industry Reaction
Blustering is a professional sport in Washington. Too many healthcare leaders scorn the CMS’ methodology because they don’t like its results.
Dr. Darrell Kirch is the president and CEO of the Association of American Medical Colleges. In a Modern Healthcare commentary titled “CMS’ New Star Ratings are Unfair to Teaching and Safety-Net hospitals,” Dr. Kirch castigates CMS’ ratings in hyper-charged language,
"Instead of providing useful information, the new ratings paint a confusing and conflicting picture of the quality of U.S. hospital care because of a deeply flawed methodology that ignores important differences in the patient populations and the complexity of conditions that different types of hospitals treat."
These types of broad-based critiques are disingenuous. It would be equally fair to state that U.S. News’ intense focus on specialty care and dependence on specialist opinion surveys biases their rankings against well-run community hospitals. What else explains that all 20 “Honor Roll” hospitals are academic medical centers?
It’s worth noting that highly-rated U.S. News hospitals spend gargantuan sums advertising their success. The same will be true for CMS’ 5-star hospitals. During my recent Lake Michigan beach vacation, I saw several billboards where 5-Star Holland Community Hospital proclaimed itself the best hospital in Michigan.
Back to Dr. Kirch’s commentary. In fairness, he makes the following valid points regarding CMS’ methodology:
- that reporting more metrics lowers some hospitals’ scores;
- that socio-economic status likely influences care outcomes; and
- that current ratings may not accurately reflect the benefits of specialization.
CMS can and will adjust its methodology to address these concerns.
To drive his “unfairness” point home, Dr. Kirch notes that “only one major teaching hospital received five stars [and] nearly 90% rated three stars or below.” He then asks, “How is this possible?”
Ironically, that is exactly the question lower-rated hospitals should ask themselves. Despite the challenges Dr. Kirch outlined, Mayo Clinic, Cleveland Clinic, Mass General, New York Presbyterian and four other “Honor Roll” hospitals achieved 4 and 5-star ratings. Rather than complain about the rating process, lower-rated hospitals should investigate what these institutions are doing right and replicate their superior performance.
The beauty and wonder of market-driven reform is that companies respond to “signals” from customers to win their business. Enlightened hospitals engage in constant and consistent performance improvement.
“Yelping” It: Ratings Speak to Customers
In another intriguing Health Affairs study, researchers employed natural language processing to evaluate Yelp hospital ratings relative to CMS' Hospital Consumer Assessment of Healthcare Providers and Systems quality scores (HCAHPS).
They found that hospitals receiving at least 3 Yelp ratings had almost identical ratings to CMS’ HCAHPS scores. They also found Yelp’s hospital reviews were more expansive, covering 12 domains (i.e. staff compassion, billing experience, nursing quality, amenities, etc.) beyond the 32 domains covered by HCAHPS.
Most intriguing, these researchers concluded that Yelp “may provide a more nuanced view [than HCAPHS] of aspects of hospital quality that patients value.” Their conclusions illustrate why hospitals must listen to, understand, and respond to customer needs. Those that do will prosper in post-reform healthcare.
Like rock ‘n roll, hospital ratings are here to stay. CMS' star ratings for nursing homes, Medicare Advantage insurance plans and dialysis centers are in widespread use. Consumers find these ratings helpful. They influence their healthcare purchasing decisions. CMS plans to develop star ratings for physicians. It’s adapt or die time for providers.
Hospitals that acclimate to healthcare consumerism and improve performance, particularly in patient engagement, will win consumer trust and remain viable market participants. Paraphrasing Shakespeare, “the stars” will reward hospitals that recognize “the faults in themselves” and work to correct them. Outcomes matter. Customers count. Value Rules!
[1] William Shakespeare[1] (Julius Caesar, Act I, Scene ii, lines 140-41)
[2] Steve Findlay, Health Affairs, April 2016, "Consumers' Interest In Provider Ratings Grows, And Improved Report Cards And Other Steps Could Accelerate Their Use", page 2
