More Evidence that public SNF Quality Measures aren’t very useful.November 17th, 2014 by
What is the difference between a cynic and a realist? How about this definition – the cynic’s opinion about a subject is based on beliefs, the realist tries to use verifiable facts.
Longtime readers of this blog will see it fluctuates between those two points of view. Today’s blog about the CMS Nursing Home 5 Star Quality rating system is written by a realist.
JAMA, the AMA’s flagship journal published an article October 15th, 2014 titled Association Between Skilled Nursing Facility Quality Indicators and Hospital Readmissions; Mark D. Neuman, MD, MSc; Christopher Wirtalla, BA; and Rachel M. Werner, MD, PhD, were the authors. Dr Neuman and colleagues are prolific researchers with numerous peer reviewed studies in publication. Here is the objective, and conclusions, of the study:
|Importance Hospital readmissions are common, costly, and potentially preventable. Little is known about the association between available skilled nursing facility (SNF) performance measures and the risk of hospital readmission. Objective To measure the association between SNF performance measures and hospital readmissions among Medicare beneficiaries receiving postacute care at SNFs in the United States. Conclusions and Relevance Among fee-for-service Medicare beneficiaries discharged to a SNF after an acute care hospitalization, available performance measures were not consistently associated with differences in the adjusted risk of readmission or death.|
The real importance of this study is the scope of the data set used for analysis – it began with a 100% sample of all SNF admissions for the period 9/2009-8/2010. This included:
- 100% of 2008-2010 Medicate Provider Analysis & Review files (inpatient services)
- 2009-2010 MDS files for all SNF episodes
- 2009-2010 Medicare Beneficiary Summary files (identifies Medicare Part C beneficiaries)
- 2009-2010 Nursing Home Compare data files
All of these data files contain individually identifiable records. This isn’t an inferential study, but one using actual episodes of care.
The researchers sorted and filtered the data to focus on Medicare Fee-for-Service beneficiaries (traditional Medicare) to control for any external care management services associated with Medicare Part C plans. Those patients were eliminated from the study’s cohort.
After these exclusions, there remained over 1.5Million SNF admissions during the study year – essentially 100% of the SNF admission population with traditional Medicare during all of 2009 and 2010.
Starting with that population, the researchers assessed the correlation of CMS Nursing Home quality measures (a/k/a 5Star rating system) with 30 day all cause rehospitalization + death rates. The analysis showed small, but statistically significant variations between rehospitalization/death rates and the familiar 5 Star ratings. Facilities with better ratings had lower rehospitalization rates.
Using their access to a comprehensive catalog of individual level data from CMS allow a subsequent, risk adjusted analysis. The adjustments included controlling for;
- the referring hospital’s variance from the norm in rehospitalization rates
- The facility’s payor mix
- Facility Staffing Ratios, and
- Patient characteristics (demographics, diagnoses/problems, etc.)
Once these adjustments were applied almost all of the earlier correlation between Quality ratings and rehospitalization rates had disappeared. Here is the table from their abstract:
After reading the article, I sent a copy to John Sheridan, CEO of eHealth Data Solutions, for his review and analysis.
For readers who don’t know John, his Cleveland based company manages MDS file submission and analysis for ~ 1,700 LTC facilities. Part of their service is tracking facility and patient data over time, and giving feedback reports to facility owners and operators about quality related activities for that facility.
After years of performing these analytical reviews of facility data, John is uniquely familiar with the MDS reports and the facility specific variables that drive much of the 5 Star system.
When we finally had a chance to discuss the paper, John’s analysis was quick and to the point (to paraphrase his observations) –
‘The study demonstrates the obvious. Facilities with high Medicaid populations have uniformly low reimbursement; low reimbursement leads to low staffing ratios which lead to higher rehospitalization rates. Medicaid payment levels set the staffing which states are willing to pay for. Those states with mandated staffing pay more for Medicaid residents. Low staffing levels may cause a cascade of quality related problems which generally lead to lower ratings in the CMS 5 Star quality rating system.’
This doesn’t mean that the 5 Star Quality guides are irrelevant – they aren’t. The point is that they aren’t measuring real differences in Quality – they accurately reflect differences in patient populations. This problem pervades all of CMS’s Quality Measurement Strategies – Namely, CMS Quality Measures affect those states where Medicaid payment is less than in those states where Medicaid payment is more. CMS in seeking non-financial adjustments missed a big opportunity to improve quality. You decide, did CMS pass or fail at accurate Risk Adjustment!
Nearly all of us working in the LTPAC arena agree that reducing avoidable rehospitalizations is a desirable objective; a high quality/high performance care system should deliver superior results. This study shows that the current 5 Star Quality Measures do not correlate with high quality/ high performance when accurately risk adjusted.
John Sheridan went on to make a particularly useful observation. The 5 Star rating scheme may not accurately reflect risk adjusted quality, but the CMS initiative to reduce 30 Day All Cause Rehospitalization Rates has delivered results. During the year of this Study (2009-10), the average 30 day rehospitalization rate was 23.3%. According to the American Healthcare Assn., the national 30 day risk adjusted rate was 15.7% in Q1 2014. This isn’t an apples-to-apples measure because AHCA measures at the time of Discharge, and CMS uses a retrospective analysis (capturing events after SNF discharge but within 30 days). Regardless, the AHCA data shows that rehospitalizations decreased more that 14% since they began tracking this metric in 2011.
The lesson? If you want real performance improvement – pick something simple and let other providers manage the program, not CMS. That’s what happened with the rehospitalization program – the penalties/rewards accrue to the Acute Care Hospitals. They have a vested interest in reducing their own rehospitalization rates – which means managing the flow of patients to facilities that will partner in this effort. The reality is that all Nursing Homes are trying to improve their own baseline performance to maintain good relations with their area Hospitals. This study tells those hospitals that the CMS 5 Star quality ratings are not the best choice as a tool to assess an individual Nursing Facility as a post-acute partner.