Predictions are Difficult When Talking about the Future

January 2016

Every year around this time the pundits and prognosticators make predictions about the events to come.  They’re often wrong (“the Cubs will go to the World Series in 2015”).  And that’s okay, because nobody takes them too seriously.  But that’s not true in today’s healthcare sector, where predictive modeling has become a key tool in the shift from volume to value.  Done correctly, predictive modeling allows payers and providers to better understand how care redesign can affect the outcomes of that care.  Unfortunately, the process of making predictive models to evaluate those outcomes has been challenging. To paraphrase the Nobel Laureate Niels Bohr with a quotation popularized by Yogi Berra, predictions are difficult—especially when talking about the future.  Still, we’ve had success in this area, so we’d like to share some tips for making your predictions more precise.

Part of the reason predictive modeling in healthcare is so hard is that a typical patient population has a highly variable number of chronic diseases and associated medical conditions.  These understandably pose different risks for major operations or medical hospitalization.  All would understand that a patient with fragile diabetes or severe chronic lung disease is a greater risk for an abdominal operation than would be the case for a patient without these co-existing issues. Other potential risk factors such as essential hypertension have been a source of concern but are commonly not statistically valid in association with suboptimal outcomes of care.

Another challenge is the definition of an adverse outcome.  The term “adverse outcome” encompasses death following operation; severe complication from a procedure; and in recent years, associated readmission to the hospital following inpatient care. Each of these issues are important in their own right, but they also cloud the process of predictive modeling.  How many days following an operation should be counted in mortality rates or readmissions? What complications of care are severe enough to be counted as an adverse outcome and what events are minor nuisances that do not materially affect the recovery of the patient or the cost of care?  The effect is that different risk factors are associated with the prediction of each of these components of the adverse outcome.

To begin to address challenges like these, we have included the following in our approach to get fair and accurate models:

  1. A Broad Set of Risk Factors – Over 500 candidate risk factors are used to develop predictive models for inpatient deaths; prolonged lengths-of-stay (a surrogate for severe inpatient complications); 90-day post-discharge deaths; and 90-day readmission rates.
  2. Models should work 90 days Post-Discharge – The 90-day post-discharge interval is chosen because this is the interval that is being used by Medicare in the development of its “Bundled Care” payment model. Accordingly, we developed risk-adjusted cost of care models to integrate with outcomes modeling to give providers a better understanding of how to achieve cost-effective care.
  3. Administrative Claims Data Can Work – A criticism of prediction models is that they are often derived only from administrative claims data. While it is true that administrative data have shortcomings, it is also true that it will be administrative data that are being used by Medicare and other payers for the development of bundled payment models going forward. We are experienced at using administrative data to develop discriminating and powerful prediction models to provide an accurate evaluation of your hospital’s results.

Of course there is a lot more to consider when building predictive models.  We would love to provide a detailed presentation to you on risk-adjustment and predictive models and how they can help you improve your outcomes.  The fact is predictions are difficult when talking about the future, but they aren’t impossible.  Your organization’s future may not be the same without them.

Donald Fry

Donald E. Fry, M.D. is Executive Vice President for Clinical Outcomes Management at MPA Healthcare Solutions, Adjunct Professor of Surgery at Northwestern University, and Professor Emeritus of Surgery at the University of New Mexico School of Medicine. At MPA Healthcare Solutions, Dr. Fry provides clinical leadership in analyzing and evaluating clinical performance, guiding quality improvement, and creating incentives for coordinated, cost-effective care.

Follow Us

Contact Us