The most commonly heard comment in healthcare these days is that we have to move from paying for volume to paying for value.  While it may sound trite, it also turns out to be pretty true.  Right now, most healthcare services are paid for on a fee-for-service basis — with little regard for the quality of that service.  We clearly need to move towards value-based payments (sometimes referred to as pay-for-performance or P4P).

Although a few folks remain skeptical about whether VBP/P4P can work (as though our pay for volume strategy is working out so well), asking whether we should pay for volume versus pay for quality no longer seems like a particularly interesting question.  The far more compelling and difficult question is how best to pay-for-performance? As I have written before, we need bold experiments with new payment models that employ three key principles: putting real money on the table, focusing on outcomes, and keeping the reward system simple (i.e. the better you do, the more you should get).

One such new payment model is the value-based purchasing (VBP) program from CMS, the largest payer of hospital care in America.  It’s a modest program but an immensely important one.  It is modeled after the Premier Hospital Quality Incentives Demonstration (HQID), which ran for 6 years and had modest effects on hospital performance on process measures and no effect on patient outcomes.  Despite these disappointing findings, the U.S. Congress, in crafting the Affordable Care Act, modeled VBP closely on HQID.  The incentives in the program are small (currently at 1.25% of total Medicare payments) and still more heavily weighted towards process measures than outcomes.

The key question for VBP is whether it will work — whether patients will be better off because of it.  We don’t know and realistically, we won’t for another year or so.  But what we do know is that 2 years into the program, certain hospitals seem to be doing well and others, not so much.  Yes, the incentives are small and my guess is that any impact will be very modest as well.  But, it’s still worth taking a look at how different types of hospitals are faring under VBP.  So we ran some numbers.

What we did

We took the latest release of the CMS data on how much each hospital is penalized.  We linked these data to the American Hospital Association annual survey, which provides us with a series of hospital characteristics and the Medicare Impact File, which tells us a hospital’s Disproportionate Share Hospital (DSH) Index.  The DSH Index is widely used to measure the percentage of patients who are poor and high DSH Index hospitals are often referred to as safety-net institutions.  We then examined “bivariate” relationships — like whether size of the hospital, ownership, or DSH Index was associated with a higher or lower VBP penalty.  Finally, we built a multivariable model to see if, holding other characteristics constant, certain types of hospitals seem to do better than others.

What we found

Interestingly, we found that some things don’t matter that much — hospital size and teaching status just don’t have much of an effect.  Small hospitals did about as well as large ones and major teaching hospitals showed up in each of the performance quartiles about equally often.  There seem to be moderate regional differences:  hospitals in the Northeast and West do not do as well as Midwestern hospitals.

What’s interesting, and possibly most challenging, is that public hospitals and safety-net hospitals — those in the highest quartile of DSH index, tend to do worse.  They are losing money on VBP (see first column of results in table).  Because many of these hospital characteristics are overlapping (major teaching hospitals are usually large, public hospitals often have high DSH index, etc), the table also displays results from a multivariable model.

The story is similar: hospitals in the Northeast and West get bigger penalties, as do public hospitals and those in the highest quartile of DSH.  Its additive.  A large urban public hospital in the Northeast with a high DSH index gets an average penalty of about 0.30% of their Medicare payments.  Is that a lot?  No — but it’s not irrelevant for a safety-net hospital that may be operating with razor thin financial margins.

Table: Average total penalty, unadjusted and adjusted for VBP 2014

* Adjusted for all other variables in the table.  For example, public hospitals will get an additional 0.10% penalty, holding size, teaching, urban, region, DSH Index and proportion of Medicare patients constant.

So how do we interpret this?  Is the VBP program disproportionately penalizing safety-net hospitals?  Yes.  Is it unfair?  We don’t know.  We don’t know why public, safety-net hospitals do worse on VBP.  My suspicion is that much of the difference is driven by differences in patient experience scores.

The challenge for all of us is to understand why safety-net hospitals generally have worse patient experience scores.  Is it that poorer or minority patients are just less likely to give high scores on patient experience? Or are safety-net hospitals not doing as good of a job on patient-centered care?  Until we know, we must be careful declaring that this is an unfair playing field.  This is in contrast to the medical readmissions measure, where we know that so much of what drives a readmission is about what happens after the patient leaves the hospital (resources at home, access to effective primary care, etc).  If you have more poor patients, your readmission rate will likely be higher.  In the readmissions program, creating a more even playing field, as MedPAC has suggested, is a good idea.  For VBP, the jury is still out.

So — VBP is heading into year 2 — and we can see that some hospitals are doing well while others are struggling.  What we don’t know is if this program is fundamentally changing the way hospitals are working on quality.  Over the past few years, I’ve spoken to a lot of hospital leaders.  For most, the VBP measures remain a “checkbox” item — something they invest just enough time and energy to hit their marks, but not much more.  As more outcomes measures are included in VBP, I hope that mentality changes and VBP becomes part of a broader agenda to make quality and value central to how we pay for healthcare.

Ashish Jha is an associate professor of health policy and management, Harvard School of Public Health.  He blogs at An Ounce of Evidence and can be found on Twitter @ashishkjha.