Unplanned readmissions to the hospital have been the focus of much attention in recent years for obvious reasons. First, they are relatively easy to measure using administrative claims data. Second, like all inpatient hospitalizations, they cost a lot of money–and are therefore a target for reducing spending. Third, they are a proxy for quality of care, as at least some portion of them are likely avoidable if the hospital does its job well. On this last point, many disagree, citing the lack of continuity of care that exists post-discharge as a major source of readmissions. According to the folks in this camp, the patients themselves and their primary care physicians–not the hospital–are to blame for many of the unexpected returns to the hospital.

While this debate rages on, however, the federal government is taking action. Since 2009 they have published data on hospital quality using the Hospital Compare website, so that the public can be better informed. Then, starting last October, readmission rates for three conditions (heart attack, congestive heart failure, and pneumonia) were tracked, and hospitals with higher than expected rates were subjected to a reduction in Medicare reimbursement.

But a recent study from Matthew Press and colleagues in the June issue of Health Affairs finds that hospital readmission rates may not be such a good indicator of hospital quality after all. First, they found that across all hospitals, readmission rates for heart attack ranged from a low of 15.3% to a high of 25.6%. When they divided the hospitals up into quartiles, they found that only 1.7 percentage points separated the bottom 25% from the top 25%. Then, not surprisingly given the limited distance between the groups, they found that in just two years, many of those in the best performing group moved into the worst performing group and vice versa. Part of the explanation is what statisticians and econometricians call “regression to the mean.” In short, if you’re at the top of the pack, it is statistically more likely that you will move down than move up, just because you’ve got much more room to move in one direction than the other. The same is true in the reverse for the low performers. The investigators also found that, with few exceptions (e.g., teaching status), risk-standardized readmission rates were not correlated with other measures of hospital quality.

So what does this mean? Well, the authors suggest, there could be quite a few problems with policies that rely heavily on readmission rates alone as an indicator of hospital quality. Instead, they argue that other measures should be considered in addition to readmission rates when comparing hospital quality and that it is important to take regression to the mean into account by adjusting accordingly. In short, when it comes to measuring hospital quality, the more ways in which it is measured, the better.

Brad Wright is an assistant professor of health management and policy who blogs at Wright on Health.