Is it possible to improve clinical performance without measuring it?

Like it or not, measuring physician performance is now a key part of the conventional wisdom on improving our health care system. Borrowing from management guru Peter Drucker’s mantra “You can’t manage what you can’t measure” health care policy makers have embraced performance measurement as being central to managing our heretofore unmanageable health care system. But there is a small but seemingly growing group of Don Quixote-like dissenters who are tilting at the performance measurement windmill, arguing that these measures will not achieve the ends of improving quality and saving money and may instead do considerable harm.

Dr. Bob Centor, author of DB’s Medical Rants, is one of them. (Disclosure: Dr. Centor is chair-elect of ACP’s Board of Regents, although the views he expresses in his blog are his own, not ACP policy). One of his posts, titled “What has performance measurement wraught?” calls them madness:

Most readers know that I am obsessed with performance measurement and why it not only rarely works but often causes negative unintended consequences. As I have pondered this question recently, computers cannot replace physicians as diagnosticians. And the same misunderstanding of medicine that would advocate such a position drives the performance measure movement.

Physician decision making requires a complex weighing of disease severity, number of diseases, social situation, cost of medications, the patient’s desires and willingness to address issues and more that you can imagine. To think that we can apply simple rules to such decision making represents an unjustifiable conceit that patient care is simple and can therefore be broken down into RULES.

The unintended consequences of this movement are many. We now have nonsensical report cards and, here the author gasps, public reporting. If we could define excellence, then public reporting would make sense. But we cannot define excellence through rules that cover only selected diseases and only one aspect of doctoring.

How do we stop this madness?

In an earlier post, he cites a commentary in the Journal of the American Medical Association which suggests that poorly-designed performance measures can cause harm to patient care. “Too often we have measures based on a religious belief (e.g. lowering HgbA1c is always the proper goal) and not based upon good prospective data,” he writes. “Whenever we have to struggle to meet a performance goal, we run the risk of unexpected consequences. This irresponsible process likely harmed patients. Let me repeat that sentence. This irresponsible process likely harmed patients. The reasons now are clear. Some, including the authors of this commentary complained bitterly back in 2006. We allow organizations to establish performance measures without expecting the same rigorous testing that any other intervention must have prior to approval. We would not approve a new drug without careful testing for both efficacy and safety. Should we not hold performance measurement to the same standard?”

But is it possible to improve clinical performance without measuring it? The Institute for Healthcare Improvement, formerly headed by ex-CMS administrator Don Berwick, MD, says that “Measurement is a critical part of testing and implementing changes; measures tell a team whether the changes they are making actually lead to improvement.” In 2008, Dr. Berwick co-authored an article published in Health Affairs that presented the Institute’s now widely accepted Triple Aim of improving individual patient outcomes, improving population outcomes, and lower per capita costs. He writes that “in general, opacity of performance is not a major obstacle to the Triple Aim. Many tools are in hand to construct part of a balanced portfolio of measures to track the experience of a population on all three components. At the Institute for Healthcare Improvement (IHI), for example, we have developed and are using a balanced set of systemwide measures closely related to the Triple Aim. A more complete set of system metrics would include ways to track the experience of care in ambulatory settings, including patient engagement, continuity, and clinical preventive practices.”

Measurement for the purpose of helping groups of physicians assess how well they are doing in achieving the triple aim may be challenging enough, paying based on performance measures raises a whole host of other issues. All payment systems create a mix of potentially good and potentially bad results. Fee-for-service achieves the potentially good outcomes of creating incentives for physicians to actually see their patients and not undertreat them, because FFS pays them on how many patients they see and how many procedures they do, but it can also have the undesirable outcomes of “rushed” assembly line visits and over-testing and over-treatment. Capitation achieves the potentially good outcomes of encouraging physicians to be more efficient and not over-treat their patients, since they are paid the same amount per patient no matter how many procedures or visits provided, but it can also have the undesirable outcome of incentivizing physicians to not see patients enough, not treat sicker patients, or undertreat them. Payment systems linked to performance measures can have the desirable outcomes of creating incentives for physicians to organize their care to achieve better outcomes for their patients, better care of the patient population they see, and maybe, lower costs (the Triple Aim), but also the undesirable outcomes of “treating to the measure” (paying attention only to things being measured, and less to things not being measured), and creating disincentives for physicians to take care of sicker patients and those with lower socioeconomic status because such patients may adversely affect their performance “score.”

Performance measures though could help level out the potentially undesirable incentives existing in FFS or capitation: FFS tied to performance measures could help counter the incentives for over-treatment because physicians who over-treat with no improvement in outcomes wouldn’t score as well on measures of individual, population or per capita cost outcomes. Capitation tied to performance measures—if accompanied with appropriate risk adjustment– could help counter incentives for physicians to under-treat patients, since under-treatment would result in poorer “scores” on individual and population-based health outcomes and patient experience with the care provided.

My sense is that the performance measurement genie is out of the bottle and isn’t going away. We live in an era where just about everything and everybody is being measured and held accountable for getting better results as efficiently as possible. Health care is so damn expensive that the public (through government) and insurance company shareholders will want to know if physicians are achieving the best possible results and the lowest possible cost—how can they know what results they are getting without measuring it?

But as measurement becomes increasingly imbedded in our health care system, we should pay attention to potential unintended consequences. We should insist on meaningful measures that are based on the best available science through a transparent process, not measurement for the sake of measurement. We should test measures whenever possible before they are widely adopted, just as we do for new drugs, and withdraw measures that turn out to be harmful, just as the FDA withdraws newly approved drugs if they are found to have unforeseen harmful side effects. We need to be very careful as we design payment models that incorporate performance measure so that what is best for the patient, not what is best for the measure, always comes first. All of these, and more, safeguards are called for in ACP policy on performance measurement.

And rather than starting with measurement as the be-all and end-all goal, we should begin by defining how best to organize care to achieve the best possible results for patients, through models like patient-centered medical homes, then determine a payment model that best supports those models, and then build and incorporate measures that actually help the physicians in these systems monitor and achieve the best possible outcomes for their patients—not the other way around.

If we really believe, as ACP does, that a well-trained internist, in a system of care designed to achieve the best outcomes for patients, will be shown to be the best bargain in American medicine, then performance measures can be our friends—but only if they are the right measures, measuring the right things, for the right reasons, and with the right oversight. And we should always keep in mind the cautionary note from sociologist William Bruce Cameron, sometimes misattributed to Albert Einstein, “That not everything that can be counted counts, and not everything that counts can be counted.”