The VA scandal exposes the folly of metrics

The Veteran’s Administration is under fire for covering up deaths. Men and women who were eligible for care languished on impossibly long waiting lists and even worse when some died waiting for care their deaths were covered up. This is horrific and everyone wants to know how this tragedy could have happened?

Veteran’s hospitals have long waits in my experience because they are underfunded, many (if not all) patients with complex medical issues and often complex social ones, and have systems so complex that you need a lifetime of working there to navigate the system (that’s what happens when you breed the bureaucracy of the military with the bureaucracy of hospital administrators).

I spent a little time at the VA in Colorado and I could never understand the system of who was allowed to get what care or when or how. We had a clinic nurse who cared deeply (and was doing the job of three people) and she served as our universal VA translator. Think dealing with insurance companies is a challenge? The VA makes them look like red tape amateurs.

To solve the issue of delays in care a metric was born: timely care within 14 days.

Goals are important in medicine, whether it’s access to care, choosing the right first-line antibiotic, or reducing bed sores. Without goals and data it’s hard to know if the changes you have implemented are helping your patients. There are many areas for improvement in medicine and metrics can help us see that. However, metrics also have a very dark side because not everyone is honest and some people may start out with the best of intentions but when flummoxed by a seemingly insurmountable challenge don’t always do the right thing.

With the VA fiasco a bonus for the senior staff was tied to the metric of timely care within 14 days. Tying rewards or penalties to metrics seems to encourage some people to think even more about the metric and less about the actual problem. To meet the metric of timely access VA administrators could:

overhaul the system
go public with how it was impossible to fix the system given the rule book and the money allowed
use sleight of hand to drop names from the waiting list

We can all agree that identifying a problem and setting a goal is important. You can’t change your antibiotic prescribing patterns if you don’t know where you are now and where you should be. However, carrots and sticks may not always be the right way to achieve the desired outcome.

Let’s take the urinary tract infection (UTI) example a little further and say that 40% of uncomplicated UTIs are getting the antibiotic ciprofloxacin. Ciprofloxacin is not a first line antibiotic so you want to reduce the prescribing rate to 5%. Ways to go about that include educating physicians, pharmacists, and patients in addition to tracking data, providing feedback and maybe individual problem solving for those physicians who just can’t stop giving ciprofloxacin inappropriately.

Now tie money to that outcome, do you think one (or more) physicians or administrators or pharmacists would be more or less likely to change one or two diagnosis from uncomplicated to complicated urinary tract infection thus satisfying the metric? I’m not saying this would even be intentional. Say Mrs. Smith is insisting that only ciprofloxacin “works for her,” yet you know from the test that nitrofurantoin will work and she has no contraindications to that drug.

Might one doctor somewhere when faced with an ever-growing delay in his/her day as the conversation with Mrs. Smith takes longer and longer convince him or herself that maybe Mrs. Smith actually has a complicated UTI and thus will fall out of the metric so ciprofloxacin is really OK? That would be the wrong decision for many reasons, but I can see it happening and not even driven by anything nefarious like money but rather desperation for the day, exasperation, and a desire to please Mrs. Smith (because pleasing the patient is actually metric too).

Whether it is on purpose or just a crappy day metrics are at risk of being fudged when people start to think more about numbers than the patients they represent. And adding money into the mix? Higher stakes may entice some to do the wrong thing. Some professional athletes dope because they want the gold medal. Some police departments downgrade rapes or don’t investigate them at all to make it look like they are meeting the crime rate metrics (22% of police departments that serve populations >100,000 have severe irregularities in their rape reporting statistics) as for many departments funding is metric driven. Metric madness is not a medicine-only phenomenon.

I really believe that most individuals want to do the right thing, but when faced with what may seem like insurmountable metrics (based on the actual money allotted or the nature of the problem), well, strange things can happen to data under bad leadership.

Metrics sadly teach some people to satisfy the metric, not solve the problem.