Like it or not, “garbage in, garbage out” is the rule rather than the exception for the vast majority of bureaucratically dictated “quality control” programs that we’re ever likely to launch.  There are three primary reasons for this.

First and foremost, there’s Goodhart’s Law:

“Originally, an economic theory stating that if a particular definition of the money supply were to be used as the basis for monetary policy, the stability of its statistical relationship with spending on the economy would break down and the policy would prove ineffective. The law is now used more widely to highlight the problems of focusing on the value of any specific variable as an indicator. In simple terms, when a measure becomes a target, it ceases to be a good measure. The applications to performance measurement in management accounting and other aspects of business are obvious.”

Here’s how it works.  In its natural state, the percentage of diabetes getting Hgb A1c tests might tell us something useful about the need for more education about diabetes management, the reluctance of patients to get these tests, an inability to pay for them, the inappropriateness of getting these tests in certain patients, or a whole host of other possible explanations for less than 100% testing rates.  But when we start to use the presence or absence of the test as a basis for reward or punishment, the statistics stop telling us much of anything.  Doctors will get the test whether it’s appropriate or not.  Patients who fail to comply will be discharged from practices.  And even when testing rates climb to 100%, the existence of the test alone tells us nothing about the actual quality of diabetes management for these patients.  All it does is tell us that the threats of reward or punishment increased the reported rate of testing.  Period.

Dr. James Gaulte’s blog recently highlighted an example of patient selection bias that will become inevitable as “pay for performance” initiatives expand.

Second, much if not most of medicine cannot readily be standardized in a way that makes “quality” indicators meaningful.  Dr. Rich Fogoros has eloquently described why this is the case:

“…not all medical processes are suitable for standardization.

The standardization tools of managed care work only when you’re dealing with a process that can be broken down into a predictable series of discrete, reproducible tasks that will generate reproducible results.  In other words, industrial management tools work best when the process of care is similar to the process of making widgets.

Hip replacement surgery, for instance, tends to be reasonably widget-like.  We know, for instance, that on Day 1 the hip replacement operation itself will take place.  We also know that since hip replacement is usually an elective procedure, any other medical conditions the patient may have will have been stabilized prior to surgery (and prior to entrance to the critical pathway), and so should not present unexpected problems during the hospitalization.  Thus the critical pathway can focus solely on steps to minimize the risk of complications of surgery, and to maximize rapid recovery. For hip replacement and many other elective surgical procedures, the use of critical pathways has resulted in reduced lengths of hospital stays, less cost, and more rapid (or at least, no worsening in the time of) recovery.

In contrast, developing critical pathways for many non-surgical hospital admissions has proven extremely problematic.  For many medical illnesses, neither the diagnostic procedures nor the treatments that may be employed are possible to predict, or thus to standardize.  For instance, consider what happens when we try to develop a critical pathway for congestive heart failure (CHF). Patients with CHF may have one or more of a variety of underlying conditions that caused their heart failure in the first place (such as coronary artery disease, valvular heart disease, viral infections of the heart muscle, and many others); they vary widely in their severity of illness (from mildly ill to moribund); and they often have related complicating disorders of one or more additional organ systems, such as kidney failure or peripheral vascular disease.  These factors, along with a multitude of others, ultimately determine what diagnostic and therapeutic maneuvers will be necessary.  Knowing only that a patient has been admitted to the hospital with CHF tells you nothing about whether that patient will require cardiac catheterization, angioplasty, bypass surgery, valve replacement, a pacemaker, an implantable defibrillator, a mechanical ventilator, a prolonged and complicated stay in the intensive care unit, or just a couple of diuretic tablets and overnight observation.  No two patients with CHF are alike; and there is no such thing as a “standardized” patient.

For medical conditions like this, in which every patient tends to be unique, managed care techniques tend not to be very useful.  Unfortunately, the majority of non-surgical hospital admissions fall into this category…

…In summary, “pure” managed care has given us some very useful ideas about how to make healthcare delivery more efficient without diminishing medical outcomes.  The principles of managed care are being widely and profitably used in most large hospitals in America today, and without a doubt have the potential of even broader applicability. However, contrary to the dogma, these principles are not applicable in many cases, nor do they always yield favorable results.”What all of this means in real-world terms is that most of the medical “quality” problems that happen in the real world can’t be meaningfully tracked, categorized or addressed by brainless, bureaucratically mandated pay-for-performance programs or penalties.  Teasing out the crummy decisions from the rational ones would take serious case review, thought and evaluation by other physicians who actually know something about the anatomy, physiology, chemistry, psychology and economic particulars of the individuals involved.  Anything else is hogwash.

Unfortunately, that’s not how bureaucrats roll.  It’s a heck of a lot easier for them to simply ask that doctors waste hours turning in sets of forms and checklists.  All of which costs you the tax- and rate-payer a fortune, and does absolutely nothing to improve the quality of care.

The third systemic problem with “quality” programs is the one we’ve already identified: garbage data in, garbage reports out.  The Obama administration and the healthcare information technology lobby have used this problem as a major reason to push for the immediate purchase and use of electronic medical records by physicians, as well as spending billions of taxpayer dollars to subsidize these purchases.  Consider the language used in the 2009 stimulus bill that allocated about $20 billion to HIT purchases:

(b) PURPOSE.—The National Coordinator shall perform the duties under subsection (c) in a manner consistent with the development of a nationwide health information technology infrastructure that allows for the electronic use and exchange of information and that—

(1) ensures that each patient’s health information is secure and protected, in accordance with applicable law;

(2) improves health care quality, reduces medical errors, reduces health disparities, and advances the delivery of patient-centered medical care;

(3) reduces health care costs resulting from inefficiency, medical errors, inappropriate care, duplicative care, and incomplete information;

(4) provides appropriate information to help guide medical decisions at the time and place of care;

While there is certainly a great deal of work that could be done to improve healthcare information and communications, the systems being forced upon providers nationwide are highly unlikely to have the spectacular effects advertised.  There are very real questions about the safety of these systems themselves.  Interoperability among the complex, expensive proprietary systems on the market today is minimal.  Most importantly, however, as long as incentives are designed to look for one specific result or another, providers and information systems will dutifully produce that result as ordered.

All of these problems might not matter so much if such “quality” programs were costless or without social and medical consequences, but they are clearly not.  Ironically, it is practically impossible to determine what the true cost of these programs really is.  No good estimates of the total spending for government and/or private “quality” programs seems to exist.  Cost-benefit analyses do not seem to be a common component of these programs.  What is certain is that physician-insurer interactions already cost medical practices between $23 billion to $31 billion per year, and consume an average of over one-half work day per provider per week.

Doug Perednia is an internal medicine physician and dermatologist who blogs at Road to Hellth.

Submit a guest post and be heard.