I recently spoke to a quality measures development organization and it got me thinking — what makes a good doctor, and how do we measure it?

In thinking about this, I reflected on how far we have come on quality measurement.  A decade or so ago, many physicians didn’t think the quality of their care could be measured and any attempt to do so was “bean counting” folly at best or destructive and dangerous at worse.  Yet, in the last decade, we have seen a sea change.  We have developed hundreds of quality measures and physicians are grumblingly accepting that quality measurement is here to stay.

But the unease with quality measurement has not gone away and here’s why.  If you ask “quality experts” what good care looks like for a patient with diabetes, they might apply the following criteria:  good hemoglobin A1C control, regular checking of cholesterol, effective LDL control, smoking cessation counseling, and use of an ACE inhibitor or ARB in subsets of patients with diabetes.  Yet, when I think about great clinicians that I know — do I ask myself who achieves the best hemoglobin A1C control? No. Those measures — all evidence-based, all closely tied to better patient outcomes — don’t really feel like they measure the quality of the physician.

So where’s the disconnect?  What does make a good doctor?  Unsure, I asked Twitter:

Over 200 answers came rolling in.  Listed below are the top 10.  Top answer? Having empathy. #2? Being a good listener.  It wasn’t until we get to #5 that we see “competent/effective.”

Even though the survey results above come from those I interact with on Twitter, I suspect the results reflect what most Americans would want. As I read the discussions that followed, I came to conclude one thing:  Most people assume that physicians meet a threshold of intelligence, knowledge, and judgment and therefore, what differentiates good doctors from mediocre ones is the “soft” stuff.

It’s an interesting set of assumptions, but is it true?  It is, at least somewhat.  Most American physicians meet a basic threshold of competence — our system of licensure, board exams, etc. ensure that a vast majority of physicians have at least a basic level of knowledge.  What most people don’t appreciate, however, is that even among this group, there are large, meaningful variations in capability and clinical judgment.

And, of course, a small minority of people are able to get licensed without meeting the threshold at all.  We all know these physicians — a small number to be sure — that are dangerously ineffective.  We, the medical community, have been terrible about singling these physicians out and asking them to get better — or leave the profession.

In the Twitter discussion, there was a second point raised by John Birkmeyer and that was likely on the minds of many respondents.  He said “I’d want different things from my PCP and heart surgeon. Humility. Overrated for the latter.” John was raising a key distinction between what we want out of a physician (an internist or a family practitioner) versus a surgeon.  Yes, in order to be “good,” humility and empathy are important, even for cardiac surgeons. But when they are cutting into your sternum?  You want them to be technically proficient and that trait trumps their ability (or lack thereof) to be empathic. Surgeons’ empathy and kindness matter — but it may not be as critical to their being an effective surgeon as their technical and team management skills. For internists, effectiveness is much more dependent on their ability to listen, be empathic, and take patients’ values into consideration.

A final point.  My favorite tweet came from Farzad Mostashari, who asked, “If your doctor doesn’t use the best data available to them to take care of you, do they really care about you?” In all the discussions about being a good doctor, we heard little about effective use of beta-blockers for heart disease, or good management of diabetes care.  That’s the stuff we measure, and it’s important. We use them as part of the Physician Quality Reporting System (PQRS).  But I’m not sure they really measure the quality of the physician.  They measure quality of the system in which the physician practices.  You can have a mediocre physician, but on a good team with excellent clinical support staff, those things get done.  Even the smartest physician who knows the evidence perfectly can’t deliver consistently reliable care if there isn’t a system built around him or her to do so.

So, when it comes to thinking about ambulatory care quality — we should think about two sets of metrics: What it means to be a good doctor and what it means to work in a good system.  In measuring doctor quality, we might focus on “soft” skills like empathy, which we can measure through patient experience surveys.  But we also have to focus on intellectual skills, such as ability to make difficult diagnoses and emotional intelligence, such as the ability to collaborate and effectively lead teams — and we don’t really measure these things at all, erroneously assuming that all clinicians have them.

For measuring good systems, we could use our current metrics such as whether they achieve good hypertension and diabetes control.  We need to keep these two sets of metrics separate and not confuse one for the other. And, alas, for surgeons, we need a different approach yet.  Yes, I still believe that humility and empathy go a long way — but these qualities are no substitute for sound judgment and a steady hand.

Ashish Jha is an associate professor of health policy and management, Harvard School of Public Health, Boston, MA.  He blogs at An Ounce of Evidence and can be found on Twitter @ashishkjha.