Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

Why ProPublica’s Surgeon Scorecard is worth celebrating

Ashish Jha, MD, MPH
Physician
July 26, 2015
Share
Tweet
Share

shutterstock_153912641

Recently, Marshall Allan and Olga Pierce, two journalists at ProPublica, published a surgeon report card detailing complication rates of 17,000 individual surgeons from across the nation. A product of many years of work, it benefitted from the input of a large number of experts (as well as folks like me). The report card has received a lot of attention … and a lot of criticism. Why the attention? Because people want information about how to pick a good surgeon. Why the criticism?  Because the report card has plenty of limitations.

As soon as the report was out, so were the scalpels. Smart people on Twitter and blogs took the ProPublica team to task for all sorts of reasonable and even necessary concerns. For example, it only covered Medicare beneficiaries, which means that for many surgeries, it missed a large chunk of patients. Worse, it failed to examine many surgeries altogether. But there was more.

The report card used readmissions as a marker of complications, which has important limitations. The best data suggest that while a large proportion of surgical readmissions are due to a complication, readmissions are also affected by other factors, such as how sick the patient was prior to surgery (the ProPublica team tried to account for this), his or her race, ethnicity, social supports — and even the education and poverty level of their community. I have written extensively about the problems of using readmissions after medical conditions as a quality measure. Surgical readmissions are clearly better but hardly perfect. They even narrowed the causes of readmissions using expert input to improve the measure, but even so, it’s hardly ideal. ProPublica produced an imperfect report. 

How to choose a surgeon

So what to do if you need a surgeon?  Should you use the ProPublica report card?  You might consider doing what I did when I needed a surgeon after a shoulder injury two years ago: ask colleagues. After getting input about lots of clinicians, I honed in on two orthopedists who specialized in shoulders. I then called surgeons who had operated with these guys and got their opinions. Both were good, I was told, but one was better. Yelp?  I passed. Looking them up on the Massachusetts Registry of Medicine?  Seriously?  Never crossed my mind.

But what if, just by chance, you are not a physician? What if you are one of the 99.7 percent of Americans who didn’t go to medical school? What do you do?  If your insurance covers a broad network and your primary care physician is diligent and knows a large number of surgeons, you may get referred to someone right for you. Or, you could rely on word of mouth, which means relying on a sample size of one or two.

So what do patients actually do?  They cross their fingers, pray, and hope that the system will take care of them. How good is that system at taking care of them? It turns out, not as good as it should be. We know that mortality rates vary three-fold across hospitals. Even within the same hospital, some surgeons are terrific, while others? Not so much. Which is why I needed to work hard to find the right orthopedist. Physicians can figure out how to navigate the system. But what about everyone else?

I was on service recently and took care of a guy, Bobby Johnson (name changed, but a real guy), who was admitted yet again for an ongoing complication from his lung surgery. He had missed key events because of his complications — including his daughter’s wedding — because he was in the hospital with a recurrent infection. He wondered if he would have done better with a different hospital or a different surgeon. I didn’t know how to advise him.

And that’s where ProPublica comes into play. The journalists spent years on their effort, getting input from methodologists, surgeons, and policy experts. In the end, they produced a report with a lot of strengths, but no shortage of weaknesses. But despite the weaknesses, I never heard them question whether the endeavor was worth it at all.  I’m glad they never did.

Because the choice wasn’t between building the perfect report card and building the one they did. The choice was between building their imperfect report card and leaving folks like Bobby with nothing. In that light, the report card looks pretty good. Maybe not for experts, but for Bobby.

A step towards intended consequences

 Colleagues and friends that I admire, including the brilliant Lisa Rosenbaum, have written about the unintended consequences of report cards. And they are right. All report cards have unintended consequences. This report card will have unintended consequences. It might even make, in the words of a recent blog, “some Morbidity Hunters become Cherry Pickers” (a smart, witty, but quite critical piece on the ProPublica Report Card). But asking whether this report card will have unintended consequences isn’t the right question. The right question is — will it leave Bobby better off? I think it will. Instead of choosing based on a sample size of one (his buddy who also had lung surgery), he might choose based on sample size of 40 or 60 or 80. Not perfect. Large confidence intervals? Sure. Lots of noise?  Yup. Inadequate risk adjustment? Absolutely. But, better than nothing? Yes. A lot better.

All of this gets at a bigger point raised by Paul Levy:  Is this really the best we can do? The answer, of course, is no. We can do much better, but we have chosen not to. We have this tool — it’s called the National Surgical Quality Improvement Program (NSQIP). It uses clinical data to carefully track complications across a large range of surgeries, and it’s been around for about twenty years. Close to 600 hospitals use it (and about 3,000 hospitals choose not to). And no hospital that I’m aware of makes its NSQIP data publicly available in a way that is accessible and usable to patients. A few put summary data on Hospital Compare, but it’s inadequate for choosing a good surgeon. Why are the NSQIP data not collected routinely and made widely available? Because it’s hard to get hospitals to agree to mandatory data collection and public reporting. Obviously those with the power of the purse — Medicare, for instance — could make it happen. They haven’t.

Disruptive innovation, a phrase coined by Clay Christensen, is usually a new product that, to experts, looks inadequate. Because it is. These innovations are not, initially, as good as what the experts use (in this case, their network of surgeons). They initially dismiss the disrupter as being of poor quality. But disruptive innovation takes hold because, for a large chunk of consumers (i.e., patients looking for surgeons), the innovation is both affordable and better than the alternative. And once it takes hold, it starts to get better. And as it does, its unintended consequences will become dwarfed by its intended consequences:  making the system better. That’s what ProPublica has produced.  And that’s worth celebrating.

ADVERTISEMENT

Ashish Jha is an associate professor of health policy and management, Harvard School of Public Health, Boston, MA.  He blogs at An Ounce of Evidence and can be found on Twitter @ashishkjha.

Image credit: Shutterstock.com

Prev

Analyzing the deficit of African-Americans in academic medicine

July 25, 2015 Kevin 40
…
Next

I will not let a computer come between me and my patient

July 26, 2015 Kevin 23
…

Tagged as: Surgery

Post navigation

< Previous Post
Analyzing the deficit of African-Americans in academic medicine
Next Post >
I will not let a computer come between me and my patient

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

More by Ashish Jha, MD, MPH

  • Ranking the world’s health systems: These results may surprise you

    Ashish Jha, MD, MPH
  • How much does it matter which hospital you go to?

    Ashish Jha, MD, MPH
  • Men and women doctors versus correlation and causation

    Ashish Jha, MD, MPH

Related Posts

  • A physician’s addiction to social media

    Amanda Xi, MD
  • Why creative endeavors are important for the future surgeon

    Thomas L. Amburn
  • Medicine was consuming this medical student. Was it worth it?

    Sarah B. El Iskandarani
  • Paging the surgeon general: America needs you

    Linda Girgis, MD
  • The irony of me observing death on a day celebrating my birth

    Johnathan Yao, MD, MPH
  • A trauma surgeon reflects on the Yale System, 20 years later

    Ara Feinstein, MD, MPH

More in Physician

  • Why some doctors age gracefully—and others grow bitter

    Patrick Hudson, MD
  • The hidden incentives driving frivolous malpractice lawsuits

    Howard Smith, MD
  • Mastering medical presentations: Elevating your impact

    Harvey Castro, MD, MBA
  • Marketing as a clinician isn’t about selling. It’s about trust.

    Kara Pepper, MD
  • How doctors took back control from hospital executives

    Gene Uzawa Dorio, MD
  • How art and science fueled one woman’s path to medicine

    Amy Avakian, MD
  • Most Popular

  • Past Week

    • Why Medicaid cuts should alarm every doctor

      Ilan Shapiro, MD | Policy
    • When the diagnosis is personal: What my mother’s Alzheimer’s taught me about healing

      Pearl Jones, MD | Conditions
    • 2 hours to decide my future: How the SOAP residency match traps future doctors

      Nicolette V. S. Sewall, MD, MPH | Education
    • Key strategies for smooth EHR transitions in health care

      Sandra Johnson | Tech
    • Reassessing the impact of CDC’s opioid guidelines on chronic pain care [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why removing fluoride from water is a public health disaster

      Steven J. Katz, DDS | Conditions
  • Past 6 Months

    • Why tracking cognitive load could save doctors and patients

      Hiba Fatima Hamid | Education
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • The silent toll of ICE raids on U.S. patient care

      Carlin Lockwood | Policy
    • “Think twice, heal once”: Why medical decision-making needs a second opinion from your slower brain (and AI)

      Harvey Castro, MD, MBA | Tech
    • Why we fear being forgotten more than death itself

      Patrick Hudson, MD | Physician
    • Bureaucracy over care: How the U.S. health care system lost its way

      Kayvan Haddadan, MD | Physician
  • Recent Posts

    • How to survive a broken health care system without losing yourself [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why some doctors age gracefully—and others grow bitter

      Patrick Hudson, MD | Physician
    • Why removing fluoride from water is a public health disaster

      Steven J. Katz, DDS | Conditions
    • What the research really says about infrared saunas

      Khushali Jhaveri, MD | Conditions
    • How the cycle of rage is affecting physicians—and how to break free

      Alexandra M.P. Brito, MD and Jennifer L. Hartwell, MD | Conditions
    • Why ADHD in adults is often missed—and why it matters [PODCAST]

      The Podcast by KevinMD | Podcast

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 12 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

ADVERTISEMENT

  • Most Popular

  • Past Week

    • Why Medicaid cuts should alarm every doctor

      Ilan Shapiro, MD | Policy
    • When the diagnosis is personal: What my mother’s Alzheimer’s taught me about healing

      Pearl Jones, MD | Conditions
    • 2 hours to decide my future: How the SOAP residency match traps future doctors

      Nicolette V. S. Sewall, MD, MPH | Education
    • Key strategies for smooth EHR transitions in health care

      Sandra Johnson | Tech
    • Reassessing the impact of CDC’s opioid guidelines on chronic pain care [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why removing fluoride from water is a public health disaster

      Steven J. Katz, DDS | Conditions
  • Past 6 Months

    • Why tracking cognitive load could save doctors and patients

      Hiba Fatima Hamid | Education
    • What the world must learn from the life and death of Hind Rajab

      Saba Qaiser, RN | Conditions
    • The silent toll of ICE raids on U.S. patient care

      Carlin Lockwood | Policy
    • “Think twice, heal once”: Why medical decision-making needs a second opinion from your slower brain (and AI)

      Harvey Castro, MD, MBA | Tech
    • Why we fear being forgotten more than death itself

      Patrick Hudson, MD | Physician
    • Bureaucracy over care: How the U.S. health care system lost its way

      Kayvan Haddadan, MD | Physician
  • Recent Posts

    • How to survive a broken health care system without losing yourself [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why some doctors age gracefully—and others grow bitter

      Patrick Hudson, MD | Physician
    • Why removing fluoride from water is a public health disaster

      Steven J. Katz, DDS | Conditions
    • What the research really says about infrared saunas

      Khushali Jhaveri, MD | Conditions
    • How the cycle of rage is affecting physicians—and how to break free

      Alexandra M.P. Brito, MD and Jennifer L. Hartwell, MD | Conditions
    • Why ADHD in adults is often missed—and why it matters [PODCAST]

      The Podcast by KevinMD | Podcast

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

Why ProPublica’s Surgeon Scorecard is worth celebrating
12 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...