Skip to content
  • About
  • Contact
  • Contribute
  • Book
  • Careers
  • Podcast
  • Recommended
  • Speaking
  • All
  • Physician
  • Practice
  • Policy
  • Finance
  • Conditions
  • .edu
  • Patient
  • Meds
  • Tech
  • Social
  • Video
    • All
    • Physician
    • Practice
    • Policy
    • Finance
    • Conditions
    • .edu
    • Patient
    • Meds
    • Tech
    • Social
    • Video
    • About
    • Contact
    • Contribute
    • Book
    • Careers
    • Podcast
    • Recommended
    • Speaking

When science ceases to be value-neutral, science ceases to be science

Dr. Saurabh Jha
Physician
May 22, 2017
Share
Tweet
Share

Recently, the Harvard Chan School of Public Health reported on the effect of surgical checklists in South Carolina. The press release was titled, “South Carolina hospitals see major drop in post-surgical deaths with nation’s first proven statewide Surgical Safety Checklist Program.”

The Health News Review, for which I review, grades coverage of research in the media. Based on their objective criteria, the Harvard press release would not score highly.

The title exudes certainty — “nation’s first proven.” The study, not being a randomized controlled trial (RCT), suggests that checklists are effective, but far from proves it. At least one study failed to show that surgical checklists improve outcomes.

The press release’s opening line is “South Carolina saw a 22 percent reduction in deaths.” It reports relative risk reduction (RRR). Reporting RRR is now considered a cardinal sin in health care journalism because RRR inflates therapeutic optimism by making the intervention sound more efficacious than it is.

A good press release gives the reader the whole picture of competing evidence, particularly where there’s controversy. The press release mentioned one other study which showed an even higher mortality benefit of surgical checklists.

The press release goes on to say:

Adoption of a safe surgery checklist has been demonstrated to reduce deaths in controlled research studies since 2009. But the ability to produce improved outcomes at large scale has remained questioned.

In other words, the science is settled. But there is no mention of the Canadian study which failed to show surgical checklists improve outcomes. The reader is lulled into a false sense of certainty about the benefits of surgical checklists.

If this was a press release for a statin or a device, which, ironically, subjects itself to greater methodological rigor, it would have been taken to task. Yet, no one batted an eyelid over the press release about surgical checklists. Why does a sensational press release about an inferior-quality study about surgical checklists not induce the same ire as a sensational press release about an RCT of a drug or device?

Future anthropologists might better answer this question. I’m not an anthropologist, let alone a future anthropologist. But I will speculate. In our rational age, we’re still indefatigably prejudiced, self-righteous creatures. Witnessing pharma make big bucks on false therapeutic optimism gets under our skin. Money corrupts. Nothing else quite corrupts to the same degree, we believe.

The proponents of surgical checklists don’t make money off checklists. That the proponents of checklists might advance their academic careers despite the therapeutic optimism of checklists doesn’t perturb. The nobility of their intent — they wish to make surgery safer — shields them from the harsh scrutiny reserved for drug and device industries.

It’s not readily apparent that the advocates of checklist might have the “this is my bright idea” bias. In this regard, the reaction of Atul Gawande, the author of Checklist Manifesto, who is self-evidently a true believer of checklists, to the Canadian study is instructive. He “wished the Ontario study was better.” He felt the study was underpowered and the implementation of checklists was weak.

When you don’t like the results of a study, the next step is to find weaknesses in the study design. The astute reader will note that I’ve done exactly what I’m accusing Gawande of doing. Guilty as charged. I have my biases. And there is no way to determine if one’s search for methodological weaknesses in a study is actuated by intrigue or bias. But my point isn’t whether surgical checklists work or not. My question is why do we not see the same skepticism for surgical checklists, and other policy measures, that we see for drugs and devices.

ADVERTISEMENT

Of course, there are differences between checklists and drugs. Simple checklists cost nearly nothing to implement. If you’re wrong about the efficacy of a drug, that could be costly. If you’ve overestimated the efficacy of a surgical checklist, that’s no big deal.

But the differential standards, as justified as they may be, may lead to a deep mistrust of science and of dabblers in science. A casual observer, unfamiliar with epistemic nuances, may suspect that ideology is at play. The observer may see, in the sliding scale of methodological rigor, blatant double standards. The observer may lose faith in statistics seeing that it can be fine-tuned, like the thermostat of a shower, to prove and disprove what we wish proven or disproven.

When Anne Case and Angus Deaton showed that middle-class white men were not reaping the mortality benefits of other demographics, the demolition of their analysis was swift.

In a sense, this is good because science needs refutation. But would their study have inspired the same statistical scrutiny if it had shown that Hispanic or South Asian men were dying sooner than white men? Skepticism loses credibility if seen to be applied selectively. And if we can’t trust skeptics, who can we trust?

Statisticians can debunk or defend a study using their deep knowledge of statistics by selectively highlighting the strengths and weaknesses of a study — no study design is perfect. The degree to which the weaknesses of a study can be exposed is arbitrary — there’s no normative frame. Thus, statisticians have become the new lawyers. I wonder how long before we have a “right to a statistician.”

The ideologue with a deep knowledge of statistics is the most dangerous ideologue. The ideologue who claims God or history is on their side is easy to identify. The ideologue who claims to have science, which really means statistical technique, on their side is more difficult to identify and refute. Few have deep enough knowledge of statistics to call BS on the callers of BS. The result is that people mistrust science.

Science has become a substitute for morality. It has become a weapon to fight social injustice (whatever that term means), to reduce inequality, to mock incompatible world views. This is a blow for both science and morality. Historically, our species never resorted to science to do the right thing. Slavery was bad because it was morally bad. The Civil Rights Movement was right because it was morally right. The suffragettes were on the right side of history, not because an academic proved the net benefits of women voting in an election (p <0.0005), by analyzing a secondary database. The suffragettes were right because they were morally right. India’s freedom struggle was not initiated by a macroeconomic analysis, but because independence was morally right. If your moral intuitions can’t tell you that same-sex marriage should be permitted, I doubt any scientific analysis will convince you.

It may be morally right to be politically correct. Then we should say so, and keep science away from political correctness. Science loses most credibility when seen to selectively address the favored social injustices of the time. It is not fashionable in elite circles to be perturbed by the plight of the working class white man. In this circle, which I sometimes frequent, a physician seeping social justice from every pore, once said that it was imperative that we developed a vaccine for malaria, because the Africans have suffered enough from imperialism. He was right and wrong. He was right that a malaria vaccine will reduce the suffering of Africans tremendously, and that it was imperative that it be developed. But he was wrong. The vaccine is no more, or no less, important because of the history of the African continent. Alexander Fleming didn’t discover penicillin because he wanted to fight capitalists.

We may discourage reporting RRR for drugs and devices to curb therapeutic optimism and encourage reporting RRR for surgical checklists to encourage compliance. This is understandable, because how we frame a message, and reporting relative risk reduction is just a way of framing, depends on what we wish to achieve. But here lies another problem. Note, science is not informing us what to do. We’ve already decided what we want to do. Science is helping us implement, helping us engineer, helping us design, not a bridge, not a spacecraft, not a computer, but ourselves and society. This was once the dominion of religion. That this is now the dominion of science is ironic. But more troubling than irony is that science is no longer value-neutral. And when science ceases to be value-neutral, science ceases to be science.

Saurabh Jha is a radiologist and can be reached on Twitter @RogueRad.  This article originally appeared in the Health Care Blog.

Image credit: Shutterstock.com

Prev

How are we supposed to deal with death?

May 22, 2017 Kevin 0
…
Next

It's time we look at the Australian health system for lessons

May 22, 2017 Kevin 6
…

Tagged as: Primary Care

Post navigation

< Previous Post
How are we supposed to deal with death?
Next Post >
It's time we look at the Australian health system for lessons

ADVERTISEMENT

More by Dr. Saurabh Jha

  • Masks are an effigy of American technocratic incompetence

    Dr. Saurabh Jha
  • False negative: COVID-19 testing’s catch-22

    Dr. Saurabh Jha
  • Why the Lancet’s editorial on Kashmir is unhelpful

    Dr. Saurabh Jha

Related Posts

  • Is social media a friend or foe of science?

    Michael Joyce, MD
  • Take politics out of science and medicine

    Anonymous
  • Fight gun violence with science

    Jamie Coleman, MD
  • Quality measures have gotten ahead of the science of quality measurement

    Peter Ubel, MD
  • How the science of learning salvaged my college career

    Elijah Hamm
  • Medical school and the science of sleep

    Sarah Murad

More in Physician

  • The quiet grief behind hospital walls

    Aaron Grubner, MD
  • a desk with keyboard and ipad with the kevinmd logo

    How to advance workforce development through research mentorship and evidence-based management

    Olumuyiwa Bamgbade, MD
  • The truth about perfection and identity in health care

    Ryan Nadelson, MD
  • Civil discourse as a leadership competency: the case for curiosity in medicine

    All Levels Leadership
  • When a medical office sublease turns into a legal nightmare

    Ralph Messo, DO
  • Why the heart of medicine is more than science

    Ryan Nadelson, MD
  • Most Popular

  • Past Week

    • Forced voicemail and diagnosis codes are endangering patient access to medications

      Arthur Lazarus, MD, MBA | Meds
    • Why specialist pain clinics and addiction treatment services require strong primary care

      Olumuyiwa Bamgbade, MD | Conditions
    • Who gets to be well in America: Immigrant health is on the line

      Joshua Vasquez, MD | Policy
    • When a medical office sublease turns into a legal nightmare

      Ralph Messo, DO | Physician
    • America’s ER crisis: Why the system is collapsing from within

      Kristen Cline, BSN, RN | Conditions
    • FDA delays could end vital treatment for rare disease patients

      GJ van Londen, MD | Meds
  • Past 6 Months

    • Forced voicemail and diagnosis codes are endangering patient access to medications

      Arthur Lazarus, MD, MBA | Meds
    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
    • Here’s what providers really need in a modern EHR

      Laura Kohlhagen, MD, MBA | Tech
  • Recent Posts

    • The quiet grief behind hospital walls

      Aaron Grubner, MD | Physician
    • Why peer support can save lives in high-pressure medical careers

      Maire Daugharty, MD | Conditions
    • Bundled payments in Medicare: Will fixed pricing reshape surgery costs?

      AMA Committee on Economics and Quality in Medicine, Medical Student Section | Policy
    • How Project ECHO is fighting physician isolation and transforming medical education [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why clinical research is a powerful path for unmatched IMGs

      Dr. Khutaija Noor | Education
    • Addressing menstrual health inequities in adolescents

      Callia Georgoulis | Conditions

Subscribe to KevinMD and never miss a story!

Get free updates delivered free to your inbox.


Find jobs at
Careers by KevinMD.com

Search thousands of physician, PA, NP, and CRNA jobs now.

Learn more

View 4 Comments >

Founded in 2004 by Kevin Pho, MD, KevinMD.com is the web’s leading platform where physicians, advanced practitioners, nurses, medical students, and patients share their insight and tell their stories.

Social

  • Like on Facebook
  • Follow on Twitter
  • Connect on Linkedin
  • Subscribe on Youtube
  • Instagram

ADVERTISEMENT

  • Most Popular

  • Past Week

    • Forced voicemail and diagnosis codes are endangering patient access to medications

      Arthur Lazarus, MD, MBA | Meds
    • Why specialist pain clinics and addiction treatment services require strong primary care

      Olumuyiwa Bamgbade, MD | Conditions
    • Who gets to be well in America: Immigrant health is on the line

      Joshua Vasquez, MD | Policy
    • When a medical office sublease turns into a legal nightmare

      Ralph Messo, DO | Physician
    • America’s ER crisis: Why the system is collapsing from within

      Kristen Cline, BSN, RN | Conditions
    • FDA delays could end vital treatment for rare disease patients

      GJ van Londen, MD | Meds
  • Past 6 Months

    • Forced voicemail and diagnosis codes are endangering patient access to medications

      Arthur Lazarus, MD, MBA | Meds
    • How President Biden’s cognitive health shapes political and legal trust

      Muhamad Aly Rifai, MD | Conditions
    • Why are medical students turning away from primary care? [PODCAST]

      The Podcast by KevinMD | Podcast
    • The One Big Beautiful Bill and the fragile heart of rural health care

      Holland Haynie, MD | Policy
    • Why “do no harm” might be harming modern medicine

      Sabooh S. Mubbashar, MD | Physician
    • Here’s what providers really need in a modern EHR

      Laura Kohlhagen, MD, MBA | Tech
  • Recent Posts

    • The quiet grief behind hospital walls

      Aaron Grubner, MD | Physician
    • Why peer support can save lives in high-pressure medical careers

      Maire Daugharty, MD | Conditions
    • Bundled payments in Medicare: Will fixed pricing reshape surgery costs?

      AMA Committee on Economics and Quality in Medicine, Medical Student Section | Policy
    • How Project ECHO is fighting physician isolation and transforming medical education [PODCAST]

      The Podcast by KevinMD | Podcast
    • Why clinical research is a powerful path for unmatched IMGs

      Dr. Khutaija Noor | Education
    • Addressing menstrual health inequities in adolescents

      Callia Georgoulis | Conditions

MedPage Today Professional

An Everyday Health Property Medpage Today
  • Terms of Use | Disclaimer
  • Privacy Policy
  • DMCA Policy
All Content © KevinMD, LLC
Site by Outthink Group

When science ceases to be value-neutral, science ceases to be science
4 comments

Comments are moderated before they are published. Please read the comment policy.

Loading Comments...