A study published in JAMA looking at the brains of former football players donated to a brain bank, a highly selective sample, found signs of chronic traumatic encephalopathy (CTE) — a degenerative neurological disease — in nearly all deceased players examined.

The mainstream media converged towards numerical consensus.

• Fox News reported players in a study had CTE.”
• CNN reframed the headline, “CTE found in 99% of studied brains from deceased NFL players.”
• TIME, deciding it was time for numerical heterogeneity, said “Degenerative brain disease found in 87% of former football players.”

Though it’s heartening to see a rare consensus in the polarized American media, it is disheartening to see so much inaccuracy in that consensus. When Fox News and CNN agree it seems it’s either raining outside or they’re both more wrong than when they disagree.

Even STAT, the new kid on the media block which prides itself on evidence-based reporting, tweeted: “A new report finds a high incidence of chronic traumatic encephalopathy (brain disease) in football players.” Notwithstanding their numerical agnosticism, I suspect that by “high incidence” STAT meant more than half.

The numerical inaccuracy in the STAT tweet, which linked to an Associated Press wire story, was captured by veteran blogger, Skeptical Scalpel.

Why can't @statnews get it right? The CTE study has nothing to do with incidence. It's 111 brains with no denominator. https://t.co/jbR8vg66Qj

— Skeptical Scalpel (@Skepticscalpel) July 26, 2017

What the headlines are telling the readers, in no uncertain terms, is that CTE is a near certainty in NFL players. Once you read that headline, nuance becomes noise.

A condition that’s too complex for simplistic click-bait headlines

Chronic traumatic encephalopathy, first described in a professional football player by Bennett Omalu, a courageous pathologist, is a chronic brain disorder caused by repetitive head trauma. It was an under-recognized condition, not least because the National Football League (NFL) suppressed the link between football and CTE. As with other under-recognized entities that become the focus of media attention, there’s a risk the pendulum swings to the other extreme, and CTE becomes over-recognized and overdiagnosed.

That repetitive, subclinical, head injury may cause long term negative consequences is biologically quite plausible — as a rule, the more you repeatedly traumatize any physical structure, vestigial or useful, the more likely it’ll be damaged. Soldiers get “march” fractures. Runners can hurt their iliotibial band, which is why I stopped running. Fast cricket bowlers can damage their shoulders. Even radiologists aren’t spared — using the mouse to scroll through thousands of images can inflame our wrists.

CTE is complex. It’s an autopsy diagnosis, which seems rather late in the day to confirm the disease. But brain biopsies aren’t innocuous. The pathological hallmark of CTE, tau in the brain’s cortex, has been seen in other disorders, meaning the pathological findings of CTE are not exclusive to CTE.

CTE shares symptoms with other neurological and psychiatric disorders. How accurately can we attribute depression and suicidal ideations, which could also be present in people who watch football, to repeated head trauma?

Not everyone with a pathological diagnosis of CTE has symptoms of CTE and not everyone with symptoms attributable to CTE has pathological findings of CTE.

Uncertainties about diagnosis are more than academic bickering

The uncertainty with CTE doesn’t mean CTE doesn’t exist. Rather, it means it’s difficult determining the true prevalence of CTE. Researchers are using advanced neuroimaging such as diffusion tensor imaging to diagnose and predict CTE. But imaging is only as good as the gold standard for the diagnosis of CTE. And if the gold standard for CTE is wobbly, or uncertain, so will be the diagnostic and predictive tools for CTE.

The true prevalence of CTE is the crucial question. If the prevalence of CTE truly is 99 percent — that is, 99 percent of NFL players will develop clinically significant CTE (meaning they develop symptoms of impairment during their lifetime), the implications are profound. But we must remember that clinically significant CTE is not the same as autopsy-confirmed CTE (which indicates brains showing pathological evidence of the disease after death, from individuals who may or may not have had symptoms while alive).

If the prevalence of clinically significant CTE is only 10 percent the implications can’t be ignored, but aren’t as profound as with 99 percent prevalence. Note, the dispute isn’t between a prevalence of 48 percent versus 44 percent — that’s just academic bickering. It’s between a very high prevalence, near certainty, and a much lower prevalence of clinically significant CTE.

Like concussion denial, overstatement of CTE can cause harm

I, for one, shall not shed tears if American football is banned. Aside from my disdain for this sport’s merciless assault on nomenclature, watching “football” puts me at risk of falling into the abyss. I believe Americans should, instead, play cricket, a sport in which there are regular breaks so that the players can enjoy tea and crumpets.

Even if there’s certainty that a significant number of NFL players will develop clinically important CTE in their lifetimes, the matter isn’t easily resolved. For some, including many from disadvantaged communities, playing football — not necessarily at the NFL level — is a ticket to higher education. For a small number of elite players, it can become a richly rewarding career. Closing this avenue may placate an armchair pontificator like me, but is hardly generous to those who seek and need this avenue. Furthermore, there’s a trade-off between the harms and the cardiovascular gains of contact sport, in fact, any sport — even cycling has risks. At a time when more kids need to be on their feet running in the fields, scaring their parents about possible and plausible risks may cause net harm.

It’s easy for CTE to touch our inner sanctum of justice because of the narrative it induces — helpless youth from minority communities goaded by the greedy NFL into performing like gladiators for rich sponsors and the masses. When there’s such a large exchange of money, as there is in the NFL, it’s easy for our rage to ignore the uncertainty surrounding the science. And it is precisely when our rage is so easily conscripted that our skepticism and demand for rigor are most vital.

Selection bias casts doubt on 99 percent figure

The study looked at the prevalence of CTE in a brain bank. Brains were donated by families of football players. While the donation could be motivated by a genuine desire to advance science, arguably it could have also been motivated by symptoms of the deceased — i.e. a suspicion of CTE. This is a biased sample and extrapolating the numbers to the population of all football players is neglecting the denominator — a rookie statistical error.

Imagine if I extrapolated the incidence of forearm fractures in cyclists (4/4 in my last call – i.e., 100 percent) to the entire population of cyclists. My Twitter buddy and cycling enthusiast, John Mandrola, would eat me alive! I agree this is an egregious example, but you get the drift with selection bias.

Many of the media reports mentioned the selection bias in the brain bank study.

For example, the Associated Press story that ran in STAT said:

“The report doesn’t confirm that the condition is common in all football players; it reflects high occurrence in samples at a Boston brain bank that studies CTE. Many donors or their families contributed because of the players’ repeated concussions and troubling symptoms before they died.”

This makes the headline of that piece — “Brain disease CTE seen in most football players in large report” — even odder. Meaning it’s odd to cite the elevated prevalence of CTE in the headline despite knowing that it could very well be wrong, despite acknowledging the selection bias. It’s like my apologizing for doing something, and doing it regardless, hoping that my prophylactic apology mitigates my actions.

I understand that equivocation is boring, and I don’t want health care journalism to be like peer-reviewed medical journals — no one would read the articles. I understand that journalists must decide on an end message and craft their narrative and marshal their sources to convey that message. I get the allure of retaining a subtle hint of objectivity. And it seems that the message journalists wish to convey with CTE is “beware.” The message I’d like them to convey instead is “it’s not clear-cut how prevalent CTE is. Consider the trade-offs.”

Selective skepticism is an insidious form of bias

“Beware of CTE” sells more than “consider the uncertainty and trade-offs” and that’s fine, because it’s how we’re wired. But then let’s step back and see what is happening. Health care journalists, rightly, turn on their skeptical noses when they report the efficacy of a drug or device. They’re well on the ball in the the fight against big pharma. But their skepticism seems to be on a sabbatical when they report epidemiological studies, less rigorous than studies of drugs and devices, which highlight harms and risks. News outlets do less well against unwarranted risk pessimism than unwarranted therapeutic optimism. If the media constantly appeases our id, they could create a society in perpetual fear of anything and everything.

Selective skepticism is not skepticism but bias, and a bias more insidious than explicit bias because it’s hidden under a fog of pseudo-objectivity. Non-selective skepticism is hard and, as I’ll attest, doesn’t win many dinner party invitations. Nuance is hard work, but I refuse to believe that our journalists, as endowed with rhetorical flourish and wordsmithery as they are, lack the words to construct an interesting narrative conveying nuance.

An accurate headline goes a long way toward achieving the subtlety I wish to see more of.

Saurabh Jha is a radiologist and can be reached on Twitter @RogueRad.  This article originally appeared in Health News Review.

Image credit: Shutterstock.com