When diagnosed with abdominal mesothelioma, a rare cancer with a blighted future, evolutionary biologist and writer, Stephen Jay Gould, turned his attention to the statistics; specifically, the central tendency of survival with the tumor. The central tendency — mean (average), median and mode — project like skyscrapers in a populated city and are the summary statements of a statistical distribution.

The “average” is both meaningful and meaningless. The average utility of average is zero. Consider a gamble: a fair coin toss where you get $50 if it lands heads and lose $50 if it lands tails. The average (net) gains of this coin toss, if the coin is thrown hundreds of time, is zero.  But no one gets nothing: You either get $50 or lose $50. The average is twice wrong: It over estimates for some and underestimates for others. But the average of this gamble has important information. It helps you decide if you could profit from making people play this gamble; you wouldn’t profit unless you charged a small fee to play the gamble.

The median is the mid-point of a distribution. Gould’s cancer had a median survival of eight months. This meant that half (unlucky half) lived fewer than eight months and half (lucky half) lived more than eight months with the cancer. The mean is affected by outliers but the median is not. For example, consider Mumbai’s billionaires. They raise the average income of the city, not the median income. Skewness of a distribution affects the mean, not the median. Put another way, the median (Mumbai’s slums) conceals the skewness (Bollywood).

Gould, describing in his classic essay “The median is not the message,” ignored the median survival but looked at the skewness of the distribution. The distribution was right-sided: Some patients with mesothelioma who lucked out with survival lucked out big. Gould was initially despondent when he saw that the median survival of his cancer was so short. Gould was an optimist. He was dealt a rough hand but was not going down without a fight. His optimism, and fight, increased as he unraveled the distribution: first with the hope that he could be in the lucky half of the distribution, then with the hope that he could be one of the outliers in that skewed distribution, then with the hope that the treatment that he was being given, an experimental cocktail, could make him a lucky outlier.

Gould lived twenty years after his diagnosis, perhaps, in part, because of his optimism, although we don’t know for sure that optimism helped. Gould didn’t know for sure that he would be an outlier. He did not choose to be in the long positive tail: He hoped he was. He could, quite easily, have settled his affairs, written his will, and traveled the world believing he had only eight months to live. For every optimistic Gould who lives twenty years with mesothelioma, there may be ten optimistic Goulds who live only two months.

Gould’s story is at the heart of tension in evidence-based medicine (EBM). This is not a tension, strictly speaking, but an uprising. EBM is driven by central tendency — averages — amongst others. But averages are built by individuals who vary. Variation is a fact of life. All theory is gray, said Goethe. And gray, not black and white, is the only truth in medicine. Actually, variation is only half-truth; half remains concealed because whilst we know that we’re part of a variation we don’t know where exactly we’re placed, we don’t know which shade of gray we belong to.

Cancers vary in prognosis. Cancers vary in their response to treatment. This begs the question: In the absence of perfect information, what should the oncologist tell the patient? Should the oncologist reveal the median survival only? If so, why? What normative ethics say only the central tendencies of a distribution be disclosed? Should the oncologist give a whiff of hope that the patient could be an outlier? Should the oncologist mention the short left and not long right tail and stress the imminence of death so that the patient can die gracefully? What is the truth? Is it the median, the long tail of optimism or the short tail of pessimism? If all three are truths which truth should be mentioned first and which truth should be mentioned last?

The simplistic answer is that it depends on the patient. My friend, an interventional oncologist, tells me that patients seek him for hope. He gives them hope and is unapologetic about doing so. Some might say that he gives his patients false hope; but that accusation assumes a numerical probability of death, a threshold or a range, which neatly separates false from true hope, hype from reality. There is no such number and even if it existed it’d be near impossible to give every person their unique threshold of true vs. false hope, as the question will once again arise: What if I’m the lucky outlier?

When I pressed my friend further about false hope, he retorted. “I’m an oncologist, not an undertaker.”

Fair point. Many of his patients, with livers riddled with metastases and a hopeless prognosis, see the undertaker shortly after his interventions. Because hope is most in demand when reality is most hopeless. His patients see him for possibilities, not limitations. Like Gould, his patients wonder if there are outliers (there often are) in the survival distribution of their advanced cancer, a few extra months or even weeks, and if they could be that outlier.

My friend blasts his patients with chemotherapy, and if there are new agents, he tries them out as well. There are no short cuts with hope. When he suspects a complication of cancer, such as a clot in the lungs, he goes after it with hammer and tongs. Because there’s no retreating from hope.

Most patients who wish they are lucky outliers won’t be lucky outliers. The difference between hope and reality, therefore, is overtreatment. Hope and overtreatment are a dialectic: a marriage of convenience. Hope is a state of mind, a culture of expectation, will of the people, a belief in self-determination, and a rejection of the afterlife. Hope can’t be switched off by pressing a button. The most powerful driver of medical costs at the end of life is not the incentive structure. It is not doctors’ fear of being sued. The most powerful driver is hope.

Saurabh Jha is a radiologist and can be reached on Twitter @RogueRad.  This article originally appeared in the Health Care Blog.

Image credit: Shutterstock.com