Andrews was easily the most anxious patient I took care of that month, a gray Michigan February (is there any other kind?) which I spent in the hospital caring for patients admitted to the general medical ward at the Ann Arbor Veterans Affairs Medical Center. (Andrews is a pseudonym, as are all the patients I blog about, unless otherwise indicated.)  He had plenty to be anxious about, too.   His leukemia was raging out of control, his blood looking like pus, teeming as it was with malignant white blood cells.  At his age—he was almost 60—and after a decade of chronic bone marrow cancer, his disease was especially dangerous.  Odds were high he would survive for less than a year.

Unless the genetics of his cancer were favorable, indicating a good likelihood that he would respond to chemotherapy.  So Andrews and I (and the rest of my general medicine team) waited to hear back from the oncologists about the result of his genetic studies.

Andrews wasn’t afraid of dying, because he’d already had a first-hand view of death at its worst. Twenty years earlier, he was working as a card dealer in Vegas and had fallen in love with another dealer.  In the open-minded culture of that city, he had been able to have an un-closeted gay relationship without being judged.  He had been truly, ecstatically happy.  Nights at the tables; days spent with accepting friends.  And his lover, Charles: simply the best friend he had ever had.  “I’d have even loved him,” Andrews told me tearily, “if he’d had a woman’s body.”

But then Charles contracted AIDS, at a time when that disease was almost uniformly fatal.  Andrews stayed by his side, caring for Charles, administering to his growing list of bodily needs. He had a ring-side seat, in other words, while his lover wasted away. Charles was only 95 pounds at his death.  No bout of leukemia could compare to the suffering Andrews witnessed at his lover’s bedside. Charles died that year, and so too did the best part of Andrews’ life. He would never fall in love again.  Broken up by the loss of his soul mate, he moved back to rural Michigan where people weren’t so comfortable with his lifestyle. So he became withdrawn.  Life didn’t mean that much to him anymore: “Don’t get me wrong,” he said to me. “I’m not suicidal.  I don’t want to die.  It’s just that I’m not afraid to die.”

But would he be dying soon? That would depend on the test results, which would be coming any day now.

And then that day arrived. The students and I met the oncology team in the corridor outside Andrews’ room.  The news wasn’t good, the lead oncologist told me—his tumor didn’t have bad genes, it had horrible ones: “5% of people with this genetic profile,” she told us “respond to chemotherapy, and go into remission.”  I felt my shoulders sinking.  We opened the door to Andrews’ room and the oncologist took the lead in discussing treatment options.

She explained what the treatment would entail.  She compassionately broke the bad news that the genetic tests did not come out well.

Andrews remained calm, and asked how many rounds of chemotherapy he’d have to go through.  “It depends on how quickly and thoroughly you respond to treatment” she replied.  He wondered what the odds were of that kind of response. “That is hard to say,” the oncologist responded; “the first round of treatment will give us a much better picture.”

He told her he knew she couldn’t predict the future, but he still wanted to know the odds that he would lick this thing.  She paused.  These questions are always difficult to answer. And precise numbers?  Easy to give a medical colleague a 5% figure in the hallway, but here at the bedside that’s much harder.  So she took a breath, looked him in the eye and said: “20%, Mr. Andrews.  We can hope for a 20% chance of remission.”

“Well that’s a fighting chance,” he said, a new light in his eyes.  “Let’s start treatment.”

Twenty percent?!  I was stunned.  Andrews’ prognosis had increased four-fold in the three minutes since our hallway conversation.  How could this wonderful oncologist, someone who I had learned to respect for her compassion and thoughtfulness, have spoken such a blatant falsehood to my nervous patient?

I think she simply panicked.  In her left brain, the mathy part of her cerebral cortex, lay the calculations that caused her to derive that 5% estimate.  Meanwhile, her right brain, the emotional center of her neurologic system, fought back.  This man was young looking for his age, she probably though to herself.  He was such a nervous Nellie, too. That number, delivered under such circumstances, would be negligent in its cruelty. (I write about the long history of hiding cancer diagnoses in a previous post.) Her brain, I imagine, quickly recalculated his odds in that brief interval of panic, when she realized he was insisting on getting a number out of her.

In July of 1982, Steven J. Gould, the famed Harvard paleontologist, was diagnosed with abdominal mesothelioma, a rare cancer that carries a dismal prognosis, so dismal that his doctor (when Gould asked him for suggested readings on the diagnosis) told him to stay away from the library.  Gould, of course, ignored this advice and soon discovered that the median survival for patients with this diagnosis was a mere 8 months.

But Gould would not allow himself to be discouraged by the statistics.  As a scientist, he knew that the median was merely a statistical measure, signifying that half the patients lived this long and half did not.  So which half would he be in?  From his reading, he knew.

He was younger and healthier than the typical mesothelioma patient, so right there he was confident: he would live more than 8 months.

His mathematical reasoning now in high gear, no doubt nudged along by very motivated reasoning elsewhere in his brain, Gould soon convinced himself he would be a long-term survivor, a conclusion that would prove to be accurate.  He died 20 years later of an unrelated cancer. (For his version of this story, check out this link.)

I expect that the oncologist that day, when pressed by her nervous patient, did a Gouldian recalculation of her own. Maybe she convinced herself that Andrews was younger or healthier than the average leukemia patient. Or that the genetic test, somehow, wasn’t as predictive in his case as in others. Doctors, it turns out, are prey to all kinds of unrealistically optimistic impulses.

The oncologist’s behavior that day, the sudden switch from 5% to 20%, is a common phenomenon shaping communication between doctors and patients near the end of life. Indeed, I have exhibited this same behavior.  I have let my hope and optimism, my “hopetimism?”, interfere with accurate communication.

In his book, Death Foretold: Prophecy and Prognosis in Medical Care, Nicholas Christakis illuminates the many forces leading to this behavior. Lots of sociology and psychology influence this behavior.  But if I had to summarize the main problem here, I’d say it comes down to this: sometimes, doctors simply find it too painful to tell the truth.

Peter Ubel is a physician and behavioral scientist who blogs at his self-titled site, Peter Ubel and can be reached on Twitter @PeterUbel.  He is the author of Critical Decisions: How You and Your Doctor Can Make the Right Medical Choices Together.