In January, when my close friend’s lymph node biopsy came back as a rare form of T-cell lymphoma, I scoured the scientific literature. What was his prognosis?He was 56, a little overweight but otherwise healthy. He had helped us move into our home more than a decade ago, and I was like an uncle to his son and daughter.

In the literature I found jagged graphs plunging downward, indicating that my friend’s expected survival time wasn’t long. Nearly 60 percent of similar patients died within six months.But the prognosis for any particular individual is not certain, his oncologist reminded me: It depends on age, functional ability, genetic markers and many other variables. Every patient is different. This is why it is hard for doctors to draw specific conclusions from generalized population data and frustrating for patients asking their single most important question: How long will I live?

In fact, several studies have suggested that when dealing with terminal illnesses, especially cancer, doctors are usually incorrect in their prognosis, nearly always tending to believe that their patients will live longer than they actually do.

In one study involving patients in Chicago hospice programs, doctors got the prognosis right only about 20 percent of the time, and 63 percent of the time overestimated their patients’ survival.

Interestingly, the longer the duration of the doctor-patient relationship, the less accurate was the prognosis. “Disinterested doctors . . . may give more accurate prognoses,” the authors wrote, “perhaps because they have less personal investment in the outcome.”

It’s hard to be frank

The study raised the question of whether this unwarranted optimism might have adversely affected the quality of care given to patients near the end of life, possibly leading to overly aggressive treatments.

“It was a very select group of patients,” said Harvard Medical School professor Jerome Groopman, a leading cancer and AIDS researcher, told me. (He was not involved in the study.) “These patients were going from hospital to hospice,” where life expectancy is less than six months. Even the most experienced oncologist cannot predict survival within two to three weeks, he said.

When families really press, Groopman said, he will provide a likely timeline. But he added, “You want to partner with them, not be the presiding judge handing down a death sentence.”

Communicating the prognosis to terminally ill patients themselves is an even greater challenge. In one study from 2008, nearly all the doctors surveyed said they told such patients that their condition would be fatal, but only 38 percent of them usually provided a time frame; 5 percent always did so.

Another study quizzed doctors treating terminal cancer patients on what they would tell those patients if asked for a prognosis; only 37 percent said they would give a “frank” estimate of length of survival.

‘I thought we had more time’

Like many doctors, I feel uncomfortable predicting a time frame for death. I often fear dashing the hopes of a patient who will need all his strength to battle the side effects of chemotherapy.

In my friend’s case, his oncologist told him that the cancer was very aggressive and a lot would depend on his response to chemotherapy. No time frame was discussed. We said that his cancer might go into remission and that there was the possibility of a bone marrow transplant. I don’t believe we gave him false hope, just hope grounded in reality.

In March, after a round of chemotherapy, my friend was shaking with a fever of 105 and was short of breath. While he waited for a bed in the intensive care unit, I tried to have a short conversation with the family. Like deer in the headlights, they listened to me.

“We needed to hear it from the oncologist,” his daughter told me later. “That is your human God when you have terminal cancer.”

Studies show that even though doctors tend to be optimistic, their personal observations are necessary for the most accurate prognosis. So we have to overcome our hesitation in giving bad news — and patients and their families need to be willing to hear what we’re saying.

In June, five months after my friend received his diagnosis, his family was planning a reunion for the Fourth of July weekend. But before they could get together, my friend’s condition deteriorated. He was readmitted to the hospital and had another round of aggressive chemotherapy. He died a week later.

“I thought we had more time,” his daughter told me.

“So did I,” I said.

Manoj Jain is an infectious disease physician and contributor to the Washington Post, where this article originally appeared.  He can be reached at his self-titled site, Dr. Manoj Jain.

Submit a guest post and be heard on social media’s leading physician voice.