One of the most difficult decisions that patients, families and physicians face involves end-of-life care. The advance directive or “living will” has become an accepted framework for patients to delineate their own preferences about what treatment they would or would not want when faced with a life-threatening disorder. But it was not always this way.

In the past, physicians and families often shielded those with potentially fatal illnesses from candid conversations about dying. The doctor or a family member would make decisions to sustain or stop treatment, typically without consulting the patient. This has changed over the past three decades following a landmark report entitled “Deciding to Forgo Life-Sustaining Treatment” issued by a presidential commission in 1983.

Advance directives have become increasingly used to guide patients and family members. The underlying assumption is that a great deal of the stress and complexities of making decisions about therapy will be solved if the patient specifies his or her preferences in advance. But considerable research has highlighted that choices about treatment frequently change, and advance directives often fail to accurately forecast what a patient will want when actually experiencing a severe illness.

Consider the case of a 64-year-old woman diagnosed with cholangiocarcinoma. The cancer could not be fully resected. When she was informed of the extent of the tumor and the poor prognosis, she told her family that she was ready to die. “I’ve had a great life,” she affirmed. But her family prevailed upon her to undergo chemotherapy, and for eight years, the tumor was quiescent.

This woman had planned every detail of her funeral and had an advance directive that specified that should the cancer grow and her condition deteriorate, she did not want “heroic measures.” Her daughter recounted that her mother had said that “She was ready to die when her time came and that she wanted to die at home with dignity.”

After eight years of good health, the patient developed multiple hepatic metastases and liver abscesses. She required percutaneous drainage and hospitalization for intravenous antibiotics, and the metastatic lesions progressed. She became severely fatigued, spending the entirety of her day in bed. An avid reader all her life, she could hardly read more than a few pages before drifting off to sleep. Her condition continued to deteriorate.

Yet when asked, the patient insisted, “I want to keep trying. I want to fight.” The patient’s daughter told us that the family was “shocked and confused” by these sentiments. They all expected that she would reiterate her earlier wishes and forgo further treatment. Instead, the patient became determined to try other therapies. This was not due to medication or confusion; she was lucid when expressing her desire to undergo as much treatment as necessary to keep her alive.

This change in preferences around end-of-life care is not unusual. A study led by Terri Fried, MD, of Yale University, an expert in end-of-life decision making, illustrated how preferences can change. One hundred eighty-nine patients were studied over a two-year period; these patients had diagnoses typically seen at the end of life, including congestive heart failure, cancer and chronic obstructive lung disease. Although many of the patients had been hospitalized in the previous year, including some in the intensive care unit, most rated their current quality of life as good.

The study involved repeated patient interviews about their wishes to undergo specific medical interventions, such as intubation and a ventilator, and their choices about undergoing treatment that would prevent death but might, or might not, leave them bedridden or with significant cognitive limitations.

The researchers found that nearly half of the patients were inconsistent in their wishes about such treatments. Although more people whose health deteriorated over the two-year study period showed such shifts in preferences, even those whose health was stable changed their minds. Having an advance directive had no effect on whether a patient maintained or shifted his or her initial preferences about therapies.

This is one of several studies that led researchers like Dr. Fried and her colleague, Rebecca Sudore, MD, of the University of California, San Francisco to conclude that advance directives “frequently do not … improve clinician and surrogate knowledge of patient preferences.”

Muriel Gillick, MD, a geriatrician at Harvard Medical School and a researcher in end-of-life care, similarly wrote that, “Despite the prodigious effort devoted to designing, legislating, and studying of advance directives, the consensus of medical ethicists, researchers in health care services, and palliative care physicians is that the directives have been a resounding failure.”

Why do patients often deviate from their advance directives? They do so because they cannot accurately imagine what they will want and how much they can endure in a condition they have not experienced.

Our patient with cholangiocarcinoma originally set out her wishes in her advance directive, believing that life would not be worth living if she were bedridden. When she became ill, her family, being healthy, viewed her quality of life as so poor that it did not seem worth pursuing continued treatments. But the patient found that she could still take great pleasure in even minor aspects of living, enjoying the love and attention of her family.

Cognitive scientists use the term “focalism” to refer to a narrow focus on what will change in one’s life while ignoring how much will stay the same and still can be enjoyed. Another insight from cognitive psychology that is relevant to the changes in preferences for many patients is “buffering.” People generally fail to recognize the degree to which their capacity to cope will buffer them from emotional suffering. The often unconscious processes of denial, rationalization, humor, intellectualization and compartmentalization are all coping mechanisms that patients employ to make their lives endurable, indeed, even fulfilling, when ill.

Another limitation of an advance directive is that it cannot encompass every possible clinical scenario that may arise. For example, a patient is newly diagnosed with an incurable lung cancer with a life expectancy of two years or more. The patient states in his advance directive that he does not wish to be placed on a ventilator. Soon after initiation of treatment, the patient develops pneumonia, and intubation with ventilation for a few days is needed for support as the antibiotic therapy takes effect. Should this patient forgo being placed on a ventilator?

Over the past two decades, there have been attempts to refine the advance directive by having the patient specify at the time of hospital admission the types of treatments that are acceptable: full CPR or not, intravenous fluids, comfort measures like oxygen and pain medications. Physicians then write orders in the patient chart about each of these interventions.

While this refinement may be helpful, researchers in end-of-life care emphasize that there are no shortcuts around emotionally charged and time-consuming conversations that involve patients, families and physicians.

Even with detailed initial instructions, patients may change their minds. Repeated communication can help bring clarity to these difficult decisions. We believe an advance directive is an important beginning, but not the end, of understanding a patient’s wishes when confronting severe illness.

Jerome Groopman, a hematologist-oncologist and endocrinologist, and Pamela Hartzband are staff physicians at Boston’s Beth Israel Deaconess Medical Center. They are authors of Your Medical Mind: How to Decide What Is Right for You. This article was originally published in ACP Internist.