A colleague once said, “Every plan is excellent, until it’s tested. It’s execution that’s the problem.” And so it is.

Clay Shirky wrote an excellent article about the gulf between planning and reality. Although the focus was on the misadventures of Healthcare.gov, the US government’s insurance exchange website, the broader lessons that he presents are worthy of consideration in many other settings.

Shirky notes: “The management question, when trying anything new, is ‘When does reality trump planning?'”

In the case of Healthcare.gov:

For the officials overseeing Healthcare.gov, the preferred answer was “Never.” Every time there was a chance to create some sort of public experimentation, or even just some clarity about its methods and goals, the imperative was to deny the opposition anything to criticize.

Failure is always an option. Engineers work as hard as they do because they understand the risk of failure. And for anything it might have meant in its screenplay version, here that sentiment means the opposite; the unnamed executives were saying “Addressing the possibility of failure is not an option.”

Project advocates enter every endeavor with a theory of the case, a vision of how things should be. But, as my late colleague Donald Schön noted, reflective practitioners are constantly reviewing the evidence to modify their framework in response to reality.

A comment on Shirky’s article summarizes this nicely:

“Any personal opinion you had given really doesn’t mean anything.” This is the key principle behind making anything work well — from writing an essay to building a bridge to creating a website. If it doesn’t work, throw out your preconceptions and re-conceive.

There is a cognitive basis for our failure to be reflective practitioners. We are all people of habit. The attributes that permitted us as cavemen to recognize the saber-toothed tiger the second time we saw it and to respond in the appropriate way (“Run!”) work well in the highly simplistic natural world. In a Darwinian sense, we evolved perfectly for that world. We developed a learning style that gave us a competitive evolutionary advantage, a learning style based on memory, stubbornness, and brute force.

But the more difficult world of complex organizations — overladen with political, organizational, and cultural forces and with technological challenges — presents an environment in which those cognitive attributes now present as cognitive errors. We struggle with this. Indeed, as MIT professor Rosalind Picard has outlined, successful learning has three phases: interest, distress, and pleasure.

We feel distress in the second phase because it is during that portion of the cycle that we must overcome our prejudices and develop a new framework within which to proceed. We resist. Sometimes we recognize that we have hit a plateau and need to adopt a new approach to proceed. Sometimes we don’t recognize that our framework is flawed and we uselessly proceed apace, until disaster occurs or a competitor outruns us.

Learning organizations and Lean philosophy

Places that are true learning organizations have built in a structure that calls the question early and often. One such structure (but not the only) is offered by the Lean philosophy. By encouraging front-line staff to call out problems they encounter in their daily life, managers are given real-time signals as to flaws in their organization’s processes. The leadership team then visits the sites of the flaws and invents experiments to achieve incremental improvements in work flow. Using the scientific method, those experiments are tested and evaluated, with redesign being a constant part of the process. Lean organizations understand that there is no group of central planners clever enough to design an optimum complex process. Lean leaders do not lack for a strong purpose — indeed audacious goals are favored — but neither do they lack humility.

Lean and other similarly designed organizations can only exist where the senior leadership is a strong advocate for the proposition that reflective practice is the best way to achieve outstanding performance for their customers. The leaders of such organizations embed that modesty and reflection in every aspect of their lives.

I’ve had the pleasure of visiting a number of hospitals that work along these lines. The results are palpable — better service to patients, higher quality, less waste, and more staff satisfaction. Such results are irrespective of the type of payment regime employed to compensate the doctors and the hospital. They are irrespective of the societal form of health care, be it a national public system or a dispersed private pay system.

Such hospitals remain anomalies in their industry, although the number is growing. Adoption tends to center in systems with a strong communitarian spirit, where the trustees and clinical and administrative leaders view their job mainly as providing a public service as opposed to supporting the personal and institutional prerogatives of physicians. Thus, while a few academic medical centers have gotten on board, many have not, trapped by age-old patterns of deference to the doctors. Ironically, in those academic medical centers that have adopted Lean or a similar approach, physicians report tremendous satisfaction from their engagement with process improvement and from the enhanced sense of teamwork with members of the staff throughout the hospital.

The young cadre of rising health care leaders I see when I address clinical and administrative training programs, and when I speak at conferences and in hospital settings, understand that the future is brightest for learning organizations. They thirst for experience in trying out these approaches, and they intend to lead in the manner of reflective practitioners. I say to current health care leaders, when you find one of these rising stars, grab him or her for your place. They are going to teach you something special.

Paul Levy is the former president and CEO, Beth Israel Deaconess Medical Center and blogs at Not Running a Hospital. He is the author of Goal Play!: Leadership Lessons from the Soccer Field and How a Blog Held Off the Most Powerful Union in America. This article originally appeared in athenahealth’s Health Care Leadership Forum.