When the patient safety field began a decade ago with the publication of the IOM report on medical errors, one of its first thrusts was to import lessons from “safer” industries, particularly aviation. Most of these lessons – a focus on bad systems more than bad people, the importance of teamwork, the use of checklists, the value of simulation training – have served us well.
But one lesson from aviation has proved to be wrong, and we are continuing to suffer from this medical error. It was an unquestioning embrace of using incident reporting (IR) systems to learn about mistakes and near misses.
The Aviation Safety Reporting System, by all accounts, has been central to commercial aviation’s remarkable safety record. Near misses and unsafe conditions are reported (unlike healthcare, aviation doesn’t need a reporting system for “hits” – they appear on CNN). The reports go to an independent agency (run by NASA, as it happens), which analyses the cases looking for trends. When it finds them, it disseminates the information through widely read newsletters and websites; when it discovers a showstopper, ASRS personnel inform the FAA, which has the power to ground a whole fleet if necessary. Each year, the ASRS receives about 40,000 reports from the entire U.S. commercial aviation system.
In the early years of the patient safety movement, the successes of the ASRS led us to admonish hospital staff to “report everything – errors, near misses, everything!” Many caregivers listened to these admonitions (particularly nurses; few docs submit IRs, which leads IR systems to paint incomplete pictures of the breadth of hospital hazards) and reporting took off. At my hospital (UCSF Medical Center), for example, we now receive about 20,000 reports a year.
Yes, 20,000 reports – fully half of what the ASRS receives for the entire nation! And believe me, we don’t report everything. If we really did, I’d estimate that my one hospital would receive at least five times as many IRs: 100,000 yearly reports.
But even at 20,000, recall that we are only one hospital among 6,000 in the United States. Since we’re a relatively large hospital, let’s say the average hospital only collects one-quarter as many IRs as UCSF, 5,000/year. That would amount to 30 million reports a year in the United States! (Oh yeah, and then there are SNFs, nursing homes, and all of ambulatory care, but let’s leave them out for now.)
Is this a problem? Yep-per, it is. First of all, IRs are all-but-useless in determining the actual frequency of errors, though they’re often used for this purpose. When I visit hospitals to talk about patient safety, they often show me their IR reporting trends. If the number of IRs has gone up over the past year, they breathlessly proclaim, “This is great. We’ve succeeded in creating a reporting culture – the front line personnel believe that we take errors seriously. We’re getting safer!”
That would sound more credible if hospitals with downward trends didn’t invariably shout, “This is great, we have fewer errors! Our efforts are paying off!”
The point is that we have no idea which one is true – IRs provide no useful information about the true frequency of errors in an institution.
But that isn’t their major flaw. The bigger problem is that IRs waste huge amounts of time and energy that could better be used elsewhere in patient safety (or in patient care, for that matter). Let’s return to my hospital for a moment (and let me apologize to those who thought there would be no math). I’d estimate that input time for the average IR is about 20 minutes (the system requires the reporter to log in, and then prompts her to describe the incident, the level of harm, the location, the involved personnel….).
Once an IR has been submitted, it is read by several people, including “category managers” such as individuals in charge of analyzing falls or medication errors; the charge nurse and the doctor on the relevant floor; and often a risk manager, the patient safety officer, and more. These individuals often post comments about the case to our computerized IR system, and some IRs generate additional fact finding and analyses. I’d estimate that this back-end work comes to about 60 minutes per IR.
In other words, each of our IRs probably generates an average of 80 minutes of work: 20 minutes of reporting and 60 minutes of reading/analysis. For our 20,000 IRs per year, that’s 26,667 hours of work. (Of course, we could shave this number by doing nothing with the submitted IRs – a recent study found that this is precisely what happens in about one-in-four U.S. hospitals, which don’t even bother to distribute IRs to hospital leaders or managers. Sounds like something out of Catch-22 or The Office).
If we value the time of our people doing the work of reporting, reading, analyzing, and acting on IRs (an amalgam of nurses, quality and risk managers, and a few physicians) at an average of $60/hour (salary and benefits), we’re talking about a yearly investment of $1.6 million in my one hospital. Nationally, for 30 million reports, the cost (of 40 million hours of work) would be $2.4 billion! Now we’re talking about real money.
Even that expenditure (which is 50 times more than AHRQ spends on patient safety research yearly) wouldn’t be so horrible if this work was yielding useful insights, but, for the most part, it’s not. My colleague Kaveh Shojania recently wrote a terrific piece entitled “The Frustrating Case of Incident-Reporting Systems,” in which he argued that, while all events should be reported…:
Many incidents, even if important (e.g., common adverse drug events, patient falls, decubiti) do not warrant investigation as isolated incidents. In such cases, the IR system should simply capture the incident and the extent of injury to the patient, not barrage users were a series of root cause analysis-style questions about the factors contributing to these events.
This is a great idea but I’d go one step further, to a system I’ll call, “If It’s February, It Must Be Falls.” Here’s how it would work:
I’d limit complete, year-round IR reporting to only those errors that cause temporary (33% of all IRs in one large study) or serious (1.5%) harm, along with a small number of reporting categories, such as the disruptive provider, that require complete data. For the remainder of the categories, I’d switch to a monthly schedule: all medication errors get reported in January, all falls in February, all serious decubitus ulcers in March, and so on…
I’d estimate that this change would cut the number, and cost, of IRs by at least 50%, while having virtually no detrimental impact on the value derived from the systems. Risk managers would still hear about the worst errors, sentinel events would come to light to generate root cause analyses, and a month of complete data for each of the error categories would easily provide sufficient information to explicate more subtle problems. More importantly, caregivers, freed from the “report everything” mantra, would be more enthusiastic about reporting, and hospital leaders and administrators would have the time to analyze the reports and develop meaningful action plans (as well as to focus on other methods of error detection such as Executive Walk Rounds and trigger tools). As Kaveh wrote:
…organizations must recognize that the generation of periodic reports from IR systems does not constitute an end in itself. IR systems must stimulate improvement. Achieving this crucial goal requires collection of data in such a way that important signals are not lost amidst the noise of more mundane occurrences and so that hospital administrators do not experience information overload. If submitting incident reports produces no apparent response from hospital administrators, front-line personnel will predictably lose interest in doing so. In addition to undermining effort to monitor for safety problems, lack of meaningful change will negatively impact the culture of the organization in general.
I couldn’t agree more. Our unquestioning support for “report everything” incident reporting systems has created a bureaucratic, data-churning, enthusiasm-sucking, money-eating monster. It is past time we slayed it. Is anybody with me on this?
Bob Wachter is chair, American Board of Internal Medicine and professor of medicine, University of California, San Francisco. He coined the term “hospitalist” and is one of the nation’s leading experts in health care quality and patient safety. He is author of Understanding Patient Safety, Second Edition, and blogs at Wachter’s World, where this post originally appeared.
