In 2021, at four endoscopy centers in Poland, endoscopists got a new tool to aid in colonoscopies. A computer equipped with an artificial intelligence watched the video alongside them and highlighted anything that looked like a polyp. The centers were part of a trial, so, as part of the study, the doctors had access to the tool on some days and not on others. This variability allowed researchers a chance to compare how the doctors did at detecting polyps on their own before and after the introduction of the tool.
They found that the doctors found fewer polyps when doing colonoscopies on their own after the introduction of the AI tool than before. Three months of working next to a machine that spots polyps had made these physicians a little worse at spotting polyps on their own.
The study is small and it is observational, and the authors say so. Something else could have changed in those centers over those six months. I have no trouble believing the result anyway, because aviation found the same thing years earlier and paid for it in a way that is harder to argue with.
On the night of June 1, 2009, Air France Flight 447 was cruising at 35,000 feet over the Atlantic, on its way from Rio de Janeiro to Paris, when ice crystals blocked the tubes that measure airspeed. The autopilot did what it is built to do when it loses a reading it needs: It quit. The pilots had to control the airplane. The airplane was flying fine. The blockage cleared within about a minute. All they had to do was hold it level until it did.
Real physician voices, twice a week
Free, and one click to unsubscribe.
The pilot flying pulled the nose up instead. He held it up as the stall warning sounded, and he kept holding it up as the airplane fell toward the ocean for three and a half minutes. The captain, who had been on his rest break, came back to the cockpit and could not work out what was happening in time. All 228 people aboard died. Two years later the “black box” was recovered, allowing investigators to understand the details of the mishap. Out of the resulting report, one thing stood out to me: The crew had not been trained to fly the airplane by hand at high altitude, or in the procedure for a bad airspeed reading. Why would they have been? At that altitude the autopilot flew. It always had. The skill was not lost so much as never built, and the men in the seats learned the difference at the worst possible moment.
Four years later, the Federal Aviation Administration (FAA) put out a two-page safety alert that says, in plainer words than regulators usually use, that constant use of automation can erode a pilot’s ability to recover the airplane when something goes wrong, and that airlines should have their pilots fly by hand on purpose, in good weather and light workloads, to keep the skill alive. Not because the automation is bad. Because the pilot must be there when it is not.
In August, I wrote here about doctors who read a machine’s fluent draft and let it quietly replace the judgment they had already formed. That was about what happens in the moment, with the machine in the room. The colonoscopy study is about what happens afterward, when the machine leaves. Those are different problems, and the second one is harder, because the person it happens to does not feel it happening. The Polish endoscopists did not report feeling rusty. The Air France crew did not know they had a gap until the autopilot handed them one.
Surgery has its own version. A few years ago, surgeons in Dublin asked senior surgical trainees about open gallbladder operations, the kind we fall back on when the camera approach goes wrong. Nearly all of them had never done one on their own, because the laparoscope now does almost all of that work. Most said they would be comfortable switching to one in the middle of a case anyway. The skill had never been built, and the confidence had arrived without it.
Aviation did not ban the autopilot, and I would not suggest medicine refuse the polyp detector. When it is running, it finds growths that would otherwise be missed, and that is the whole reason it exists. Aviation wrote down that the human skill needs to be maintained and made plans for the maintenance. Nobody in medicine has written that down. The detectors and the drafting tools are arriving, and the question of who keeps the doctor’s eyes and hands in practice while the machine works has not been assigned to anyone. The Polish study is the first measurement of that cost. I doubt it will be the last.
Colin G. Knight is a board-certified pediatric surgeon at HCA Florida Lawnwood Hospital in Fort Pierce, on Florida’s Treasure Coast. He is a clinical assistant professor of surgery at the Florida State University College of Medicine and at the Florida International University Herbert Wertheim College of Medicine.
Knight earned his undergraduate degree at Yale University and his medical degree at the University of Virginia, where he also completed his surgical internship. He then served four years on active duty in the United States Air Force as a flight surgeon, work that shaped his interest in operating-room safety and the subject of his essays here. He completed his general surgery residency at Allegheny General Hospital in Pittsburgh, including a research year in robotic pediatric surgery at Children’s Hospital of Michigan, where he later completed his pediatric surgery fellowship.
His research spans minimally invasive and robotic pediatric surgery as well as the management of pediatric appendicitis, with work appearing in the Journal of Pediatric Surgery, the Journal of Laparoendoscopic and Advanced Surgical Techniques, and Archives of Surgery. His practice website is ped-surg.com, and he shares updates on LinkedIn, Instagram, and X.
