Doctors know high-pressure exams. The day before one is the worst. There is cramming followed by anxiety and insomnia. When sleep finally beats anxiety, the dreaded nightmare falls upon anxious test takers. Every doctor knows. Walking into the testing center, opening the exam, realizing you studied for the wrong exam. The questions might as well be a foreign language. This nightmare was a reality according to this year’s group of radiation oncology residents taking their physics and radiation biology board exams.
The exam is administered annually for radiation oncology residents. It is a relic from the bygone era. A multi-year long journey defined by a game of trivial pursuit called a board exam. It is a locked-down testing environment as opposite to clinical medical practice as possible. There is no internet, no phone, no colleagues, no tumor board discussion. It is an exam that exists in the exact same format as when it was created. Despite the vast changes in medical care, technology, and physician-colleague collaboration, the methodology to determine if a physician is competent remains unchanged.
Tradition is more important than practicality. So, every fourth-year radiation oncology resident studies dutifully for posterity’s sake. The founders of the field stay happy, the American Board of Radiology collects their testing fees, and residents happily move onto their next board exam. Questioning the exam is frowned upon.
More recently, some well respected academic radiation oncologists questioned the utility of this exam publicly. They noticed a significant amount of resident time spent preparing for an exam that has questionable practical applications. There is a significant opportunity cost. Time spent on biology trivia memorization is time not spent learning how to treat patients.
So, residents temporarily memorize that enzyme GDTK4373BY catalyzes the 18th step in the pathogenesis of small cell lung cancer (made up enzyme) until two hours after the exam when the information is forgotten. They may not know how to treat a patient that comes to their clinic with small cell lung cancer. This style of testing gives the impression that trivia memorization is preferable to safely treating patients. Being a competent physician is extraneous and untested on this exam.
The American Board of Radiology was not happy with this critique. A response was concocted by Paul Wallner and Lisa Kachnic, radiation oncologists intimately involved with and paid by the American Board of Radiology. In their response, they said, “regardless of a belief within the radiation oncology community, trends in the quality of residents accepted for training have been drifting slightly downward.” In other words, cancer patients need us to police because residents are less qualified. They cited Wikipedia as a primary source in their commentary. They were criticized heavily within the radiation oncology community. All objective measures show the quality of radiation oncology residents has increased significantly. Most of that increase coming within the last decade.
Radiation oncology residents have consistently shown great ability to perform well on standardized tests during medical school and residency. Radiation oncologists have asked the American Board of Radiology to test relevant content that is a better gauge of the minimum qualifications to practice radiation oncology. They have been unwilling to consider updating their testing methodologies to more accurately test competence in today’s health care environment.
Fast forward four months from the criticized response — and a doctor’s worst nightmare is a reality. Residents walked into an exam with what they describe as irrelevant trivia questions. The way that residents had studied for decades and the way the material was taught was inadequate for this year’s exam. Many noted it was an exam on translational cancer biology, not the radiation biology exam of the past. Nobody, including the instructors for residents, was notified of the shift. Residents complained after the exam about being expected to know molecular biology minutiae but most chalked it up to recall bias — only memorizing the few questions that were difficult.
Everyone assumed only a small percentage would fail as has been the case for decades. Then the scores returned. A significant number of deflated residents posted their failing scores. The failure rate seemed anecdotally high at first glance. As more and more residents posted about failing angst turned to anger. Failure rates began to leak. They were 3+ standard deviations from the historical average.
The American Board of Radiology confirmed the high failure rates for each of the two sections. They did not release the overall pass rate for the entire exam. They claimed the failure rate was higher this year for factors outside of the American Board of Radiology’s control. They even suggested that poor teaching led to the failure rate. Poor teaching that was 3+ standard deviations worse than one year prior. The residents are not so trusting of this statement. Some even speculate that their criticized claim that resident quality is declining was foreshadowing an unfair exam that the authors of the paper Paul Wallner and Lisa Kachnic supervised and approved. Poor scores could help prove their mocked theory. In other words, do not challenge the exam makers on whether their exam has a purpose. The American Board of Radiology would most certainly deny this accusation.
Now, many residents are left with a failed board exam on their record. They must retake the exam next year. They will pay additional hundreds of dollars. They have no idea which resources to use since the exam changed without anyone’s knowledge. They must spend another year temporarily memorizing trivia instead of mastering how to treat patients. They will sacrifice time with family, wellness activities and other mental breaks so they can temporarily memorize that protein VAXCF836273 is different from protein VAXCF2834510. They may not have a chance to solidify their clinical radiation oncology knowledge prior to entering practice but this is the game that residents must play. Trivia trumps competence.
The authors of this article are anonymous radiation oncology residents.
Image credit: Shutterstock.com
