In a widely circulated CNN article, many radiologists have been found to cheat on their board exams: “Doctors around the country taking an exam to become board certified in radiology have cheated by memorizing test questions, creating sophisticated banks of what are known as ‘recalls,’ … The recall exams are meticulously compiled by radiology residents, who write down the questions after taking the test, in radiology programs around the country, including some of the most prestigious programs in the U.S.”
The American Board of Internal Medicine faced a similar issue recently, as doctors got into trouble for using a board preparation course that used “recalled” test questions.
What’s happening with the radiologists, however, seems more egregious. Apparently, these large banks of questions were considered “prized possessions” of radiology residencies. In some cases, residents were encouraged by their program directors to use the illicit resource:
Webb, 31, said he failed the first radiology written exam, which focuses on physics, in the fall of 2008. He said the program director at the time, Dr. Liem Mansfield, told him to use the recalls in order to pass.
“He told me that if you want to pass the ABR physics exam, you absolutely have to use the recalls,” Webb said. “And I told him, ‘Sir I believe that is cheating. I don’t believe in that. I can do it on my own.’ He then went on to tell me, ‘you have to use the recalls,’ almost as if it was a direct order from a superior officer in the military.”
Thankfully, this appears to be largely confined to radiology. I’m not aware of such a bank of questions in internal medicine, or any other specialty.
The American Board of Radiology, however, may have unwittingly fostered this underground culture of recalled test questions. Apparently, half of the questions on their certification exam were the same from year to year. Is it any wonder why, then, old test questions were treated like gold?
Do other Boards use such a high proportion of repeat test questions? I’m not sure, but 50% sure sounds high.
I understand that composing high stakes board certification test questions is an exhaustive process, but if the Board really wanted to stamp out the culture of recall, simply ensuring that the board questions were all new would have been the easiest way to do so.
Had they taken that pro-active approach, the Board may have spared the entire field of radiology this black eye.
Shortly after this post was published, I received an email from Donna Breckenridge, Director of Communications and Editorial Services, The American Board of Radiology. With the Board’s permission, her email is printed below:
Dr. Pho – We saw your comments on your blog today and wanted to call your attention to a “fact” that you apparently picked up from CNN but that is wrong. Despite our discussing this with CNN multiple times, they reported that half of the questions on our exams are the same from year to year. In fact, the opposite is true and every year’s test is different. Half of the questions are brand new. The other half are drawn from a question bank of thousands of previously used questions. These questions are never the same as the ones used on the most recent exam, and they can be any combination of those thousands of previously vetted questions.
Drawing from the bank of previously used questions allows for the required statistical reliability to validate each examination. Without reused questions, the difficulty level of each new exam could vary in unknown ways from year to year. Additionally, an unknown number of the completely new test questions may perform poorly on the new examination, necessitating their removal from scoring. This could result in an unbalanced, nonequivalent examination. For these reasons, best testing practices typically incorporate the re-use of questions that are known to perform well.
I’d encourage you to look for further information on the ABR’s Exam Security Policy, and on the ABR’s exam creation process. Please see the Announcements section of the ABR’s website and “The Truth about the ABR’s Examinations.”
Kevin Pho is an internal medicine physician and on the Board of Contributors at USA Today. He is founder and editor of KevinMD.com, also on Facebook, Twitter, Google+, and LinkedIn.
