A guest column by the American Board of Internal Medicine, exclusive to KevinMD.com.
KevinMD.com recently posted a blog noting that Watson, the IBM super computer who successfully beat two contestants on Jeopardy, would eventually make board certification exams obsolete. He argues that exams that rely on “memory-based curriculum” focused on recall of facts will become unnecessary. I agree. Recall of facts does not necessarily make one a good doctor, but recall of facts is not what Maintenance of Certification (MOC) exams are designed to test.
When Dr. Pho sits for his MOC exam, he will experience a psychometrically sophisticated exam developed by physicians in practice and academia to assess clinical judgment through mini-case simulations or patient vignettes. Research has shown that MOC exam scores are positively associated with delivery of quality patient care.
When I took the MOC exam in geriatrics, I was struck by how I needed to read each question very carefully, weighing clinical clues, and synthesizing the information to come up with the best answer. Sometimes, even if I was unsure of the generic name of a medication, the context of the question would make clear the right direction. Being able to look things up would not have helped if I didn’t have clinical experience with frail elderly patients, familiarity with the literature on risks and benefits, and the context of acute and long-term care.
Because of multiple benefits, the multiple-choice question is still the most commonly used question format across all major testing organizations in the U.S. They are highly efficient, and provide reliable scores since many questions can be administered to large groups in a reasonable amount of time. The large number of questions ensures broad coverage of content areas and increased rigor and reliability across test takers.
Are the exams perfect? Absolutely not. We are consistently looking for ways to improve the exam experience and make it more reflective of physicians’ real-world environment. For example, to reduce test length and testing time, ABIM is researching an alternative “adaptive” test design for MOC. This new testing model adapts to the physician’s ability level based on the physician’s responses to previous questions. Adaptive testing is expected to yield more efficient and precise measurement of individual proficiency in less time for most physicians. ABIM is also researching the effect of adding decision support to the exam based on what physicians use in daily practice.
As you prepare for your MOC exam, keep in mind the following:
- MOC exams assess a broad range of expertise in internal medicine or a particular subspecialty. Questions often cover diagnosis and treatment of common and rare conditions that have important consequences to patients.
- The questions are designed to assess what the certified internist is expected to know without access to medical resources or references, as opposed to knowledge that is appropriate – or even mandatory – to “look up.” One has to have enough knowledge and experience on your “hard drive” to make sense of decision supports.
- The level of difficulty for each testing point is targeted to the measurement goal of the examination, which is to discriminate between candidates who possess the cognitive expertise required for Certification from candidates who do not possess this expertise.
- Blueprints for each exam are available on www.abim.org
Former ABIM Board member Abraham Verghese recently observed that while Watson may be able to answer questions about facts, it will never be able to exercise the clinical judgment of a capable physician. I hope when Dr. Pho completes his exam he will find the experience more a test of his diagnostic acumen rather than mere recall of facts. We look forward to reading about it, either way.
Christine Cassel is President and CEO of the American Board of Internal Medicine.
Submit a guest post and be heard on social media’s leading physician voice.
