Let me tell you about this fabulous new test – it’s far better than the old tests, I’ve got a decade’s worth of research publications to show you, plus it’s a floor wax and a dessert topping.
Who am I? Why, I’m simply an academic clinician, advocating for this test because it’s the next great thing to help us tailor treatments to patients. Does it matter that I’m on the unofficial payroll of the manufacturer of the test equipment?
That is the question, sadly, physicians and policy-makers are asked to make with recurring regularity when trying to translate clinical research to patient care. To make these choices, we rely on layers of experts and statisticians to do the heavy lifting in the review process. This starts at the initial process of institutional review, filters up through pilot studies, then small publications, and finally, systematic reviews and publications in the most prominent medical journals. By the time the state of the art reaches these most prominent journals, it ought to be possible that we can put unfailing faith in its veracity.
This is decidedly not the case. A couple weeks ago the Journal of the American Medical Association published a “Clinical Evidence Synopsis” covering the use of procalcitonin measurements to promote antibiotic stewardship. This synopsis strongly endorses the use of procalcitonin assays to guide antibiotic prescribing in pneumonia and summarizes 14 positive studies in support of its recommendations. I hardly count myself an expert in this area, but I recalled at the minimum a couple recent studies showing procalcitonin-guided strategies were ineffective or harmful.
To cut a long story short, the JAMA article is written by three physicians who have financial conflicts of interest with the manufacturer of the procalcitonin assay. The evidence they summarize is a Cochrane Review that counts themselves and a total of eight authors who declare COI with the manufacturer. On top of that, 11 of the 14 positive studies included in the Cochrane Review are published by authors who declare COI with the manufacturer. I reviewed each of these articles, and they count amongst themselves trials performed in heterogenous settings with small sample sizes (as few as 8!), high risks of bias, and significant limitations.
The filtering of medical evidence has clearly failed on multiple levels to reach this point. Practicing clinicians have neither the free time to comb through the original articles, nor likely the institutional access that allows them to even download them without incurring substantial costs. They are dependent on prominent journals to be vigilant against distortion in a fashion most would find excessive. JAMA, with a qualified circulation in excess of 300,000 and one of the highest impact factors in medicine, should be one of those gatekeepers.
We need to expect and demand better of these respected institutions to better reflect their mission “to promote the science and art of medicine and the betterment of the public health.”
Ryan Radecki is an emergency physician and AHRQ Patient Safety and Quality Fellow. He blogs at Emergency Medicine Literature of Note and can be found on Twitter @emlitofnote.
