Clinical medical research — finding out which treatments help and which ones don’t (or even make the situation worse) — is tough research to do. In the laboratory a scientist can control conditions so that only one thing is different between the control and the experimental groups. This isolates the effect of the particular thing and one can see if there is any difference in outcomes depending upon what is done with that thing. Clinical research is different because humans are complicated. The researcher tries to control the situation as much as possible, but ultimately she is comparing one dissimilar human to another one. Clinical research is also very expensive, and the costs grow as the number of study participants rises.
The result is that a lot of clinical studies, interventions in which researchers give patients this or that medicine and then try to find out if it worked, are underpowered. This means the studies aren’t powerful enough to answer the simple question: Does this treatment help? And if one can’t answer that key question the whole enterprise is more or less a waste of time.
Partly to solve this problem the concept of “meta-analysis” was devised. The idea is that one can take a bunch of underpowered studies and lump the information together. This can create, in effect, a single study with enough power to answer the question. Critics have compared meta-analysis to making a silk purse from a sow’s ear — trying to take a lot of poor studies and make a good study from them. This can be a problem, although investigators do their best to choose only studies with usable data. But this kind of analysis can yield very important information. It is also comparatively cheap research to do because one is essentially doing research on the research. If you’ve ever taken your child to the doctor for treatment of croup you and your child have been the beneficiaries of what meta-analysis can accomplish.
Croup is caused by swelling of the airway from a virus, and corticosteroid medicines reduce swelling. So it seemed logical to try them for croup. But although some of the early studies suggested steroids helped, they were all underpowered to answer the question for sure. Then somebody did a meta-analysis with the data and showed steroids probably helped. This information then led other researchers to spend the large amount of time and effort to do some fully-powered studies. The results? Steroids, by mouth, injection, or even inhaled, help relieve the symptoms of croup.
So in this case it was a silk purse all along.
Christopher Johnson is a pediatric intensive care physician and author of Keeping Your Kids Out of the Emergency Room: A Guide to Childhood Injuries and Illnesses, Your Critically Ill Child: Life and Death Choices Parents Must Face, How to Talk to Your Child’s Doctor: A Handbook for Parents, and How Your Child Heals: An Inside Look At Common Childhood Ailments. He blogs at his self-titled site, Christopher Johnson, MD.
