The science we conduct often reflects the society we live in. One phenomenon of current society is the rise of the reality television show. Participants are promised instant media fame without having to struggle through acting school. I worry that this has rubbed off on some of the studies we now conduct. Sir Richard Doll and Austin Hill, the architects of modern epidemiology, realized that it was hard for epidemiology to prove or disprove anything. Their studies were driven by clear hypotheses as some took five to 10 years to complete. Furthermore, their landmark paper on smoking and lung cancer devoted over a page of discussion as to why the observed association of an odds ratio greater than 10 of lung cancer in smokers may not be causal.
This scientific version of the reality TV show is exemplified by some studies linking proton pump inhibitors (PPIs), commonly prescribed for gastroesophageal reflux disease (GERD), to a variety of diseases. PPIs have been associated with GI infections (including Clostridium difficile), pneumonia, bone fractures, pernicious anemia, interactions with clopidogrel, and heart disease. To this list we can now add chronic kidney disease and dementia. When a drug is associated with a long list of unrelated bad consequences it usually turns out that most, if not all, of these associations are not causal. This phenomenon was noted over 50 years ago but seems to have been forgotten by modern epidemiologists in the pursuit of the quick high-impact paper. Indeed, there are a number of properties you can look for in an epidemiological study to try and determine whether the association might be causal. The first and most important is the strength of the association.
An old-school epidemiologist would never wake up for an odds ratio (OR) of less than two (to paraphrase the supermodel Linda Evangelista). None of the many papers reporting on the risks of PPIs have reported an adjusted OR greater than two. I am not saying that these studies should not be published, but it is important that the authors emphasize that a strong association is more likely to be causal whereas a weak association is usually due to confounding factors. This is the most likely explanation for the myriad diseases that PPIs apparently “cause.”
Every study has shown that sicker patients tend to be prescribed PPIs. They go to the doctor more often and at some point they will be prescribed these drugs if they complain of upper GI symptoms. Sick patients tend to develop other illnesses, and so PPIs will be associated with about any disease you can imagine in a database. Patients on PPIs are also more likely to have diabetes mellitus and chronic obstructive pulmonary disease, and it is only a matter of time before you see papers reporting that PPIs “cause” these diseases as well. An indicator that this is likely to be the case is that the unadjusted OR becomes much more attenuated when adjusting for confounding factors.
For example, in the chronic kidney disease paper the OR equals 1.76 in the initial analysis but fell to 1.16 in a propensity-matched analysis in a replication cohort. This study could not identify all confounding factors, as it was not designed to test this specific hypothesis. It is very likely that if the authors could adjust for all known and unknown confounding factors, it would fall further and would no longer be statistically significant. This phenomenon is seen in nearly all papers of adverse events associated with PPI therapy, but it is not mentioned as the likely explanation for the observed association in any study. Dose response is another factor to look for and is rarely seen, although there are exceptions. Authors do focus on biological plausibility, but I find this criterion very unhelpful as you can make a “biologically plausible” explanation for anything in modern medicine.
Surely it is important to know the possible risks of any drug, even if the most likely explanation is confounding factors. The reason for my rather churlish comments above, however, is that I spend a good deal of time explaining these issues to patients every time one of these associations hits the press. The American Gastroenterological Association has helpful patient talking points on both the dementia and chronic kidney disease studies that you can take advantage of if you too are also facing these questions.
In one week, I counted 15 patients that I gave an explanation to (at their request), and each took five to 10 minutes. I calculated I could have seen seven extra patients if I had not had to explain what are likely to be spurious results. When you magnify this across North America, this is a significant burden on the GI health-care community that is of uncertain benefit. The only benefit this does have is that it is another opportunity to discuss with the patient about stopping their PPI therapy, as there are a significant proportion of patients that are on these drugs unnecessarily.
However, if the patient has significant reflux symptoms or is at a major risk of GI bleed, the benefits of these drugs clearly outweigh any risks. This is because even in the unlikely event these associations are causal, the impact would be very small. We calculated that you would need to treat more than 1,000 patients for one to develop a fracture that would not have occurred anyway.
We live in an instant gratification age. Sadly, this is sometimes reflected in the science that we produce, which makes it very difficult for the general clinician to make sense of the data. I must emphasize some of the papers in this space are well done and thoughtful, but many pay too little attention to why the associations they are reporting may not be causal and are too ready to jump to conclusions that are not supported by the data. Sir Richard Doll would have been saddened by how the discipline he informed has been so sensationalized. Then again, he probably would not have liked Keeping Up With the Kardashians.
Paul Moayyedi is a gastroenterologist. A version of this article originally appeared in AGA Perspectives, the opinion magazine of the American Gastroenterological Association.
Image credit: Shutterstock.com
