Ten years ago, my tennis partner suffered a stroke.  He was a sixty-year-old at the time, working to move up into the top ten players in his age group.  In the country!  You could not have found a healthier sixty-year-old.  He played tennis three-plus hours a day, scampering across the court like a hyperactive adolescent.  In other words, he was not the kind of cigarette smoking, hypertensive elderly man you would expect to experience a stroke from clogged and battered arteries.

I wasn’t shocked to learn, then, that he had a PFO — a patent foramen ovale.  The foramen ovale is a passageway connecting the right and left atrium in the heart, a crucial opening for all of us when we were fetuses because it allowed our bodies to shunt blood from the right and left side of the heart without having to pass through what, in those circumstances, were non-functional lungs.  For most of us however, the first time we took a breath of air after being born, our foramen ovales began to close, the two sides of the wall separating the atria eventually fusing.  But in the case of my tennis partner, that FO remained a PFO.  He lived for sixty years unaware that there was a hole still open between the two sides of his heart.  Eventually that PFO contributed to his stroke, providing an opening for a blood clot to sneak across to the left side of his heart where it then was pushed, with the contraction of his left ventricle, up towards his brain.

My tennis partner survived the stroke and was soon playing tennis again.  Determined not to experience another stroke, he chose to have his doctors close his PFO.

But should he have made that choice?  And should his insurance company have paid for the procedure?

Based on the results of two randomized trials recently published in the New England Journal of Medicine, I would have to answer the first question with a strong “maybe” and the second with a definite “no.”  As I will explain below, this difficult decision has been made all the more difficult by the willingness of healthcare providers and payers to offer these treatments to patients outside of clinical trials, meaning that ten years after my buddy got that expensive procedure, we still don’t know whether he made the right decision.

When medical interventions are unproven, we should consider offering them only to patients willing to enter randomized clinical trials.

Let’s look at the evidence.  The first of these new studies was called the RESPECT trial.  The RESPECT investigators recruited patients less than sixty years old who had experienced stroke and for whom no other cause was identified other than a PFO.  The 980 patients enrolled in RESPECT were randomized to either receive medical therapy (with aspirin, or Coumadin, or some other “blood thinner”) or closure of their PFO with the help of a medical device called the “Amplatzer PFO Occluder.”  (Sounds a bit like the name my youngest son gave one of his Lego contraptions back in elementary school.)

Why 980 patients?  Because that number gave the investigators 80% “power” to figure out whether the occluder was good enough to reduce subsequent strokes, from the 4.3% of patients in the medical group they predicted would have strokes at two years down to only 1.05%.

They thought it would cut the stroke rate by more than 75%, in other words.  At the two year mark of the trial, however, only 3% of the medical group had experienced a new stroke (instead of that 4.3 figure).  And 1.6% of the occluder group experienced a stroke, meaning the risk wasn’t cut even in half.

Disappointed?  Well, perhaps that’s because the researchers raised our expectations too high.  Up front, wouldn’t you have been happy with a 50% reduction in strokes?  If you were a patient, wouldn’t you choose the occluder still, despite its benefits being less than predicted?

Not sure?  Then consider the second study, called the PC trial.  (No respectable clinical scientist leads a multi-site randomized trial without giving it a catchy acronym!)  Like RESPECT, the PC trial randomized patients to receive either medical treatment or the occluder.  Both trials, in fact, were funded by St. Jude, manufacturer of the occluder.  The PC trial accepted a wider range of patients—not just those who had strokes, for example, but also people with certain kinds of mini-strokes.  In this trial, the investigators enrolled 414 patients, a number large enough to find out whether the occluder could reduce stroke rates in these patients from 3% per year to 1% per year (which over a follow-up period of more than 4 years would amount to reducing stroke rates from 12% to 4%, pretty big difference).  As it turns out, the 4 year stroke rates were 5.2% and 3.4%, in the medical and occluder groups respectively.  Once again, not nearly as large as predicted.  But still — not 0.  And in both trials, the occluder was quite safe, with no increase in adverse events compared to medical treatment.

So I ask you once again: would you choose to be treated with the occluder based on this information?

As a fifty-year-old who plans to be a pretty darn good tennis player at age sixty, I would choose the occluder based on what I have seen.  But I am also convinced that I should not be able to choose the occluder — and I definitely should not expect insurance to pay for the treatment — unless I am willing to enter the kind of research trial that will better answer the question of whether the occluder actually benefits patients.  Because you see, both of those research trials reported nice trends, favoring the occluder over medical treatment, but neither trend reached what we researchers call “statistical significance.”  In other words, the difference in outcomes experienced by patients who receive medical treatment versus the occluder could very well be due to random chance rather than to the wonders of closing PFOs.  With both medical treatment groups doing better than expected, the trials were not large enough to give a precise estimate of the impact that the occluder has on people’s outcomes.  We need more data.

But more data will not be easy to come by, because most patients don’t want to enter a research trial when those trials leave them with a 50% chance of receiving a medical treatment — blood thinners — that does nothing to address the problems their doctors have just told them was the source of their stroke — that hole in their heart which failed to close on their birthday.  Indeed, the authors of the PC trial made a point of mentioning the difficulty they had recruiting patients to the study, and the authors of the RESPECT trial explained that some of the patients in the medical treatment group found a way to receive other treatments, outside of the trial, to close their PFOs.

We offer too many patients too many unproven treatments at too much expense.  We ought to make sure that promising but unproven treatments are not subsidized by taxpayers or insurance company enrollees, unless the patients receiving those treatments are enrolled in the kind of clinical trials that will better establish whether those treatments are safe, effective and affordable.

If I had a stroke from a PFO, I would be reluctant to enter a randomized trial.  My best guess is that the occluder offers a modest reduction in strokes.  So given a choice of receiving the occluder through my insurance company, or entering a research trial where I have a 50% chance of receiving the occluder, I certainly wouldn’t enter the research trial.

But all the rest of you — those paying into my health insurance company or, were I of Medicare age already, all you taxpayers — you should not have to subsidize my “best guess.”  You should force me to make a choice: pay for the occluder out-of-pocket (at thousands of dollars) or enter a randomized trial in which my healthcare costs will be covered by insurance.  Given that choice, I would enter the trial and we would be one patient closer to figuring out whether this treatment deserves to be part of standard healthcare coverage.

When it comes to paying for promising but expensive medical interventions, we should take a position of: prove it or lose it.

Peter Ubel is a physician and behavioral scientist who blogs at his self-titled site, Peter Ubel and can be reached on Twitter @PeterUbel.  He is the author of Critical Decisions: How You and Your Doctor Can Make the Right Medical Choices Together.