Radiologists on the new mammogram study. Who's right?

The BMJ recently published the latest results from the Canadian National Breast Screening Study (CNBSS). A brief summary of the CNBSS: Women were randomly assigned to annual mammography or breast exams and then the outcomes tracked. The results in the BMJ: mammography did not improve survival.

This is a very interesting study and when I first started working on this post I wanted to delve more into the science of this article and the growing body of literature that is shedding some doubt on the validity of mammography. Doing my due diligence I read the study several times, read the responses that have already appeared on the BMJ site, read some other studies, and read both the Canadian and the American College of Radiology responses. And that’s where I got stuck.

The Canadian radiologists basically said they agreed with the ACR who called the study “incredibly flawed and misleading” (which IMO are fighting words, it’s a bit like saying your study is a piece of garbage and should be ignored). Of course this piqued my interest.

The ACR claim rests on these 3 points:

The mammography equipment is old/bad and this was confirmed by independent experts. The lead author, Anthony Miller has refuted this claim in interviews with several Canadian news outlets and an expert from Dartmouth (Dr. Gilbert Welch) calls this study the most meticulously conducted and reported randomized trial on screening mammography. The ACR cites a paper in a radiology journal from 1990 that they say evaluated the equipment and Dr. B. Kopans, a Professor of radiology from Harvard, says he personally reviewed the equipment and found it lacking. How do you deal with these kinds of claims? Maybe ask the radiologists who read the films? It is an important point.
Only 32% of cancers were detected by mammography in the study. This is the most interesting from a scientific standpoint and not a “he said/she said” argument. I wanted to write more on this until I saw the last point made by the ACR …
Where the ACR basically accuses the authors of misconduct. The ACR statement: “To be valid, randomized, controlled trials (RCT) must employ a system to ensure that the assignment of women to the screening group or the unscreened control group is random. Nothing can/should be known about participants until they have been assigned to one of these groups. The CNBSS violated these fundamental rules. Every woman first had a clinical breast examination by a trained nurse so that they knew which women had breast lumps, many of which were cancers, and which women had large lymph nodes in their armpits many of which indicated advanced cancer. Before assigning the women to be in the group offered screening or the control women, investigators knew who had large incurable cancers. This was a major violation of RCT protocol. It most likely resulted in the statistically significant excess of women with advanced breast cancers assigned to the screening arm compared to those assigned to the control arm. This guaranteed more deaths among the screened women than the control women. The five year survival from breast cancer among women ages 40–49 in Canada in the 1980s was only 75 percent, yet the control women in the CNBSS, who were supposed to reflect the Canadian population at the time, had a greater than 90 percent five year survival. This indicates that cancers may have been shifted from the control arm to the screening arm. Coupling the fundamentally corrupted allocation process …” (the italics are mine).

However, the exact wording from the BMJ article about the randomization is as follows:

The examiners had no role in the randomisation that followed; this was performed by the study coordinators in each centre. Randomisation was individual and stratified by centre and five year age group. Irrespective of the findings on physical examination, women aged 40-49 were independently and blindly assigned randomly to receive mammography or no mammography.

So the authors are saying their randomization was blinded and the ACR’s counter-claim is that is couldn’t have been. Both can’t be right. The ACR is either accusing the author of lying or saying he had rogue study nurses who didn’t follow protocol. The ACR does not provide any references to support this claim.

This last part of the ACR claim sounds a lot like school yard taunt, “You lied. How do I know? Because I said so.” It is the kind of claim one makes when one, a) hasn’t thought it all through and is letting emotions rule, b) has an otherwise weak argument that needs to be bolstered, or c) you want to be over the top to get page clicks. This accusation actually leads me to evaluate the ACR’s other two claims with greater scrutiny.

I contacted both the Canadian and the American Colleges of Radiology for clarification. I specifically asked the ACR for hard data to back up this 3rd claim. The Canadians said they’d call (they haven’t) and the ACR replied as follows:

Critiques of study design and execution are routinely done for all scientific studies and do not constitute accusations of fraud. However, the irregularities in this trial design and execution raise valid scientific concerns. It is not a matter of whether we, or anyone else, “believe” trial coordinators, but that this is science. The responsibility lies with the trial conductors to demonstrate the soundness of the randomization process. It is not a matter of others in the scientific community to simply “believe.” The trial design and the subsequent outlier results raise these valid questions which are clearly outlined in many peer-reviewed publications cited in our response and elsewhere. We stand behind the statement that this study should not be used to formulate breast cancer screening policy.

For me this statement isn’t enough. If you are a national organization representing physicians you cannot continue to make claims about how subjects were randomized without proof. Saying the patients weren’t randomized appropriately isn’t a study design issue, it’s an ethical conduct issue because the authors say they did randomize blindly. The science goes both ways. In essence the ACR is saying, facts matter but we don’t have to provide any.

If the ACR is right and the equipment was faulty and the science doesn’t support a 32% breast cancer detection rate by mammography why bring up a completely unsubstantiated 3rd claim about a corrupted randomization process? The science should be enough, don’t you think?

It’s food for thought.