Doctors have understood for some time that it was inevitable. The American Cancer Society has acknowledged that cancer screening has been oversold.

It seems like every day you read in the newspaper that what was standard medical care yesterday is now no longer recommended. Don’t doctors know anything? Well, actually they do. And what seems like paradoxical behavior, no longer recommending aggressive screening for certain cancers, actually represents a more sophisticated understanding of the way in which cancer behaves.

The classic understanding of cancer is that once a cancer forms it will continue to grow steadily until it kills the patient. Cancer was viewed as if it were an infectious disease like syphilis. It starts small and easy to treat, may remain hidden for long periods of time, but eventually spreads to other parts of the body becoming incurable along the way. If cancer did indeed spread like that, the aggressive screening programs would make perfect sense.

But decades of research and clinical experience have led to a more sophisticated understanding of cancer. It has always been known that cancers from different parts of the body behave in very different ways. Ovarian cancer is extremely aggressive, while basal cell cancer of the skin grows very slowly. Breast cancer can and does spread to bones and brain, while colon cancer is most likely to spread only to the liver.

More recently we’ve learned that each cancer can be broken down into different subtypes, some more aggressive than others, and some better treated with one regimen instead of another. For example, breast cancers are now analyzed for the presence of hormone receptors on the outside of the cancer cells. The presence or absence of certain receptors tells us whether specific treatments will be helpful or useless, making it easier to target the cancer with the treatment most likely to work.

We have also learned that some cancers follow the model of an infectious disease like syphilis, starting small and curable and ending up throughout the body and incurable, many do not. Some cancers start small and explode aggressively. Others start small and stay small for decades. This more sophisticated understanding is a direct result of being able to diagnose cancer earlier. We now have a much better and far more nuanced understanding of the natural history of various cancers. It has become apparent that rather than finding all cancer, we need only find cancers that are aggressive and can ignore those that are known to grow very slowly if at all.

What’s the big deal? Isn’t cancer screening beneficial regardless of the natural history of the particular cancer? No, it’s not and therein lies the reason for the American Cancer Society’s call for less screening of certain cancers.

The goal of cancer screening is and has always been to reduce cancer deaths and disability, and therefore, that’s how cancer screening should be judged. By that standard, some forms of screening are total successes. For example, the Pap smear, the screening test for cancer of the cervix, has been an unalloyed bright spot in the war against cancer. The test is inexpensive and reliable, the follow up test to actually diagnose cancer (biopsies of the cervix) is harmless, and very few if any women are treated unnecessarily. Screening for cervical cancer saves many lives and has few long term side effects.

By the same standard, prostate cancer screening has been a terrible disappointment. The PSA blood test, the screening test, is notoriously unreliable. Even more problematic is the fact that many prostate cancers grow extremely slowly and are unlikely to spread. Most problematic is that the treatment has very serious side effects, impotence and incontinence. Screening for prostate cancer with the PSA test (and finding tiny cancers) saves no more lives than screening with a prostate exam (which can find cancers that are somewhat larger) and leaves many men with unnecessary long term side effects.

Whereas every cervical cancer is probably dangerous to the patient and the treatment has few long term side effects in any case (since cervical cancer is most commonly diagnosed in women who have completed childbearing), most prostate cancers are not dangerous to the patient and the treatment is often undertaken unnecessarily. It’s bad enough to endure impotence and incontinence as the side effect of life saving treatment. It is tragic to endure it as the side effect of unnecessary treatment.

Breast cancer is similar to prostate cancer. While frequent mammography is more likely to diagnose cancer, there has not been a corresponding decline in breast cancer deaths. Treating many more women with chemotherapy, lumpectomy and mastectomy has produced very few additional lives saved.

The solution to this conundrum, of course, is to develop more sophisticated screening tests, tests that can discriminate between life threatening cancers and non-life threatening cancers. In the meantime, the existing screening tests should be judged on their ability to save lives, not on their ability to diagnose cancer, since many cancers don’t need to be treated.

Screening everybody for everything and screening them often is a very blunt tool that seemed appropriate when we had an unsophisticated understanding of cancer. Now that our understanding of cancer has deepened, the use of screening tests should reflect our new knowledge.

Simply put, screening tests should be reserved for situations in which they save lives. Dialing back on screening tests is not a step backward, it is a step forward in treating only those who need to be treated and not harming anyone else in the process.

Amy Tuteur is an obstetrician-gynecologist who blogs at The Skeptical OB.

Submit a guest post and be heard.