Medical science is making such remarkable progress that soon none of us will be well.
-Aldous Huxley

The recently completed conference, “Preventing Over Diagnosis,” was easily the most important meeting I ever attended. Sponsored by the British Medical Journal, Consumer’s Reports, and Dartmouth and Bond Universities, the goal was to identify the excesses in medical care and to figure out how to correct them.

The evidence is compelling that we in the developed countries (especially the US) are overtesting for disease, overdiagnosing it, and overtreating. Wasteful medical care of milder or nonexistent problems does more harm than good to the individual patient, diverts scarce medical resources away from those who really need them, and is an unsustainable drain on the economy.

The causes of medical excess are many and powerful.

Here is a truncated list:

• With very few exceptions, the early screening and intervention touted by preventive medicine has turned out to be an oversold, dangerous, and expensive flop.

Routine PSA screening for prostate cancer is the clearest example. It used to be recommended that men of a certain age be tested yearly. It is now recommended that the test not be done at all unless a man has a family history or other special risk factors.

Why the big change? Definitive long term studies prove that the test doesn’t save lives and instead ruins them by triggering invasive interventions with painful complications. Screening is usually too late to stop fast spreading tumors and too good at identifying slow growing ones that don’t count and are better left alone. If they live long enough, the majority of men will develop an incidental and benign prostate cancer before they die from something else. Picking up these tumors early causes great grief for no return.

• Lowering the thresholds of disease definitions has identified diseases that don’t exist. The dream was that getting there early would help prevent the development of severe heart problems, hypertension, diabetes, osteoporosis, and a score of other illnesses. The reality is that getting there too early misidentifies too many people who are not really at risk and then subjects them to needless and harmful tests and treatments

• The technology is out of hand. If we do enough CT scans we can find structural abnormalities in just about everyone. But most findings are incidental and don’t have any real clinical meaning. Paradoxically, lots of otherwise healthy people will get dangerous cancers from the CT radiation that served no useful purpose.

• Doctors have gotten into the habit of ordering huge batteries of laboratory tests and treating the results while ignoring what is best for this particular patient. There needs to be retraining of those already in practice, a change in how medicine is taught to new doctors, and a realignment of financial incentives to promote best care, not excessive care.

• Except for hospice care, it has become almost impossible to die in a dignified, humane, and cost effective way. Hospitals have become frenetic torture chambers that make dying much worse than death and cost an obscene fortune. .

Curing medical excess will not be easy. Harmful overtesting and overtreating is promoted and protected by the enormous economic and political power of the medical industrial complex. Here’s just a beginning list of what needs to be done:

• Tame and shame Big Pharma. Stop the direct to consumer advertising that is allowed only in the US and New Zealand. Prohibit all pharma contributions to professional associations and consumer groups. Regulate and make transparent all the marketing ploys used to mislead doctors. Force the publication of all clinical research trial data.

• Recognize that all existing medical guidelines that define disease thresholds and make treatment recommendations are suspect. They have been developed by experts in each field who always have an intellectual conflict of interest (and often enough also have a financial conflict of interest) that biases them toward overdiagnosis and overtreatment in their pet area of research interest. New diagnostic standards are as dangerous as new drugs and need the same careful and independent vetting to tame unrealistic diagnostic enthusiasm.

• Employers, insurance companies, and government payers should be smarter consumers of health services and should stop paying for tests and treatments that do more harm than good and are not cost effective.

• Consumers should be smarter consumers and not buy into the idea that more is always better.

• Medical journals need to be more skeptical of the medical research enterprise and should look toward the harms, not just the potentials, of each new purported advance. They should stop drum beating each new study as if it is another big step toward the cure for cancer- which has proven more elusive than anyone imagined.

• The media needs to expose the real harms, not just the imagined wonders, inherent in medical procedures.

• We need to provide more resources to treat the really sick who now often get very inadequate care and at the same time need to protect the really well from getting what is often excessive and harmful care. It is wonderful that medical knowledge and tools have advanced so far, but disheartening that we are so bad at distributing them rationally.

A great deal of progress has already been made. Fifty medical professional associations in the US have seen the need to cut back on inappropriate testing and treatment. The Choosing Wisely initiative is a terrific start in reforming the disaster of our medical non system. The British Medical Journal and Consumer Reports have already played a catalytic role and are powerful platforms for spreading the evidence to physicians and patients.

The participants at the conference were 330 policy makers, clinicians, researchers, consumers, medical journal editors, and representatives of several other initiatives also trying to make medical care safer and more rational. Although the US leads the way in wasteful and harmful care, the problem is clearly international- people came to Hanover, NH came from 28 countries.

This is a David vs Goliath struggle. The forces that benefit from massive overdiagnosis can martial hundreds of billions of dollars a year to promote and protect it. The forces supporting rational medical decision making have access to just a few million dollars a year. The smart money is betting on the big bucks and the status quo.

But there is hope. Big Tobacco also seemed impregnable just twenty five years ago but was brought down by hard facts and a tiny band of dedicated reformers. Right sometimes does bring might.

Allen Frances is a psychiatrist and professor emeritus, Duke University.  He blogs at the Huffington Post.