Rethinking the clinical utility of diagnostic tests

Introduction

Approximately fifty years ago, while attending a social gathering at a suburban hospital with my first wife, I found myself speaking with an attending physician. During our conversation he made a remark that has remained with me throughout my professional life.

“Medicine is not science; it is an art.”

At the time, I accepted the statement without much reflection. Certainly medicine requires compassion, judgment, communication, and an appreciation for the complexity of human beings that cannot be reduced to equations or algorithms. Half a century later, after a lifetime spent in engineering, probability theory, statistical pattern recognition, computer vision, machine learning, and artificial intelligence, I have come to believe that the statement is only partially true. Medicine should always remain an art in its relationship with patients. Its diagnostic decision-making, however, can become considerably more scientific.

This essay is not an argument against diagnostic testing. Nor is it intended as a criticism of physicians, many of whom practice under extraordinary time pressures while attempting to provide excellent care. Rather, it is a proposal. I believe that modern medicine would benefit by incorporating concepts that have long been fundamental in engineering, operations research, economics, and artificial intelligence (Bayesian decision theory and the value of information) into routine physician-patient discussions regarding diagnostic testing.

My central thesis is remarkably simple. The purpose of a diagnostic test is not to acquire information. It is to acquire information that improves clinical decisions and ultimately improves patient outcomes. These are not equivalent statements.

Evidence-based medicine

Over the past three decades, medicine has increasingly embraced the principles of evidence-based medicine (EBM). David Sackett and his colleagues famously defined evidence-based medicine as “The conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.”

Importantly, evidence-based medicine was never intended to replace clinical judgment. Rather, it combines three complementary components:

• The best available scientific evidence
• The physician’s clinical expertise
• The values and preferences of the individual patient

This represented one of the great intellectual advances of modern medicine. Clinical decisions increasingly became grounded in carefully conducted scientific studies rather than tradition or anecdotal experience alone. Evidence-based medicine has unquestionably improved medical practice.

Yet I believe an important question remains underemphasized. Evidence-based medicine tells physicians: What does the current evidence suggest? It says considerably less about another equally important question: Should we obtain additional evidence through another diagnostic test?

Those are fundamentally different questions. One concerns scientific evidence. The other concerns scientific decision-making.

Thinking like an engineer

Engineers confront uncertainty every day. Whether designing aircraft, communication systems, autonomous vehicles, medical imaging algorithms, or machine-learning architectures, uncertainty is unavoidable. Every measurement has a cost. Measurements consume time, money, computational resources, and engineering effort, and frequently introduce their own uncertainties.

Consequently, engineers rarely ask: Can we obtain another measurement? Instead they ask: Will obtaining another measurement improve the decision? This distinction has profound implications. The objective is not to maximize information. The objective is to maximize the quality of the final decision.

Bayesian decision theory

Bayesian reasoning provides a mathematical framework for decision-making under uncertainty. The central idea is surprisingly intuitive. Before obtaining any new information, we already possess some estimate of what is most likely to be true. These initial beliefs are called prior probabilities.

A diagnostic test does not begin the reasoning process. It modifies it. Once new information becomes available, our prior beliefs are updated. The revised estimates are known as posterior probabilities. This process is called Bayesian updating.

Notice something important. The purpose of obtaining additional information is not merely to reduce uncertainty. It is to improve the decision that follows. Reducing uncertainty is only valuable if doing so changes what we actually decide to do.

Diagnostic accuracy, diagnostic yield, and clinical utility

Modern medicine frequently discusses the quality of diagnostic tests. Unfortunately, three distinct concepts are often blended together. I believe they should be considered separately.

Diagnostic accuracy

Diagnostic accuracy measures how well a test distinguishes disease from health: sensitivity, specificity, positive predictive value, negative predictive value, and receiver operating characteristic (ROC) curves. These are essential measures of test performance.

Diagnostic yield

Diagnostic yield asks a different question: How often does the test identify an abnormality? A highly sensitive test may detect numerous abnormalities. Some may be clinically important. Others may represent incidental findings unrelated to the patient’s symptoms. Diagnostic yield therefore measures how frequently information is obtained.

Clinical utility

Clinical utility asks what is, in my opinion, the most important question: How often does the information produced by this test actually improve patient outcomes? Notice that clinical utility is not the same as diagnostic accuracy. Nor is it the same as diagnostic yield. A test may possess outstanding diagnostic accuracy while providing little practical benefit in a particular patient if the result does not alter management. Conversely, a relatively simple test may possess enormous clinical utility if its results immediately change treatment.

Patients rarely undergo diagnostic studies merely to acquire interesting information. They undergo testing because they hope the information will improve their lives.

The value of information

Decision theorists have long recognized another concept that receives remarkably little attention in routine medical conversations. It is known as the value of information (VOI). The value of information asks an extraordinarily practical question: How much is additional information worth before making a decision?

This question has been studied for decades in engineering, economics, operations research, and artificial intelligence. Its answer is deceptively simple. Information possesses no intrinsic value. Its value derives entirely from its ability to improve decisions.

Suppose another measurement can be obtained. If that measurement will not alter the final decision, then obtaining it possesses little practical value regardless of how scientifically interesting the information may be. Conversely, if obtaining another measurement is likely to alter the decision in a way that substantially improves outcomes, then its value may be extremely high.

This distinction appears obvious. Yet surprisingly little of this reasoning is explicitly discussed during many physician-patient conversations.

The question that changed my thinking

Several years ago I realized that I had been asking physicians a fundamentally different question than many of my fellow patients. Most patients understandably ask, “What will this test show?” My first question gradually became, “If the test is positive, what changes?” closely followed by “If the test is negative, what changes?”

Those two questions eventually led me to a third: “What is the probability that this test will reveal information that materially changes my treatment?” At that moment I realized I was no longer thinking like a patient. I was thinking like an engineer.

A case study: applying Bayesian reasoning to a proposed vestibular nystagmogram

Recently I experienced a sudden episode of vertigo that resulted in a significant fall. Fortunately, I sustained no fractures. My injuries consisted primarily of soft-tissue trauma and a temporary exacerbation of a long-standing lumbar mechanical disorder. Within days I was again walking more than three miles each day, and my recovery continued steadily over the following weeks.

During my evaluation, an otolaryngologist quite appropriately suggested that I undergo a vestibular nystagmogram (VNG). The recommendation itself did not surprise me. What surprised me was the absence of a discussion regarding the expected clinical utility of the proposed test.

Instead of asking, “What does the VNG involve?” I found myself asking a different series of questions:

• What diagnoses are being considered?
• Based upon my particular clinical presentation, how likely is each diagnosis?
• Which of these diagnoses would actually alter my treatment if confirmed?
• What is the probability that the VNG will distinguish among these possibilities?

Those questions are entirely consistent with Bayesian decision theory. They simply ask that diagnostic recommendations be accompanied by an explicit discussion of probability and expected clinical benefit.

The differential diagnosis

Every diagnostic test begins with a differential diagnosis. In my case, several possibilities were initially considered:

• Benign paroxysmal positional vertigo (BPPV)
• Vestibular neuritis
• Ménière’s disease
• Vestibular migraine
• Persistent unilateral vestibular dysfunction
• A central neurological disorder affecting balance

This appeared entirely reasonable. The next step, however, was equally important. Each possibility should be reconsidered in light of the patient’s individual clinical history.

Vestibular neuritis

Vestibular neuritis typically presents with severe vertigo lasting many hours or even several days, often followed by weeks of imbalance during recovery. My episodes bore little resemblance to this pattern. They were brief. They resolved rapidly. Between episodes I functioned normally. Based upon those observations, vestibular neuritis appeared to represent a very unlikely explanation.

Ménière’s disease

Ménière’s disease classically combines episodic vertigo with fluctuating hearing loss, tinnitus, and a sensation of fullness within the affected ear. I had experienced none of these associated auditory symptoms. Again, this diagnosis appeared unlikely.

Persistent unilateral vestibular dysfunction

Patients with persistent vestibular loss frequently describe chronic imbalance, difficulty walking in darkness, oscillopsia, or a continuing sense of disequilibrium. My own experience was markedly different. Within days of the fall I had resumed walking more than three miles each day without difficulty. Between episodes I considered my balance entirely normal. This diagnosis likewise appeared unlikely.

Vestibular migraine

Vestibular migraine is an important diagnostic consideration in appropriate patients. My own history, however, provided little evidence supporting this diagnosis. While it could not be excluded with certainty, it appeared considerably less likely than other possibilities.

Central neurological disease

Any discussion of vertigo appropriately raises concern regarding disorders involving the central nervous system. Fortunately, my clinical presentation offered several reassuring features. My symptoms were brief. Recovery was rapid. I exhibited no persistent neurological deficits. I experienced no progressive deterioration.

None of these observations excludes a central process with absolute certainty. Medicine rarely deals in certainties. Nevertheless, they substantially reduce its apparent likelihood.

Benign paroxysmal positional vertigo

After considering the alternatives, intermittent BPPV appeared to represent the leading working diagnosis. Several observations supported this conclusion. My episodes were brief. They resolved completely. On the day of my fall I had risen abruptly from bed, a maneuver well recognized as capable of provoking positional vertigo in susceptible individuals. I had experienced a previous episode years earlier with similar characteristics.

The fact that two positional examinations performed after recovery were negative did not substantially alter my thinking, since intermittent BPPV is well known to produce normal examinations between symptomatic episodes. Consequently, my own working hypothesis remained that intermittent BPPV represented the most plausible explanation. Whether that hypothesis ultimately proves correct is less important than the reasoning process itself.

The Bayesian question

At this point I found myself asking what I believe should become a routine question in medical practice: Given my particular clinical presentation, what is the probability that the proposed VNG will produce information that materially changes my management? Notice that this question differs fundamentally from asking whether the VNG is capable of detecting vestibular abnormalities. No one disputes that it can. The relevant question is different: Will it produce information that changes what my physicians and I actually decide to do?

Suppose the VNG demonstrated findings consistent with intermittent BPPV. Treatment would consist primarily of repositioning maneuvers such as the Epley maneuver, together with behavioral strategies designed to reduce fall risk.

Suppose instead that the VNG proved entirely normal. Would that substantially alter the leading clinical hypothesis? Perhaps not, particularly if the episodes remained rare and the patient was asymptomatic at the time of testing. Suppose the study demonstrated a persistent unilateral vestibular deficit. Would that fit the remainder of my clinical presentation? Again, the answer appeared doubtful.

These are not questions regarding diagnostic technology. They are questions regarding expected clinical utility.

Diagnostic yield versus clinical utility

This distinction became increasingly important to me. The VNG may possess excellent diagnostic characteristics for certain vestibular disorders. That fact alone does not determine its value in my own situation.

The question is not: Can this test detect abnormalities? The question is: Given my clinical presentation, how likely is it that those abnormalities will lead to a different management strategy and a better clinical outcome?

Those are entirely different questions. The first concerns diagnostic yield. The second concerns clinical utility. Patients deserve discussions of both.

A second example

This was not the first occasion on which I had found myself reasoning in this manner. Several months earlier, following another episode of vertigo, I was advised to undergo an MRI of the brain and neck, including vascular imaging.

My response was not to reject the recommendation. Nor was it to question the diagnostic capabilities of MRI. Instead, I asked exactly the same question: If the study demonstrates an abnormality, what specific aspect of my treatment will change?

That question, in my opinion, lies at the heart of rational medical decision-making. If the expected benefit of the information substantially exceeds its financial cost, psychological burden, potential downstream investigations, and associated risks, then the study possesses high clinical value. If, on the other hand, the probability that the information will alter management is extremely small, then its expected value may likewise be quite limited. The objective is not to minimize testing. The objective is to maximize the value of testing.

The missing conversation

Imagine a physician saying something like this: “Mr. Bello, based upon your clinical history, I believe the leading diagnosis is intermittent benign paroxysmal positional vertigo. Several other diagnoses remain possible, although I believe they are considerably less likely. The proposed VNG may provide additional information. However, because your episodes are rare and you are currently asymptomatic, there is also a significant possibility that the study will be normal. Let’s discuss what we would actually do if each possible result occurred.”

At this point the discussion becomes not merely diagnostic. It becomes collaborative. That, in my opinion, represents true shared decision-making.

The physician as Bayesian decision-maker

Rather than portraying physicians as diagnosticians, portray them as Bayesian decision-makers. Their role is not simply finding disease. Their role is helping patients decide whether additional information is worth obtaining. That is a much richer concept.

Patients deserve transparent probabilistic reasoning

Patients deserve more than recommendations. Patients deserve to understand the reasoning that produced those recommendations. Whenever practical, physicians should discuss:

• The principal diagnostic possibilities
• Their approximate relative likelihood
• The expected diagnostic yield of the proposed study
• The expected clinical utility
• Possible downstream consequences
• How management changes under each possible result

Absolute numerical precision is neither possible nor necessary. Approximate Bayesian reasoning is sufficient. Patients deal with uncertainty remarkably well when uncertainty is explained honestly.

Artificial intelligence may accelerate this transition

Instead of criticizing AI, embrace it. Ironically, Bayesian reasoning already lies at the heart of much of modern artificial intelligence. Machine-learning systems routinely estimate probabilities, update beliefs, quantify uncertainty, and optimize decisions under incomplete information.

It would be ironic indeed if medicine employed artificial intelligence while failing to communicate these same principles to patients. I believe AI has the potential to become not merely a diagnostic assistant, but a decision-support system capable of explaining why additional testing is, or is not, likely to improve outcomes.

The economics of medical information

Now we return briefly to the health care system. Modern medicine operates under powerful incentives. Physicians must consider clinical guidelines, quality metrics, time constraints, documentation, malpractice concerns, and reimbursement structures. None of these incentives necessarily encourage lengthy discussions regarding Bayesian decision-making.

Consequently, many diagnostic decisions become protocol driven rather than individualized. I do not believe this reflects a failure of physicians. I believe it reflects a limitation of the health care system within which physicians practice.

Looking forward

Medicine should continue to embrace evidence-based medicine. But I believe it should take the next step. Evidence-based medicine should evolve toward evidence-based Bayesian decision medicine.

Evidence tells us what is known. Bayesian decision theory tells us whether obtaining additional information is expected to improve decisions. Those two concepts are complementary, not competing.

Conclusion

Now return to the physician from fifty years ago. Approximately fifty years ago I was told, “Medicine is not science; it is an art.” Half a century later, I believe a more complete statement would be this. Medicine should remain an art in its compassion. Medicine should remain an art in its understanding of individual human beings. Medicine should remain an art in helping patients confront uncertainty with honesty and empathy.

But when deciding whether another diagnostic test should be performed, medicine should increasingly embrace the probabilistic reasoning, decision theory, and value of information analysis that have transformed engineering, operations research, economics, statistics, and artificial intelligence.

The question is no longer simply, “Can we obtain more information?” The more important question is, “Will obtaining that information improve the patient’s life?” I believe that is the question every patient deserves to hear discussed before consenting to another diagnostic test.

Questions every patient has the right to ask before undergoing a diagnostic test

I would like to conclude with ten questions that I believe every patient has the right to ask before agreeing to a diagnostic study:

• What diagnoses are you considering?
• Which diagnosis do you currently believe is most likely, and why?
• What is the primary purpose of this diagnostic test?
• How likely is it that this test will reveal information that changes my treatment?
• If the test is positive, what specific aspect of my management will change?
• If the test is negative, what specific aspect of my management will change?
• What are the risks of false-positive findings, incidental abnormalities, or additional downstream testing?
• Would a period of observation or conservative management be a reasonable alternative at this time?
• How confident are you in your current assessment, and what uncertainties remain?
• If I were a member of your own family with this same history and examination, would you recommend exactly the same diagnostic approach?

Martin Bello is an expert on computer vision and machine learning.