The use of natural language processing (NLP) in the clinical domain dates as far back as the 1960s, with important early work performed in the 1970s and 1980s. While great technical advances were made during this period, clinical NLP failed to live up to initial expectations as accuracy rates were too low to use for much other than research or bulk analytics.

An AHIMA article explained why clinical text poses a challenge to NLP this way: This text is often ungrammatical, consists of “bullet point” telegraphic phrases with limited context, and lacks complete sentences. Clinical notes make heavy use of acronyms and abbreviations, making them highly ambiguous.

Further, traditional NLP has struggled to disambiguate words and phrases that may have multiple meanings in a medical context. For example, the two-letter acronym “PT” is commonly used to mean “patient,” “physiotherapy,” “posterior tibial” (as in the pulse site), “prothrombin time,” a common blood test – and depending on where the patient’s records are coming from, may even refer to the country of Portugal.

Given these shortcomings, it’s no surprise that clinicians have been hesitant to embrace a technology incapable of accurately interpreting information contained within unstructured data – which comprises 80 percent of all data in EHRs, based on industry estimates.

But recent advances in artificial intelligence (AI), large language models (LLMs), and large-scale data extraction tools have spawned a new generation of medical NLP technologies that meet the stringent needs of both clinicians at the point of care and researchers who require high-quality patient data.

Unlike traditional NLP software, modern medical AI platforms are truly clinical grade, possessing the ability to understand medical text shorthand and abbreviations. Often based on “transformers,” a new type of machine learning model, deep learning algorithms are starting to unlock the clinical value buried within unstructured medical text.

Overcoming doubt and confusion

While these are exciting developments, clinicians may be reluctant to put their faith in new technologies which sound similar to the traditional NLP that to date has underperformed. Another factor that may be causing confusion in the health care sector is the rise of generative AI algorithms such as ChatGPT, which come with their own set of considerable problems.

The vast majority of clinicians and other health care professionals are not technologists or well-versed in LLMs, so you can’t blame them for not immediately understanding 1) what medical-grade AI can do for them and their patients, and 2) how it differs from traditional NLP and generative AI.

Below are three things health care professionals should know about medical-grade AI.

Medical-grade AI can understand and extract unstructured data at scale. Medical-grade AI isn’t just an incremental improvement over traditional NLP; it’s a quantum leap, based on its ability to extract, normalize, and contextualize unstructured medical text. That’s because medical AI is powered by machine learning and LLMs that have been developed only in recent years.

While a generative AI interface may appear useful for question-answering tasks about a single document, clinicians need AI systems that can ingest and understand an entire chart for a patient, or an entire organization’s EHR system. Medical-grade AI is enterprise software that has been designed to process and understand millions of documents, not just a short snippet of text.

Unlike generative AI, medical-grade AI doesn’t “hallucinate.” ChatGPT and other generative AI programs currently are notorious for inventing facts, such as citing research papers and authors that don’t actually exist. This presents a clear danger to patients’ lives if relied upon in many clinical scenarios.

Instead, medical AI strikes a balance between recall and precision, giving clinicians the right amount of accurate and relevant data at the point of care, and linking information back to the original data in the patient’s chart, allowing clinicians to check their sources of information for veracity and accuracy.

Medical-grade AI algorithms rely on input from clinicians. Computer scientists and machine learning professionals who typically build traditional NLP software for use in health care frequently lack the medical expertise to understand what type of information would be useful to a clinician or what questions clinicians need answered.

The most effective medical AI platforms are built with a right-brain/left-brain approach using the expertise of working side-by-side with computer scientists to ensure that the AI models understand the important meanings of medical words and the subtleties that often exist in clinical language.

Embrace AI that’s ready for clinicians.

Medical-grade AI is ready, today, built upon the latest technologies, and has hundreds of use cases in clinical medicine, research, administration, and health insurance. Users should be wary of general-purpose generative AI models masquerading as medical-grade AI, and should work with enterprise software vendors who have built products from the ground up specifically to help them transform healthcare and medical research.

Tim O’Connell is a physician executive.