Social determinants of health (SDoH) have a strong potential to positively and negatively affect health outcomes, but providers have traditionally faced substantial roadblocks in identifying patients who need SDoH interventions.

The result of this gap is often missed opportunities to improve patient health at the individual and population levels.

SDoH factors are responsible for up to 80% of health outcomes, according to the Robert Wood Johnson Foundation, yet patients and providers continue to experience a disconnect related to SDoH.

For example, providers consider patient-reported SDoH in only 35% of health care decision-making, according to a 2023 report in the Annals of Family Medicine. Instead, clinicians generally learn about SDoH through patient-provider communication (76% of the time) and use prior knowledge of a patient’s social risk factors to make clinician decisions (64%).

Additionally, the report found that providers use SDoH information stored in patients’ electronic health records (EHRs) to inform clinical decisions only 46% of the time. Importantly, the use of EHR data to inform clinical decisions about SDoH depended more on how—not whether—the relevant data was stored in EHRs. Clinicians most often used SDoH data to inform clinical decisions when the data was stored in a structured manner in discrete EHR fields rather than as unstructured data in the “notes” sections of patient records.

To overcome challenfges associated with unlocking important SDoH information buried as unstructured data, health systems, and hospitals increasingly turn to artificial intelligence-based tools such as natural language processing (NLP), which can surface critical details and convert them to structured text.

SDoH in the emergency department: one health system’s success story

Despite being located in an affluent community, a top 15 teaching hospital based in the Midwest was experiencing substantial SDoH variation across its patient population. For example, life expectancy within one census tract of the health system’s coverage area trailed that of another tract in the area by 22 years.

To address this large gap, the health system’s leadership launched a program to identify patients experiencing SDoH issues and then direct them to the right local resources. However, the health system faced a challenge in identifying patients with SDoH needs when they were admitted to the emergency department (ED).

Paradoxically, the health system did not lack patient SDoH information within patient records; the challenge was that the critical information was buried as unstructured data in EHRs. This meant that the models used to identify the high-risk patients were not being informed by this information, as the health system had no way to repeatably and programmatically surface this data from the unstructured notes. Therefore, ED social care workers had to identify this information from the record themselves.

While this practice was helpful in identifying high-risk patients, it was time-consuming, and ED social workers spent 80% of their time reviewing patient data and case notes instead of spending face-to-face time with patients.

The health system’s leadership knew the process needed improvement. Accordingly, the health system adopted an NLP solution that leverages artificial intelligence and machine learning concepts to extract unstructured SDoH information from patient records and convert it to structured information.

Here’s how it works: Once a patient is admitted to the ED, the NLP application searches through patient EHRs to identify whether an SDoH need is present and, when appropriate, alerts an ED social worker with the information.

Then, the social worker verifies the SDoH gap with the patient and provides the appropriate resources or referrals. Under the previous system, social workers may not have interacted with these patients at all. In contrast, with the new system, staff in the ED can intervene for any unmet social needs.

Since launching the program, the health system has discovered that about 30% of its population had at least one SDoH factor in unstructured text. This was 300 times more than was documented in the structured fields. Today, the health system has identified 43% more at-risk patients based on their SDoH, enabling staff to connect patients with community resources that can address SDoH needs. The program has been described as a “game changer” by the social workers, who now spend 80% of their time managing patients instead of reading through copious case notes.

This program of AI augmenting clinical decision support in a busy emergency department highlights the ability of technology to reduce cclinicians’work burden and improve patient outcomes. This example of AI working hand in hand with clinicians will hopefully inspire more adoption and use of trustworthy AI solutions in health care.

Calum Yacoubian is a physician executive.