GPT-4 generated accurate neuroanatomical localization and detailed clinical reasoning based on processing of raw text from H&P
By Elana Gotkine HealthDay Reporter
WEDNESDAY, March 27, 2024 (HealthDay News) — Large language models (LLMs) such as Generative Pretrained Transformers (GPTs) can potentially localize acute stroke lesions, according to a study published online March 27 in Neurology.
Jung-Hyun Lee, M.D., from the State University of New York Downstate Health Sciences University in Brooklyn, and colleagues examined the capability of GPT-4 for lesion localization based on clinical presentation. GPT-4 was prompted using history and neurologic physical examination (H&P) from published cases of acute stroke followed by questions for clinical reasoning using Zero-Shot Chain-of-Thought and Text Classification that prompted answers for “single or multiple lesions,” “side,” and “brain region.” GPT-4 output was compared with imaging-based localization on three separate trials for each of 46 cases.
The researchers found that based on processing of raw text from H&P, GPT-4 successfully generated accurate neuroanatomical localization and detailed clinical reasoning. Across trial-based analysis for specificity, sensitivity, precision, and F1-score, performance metrics were 0.87, 0.74, 0.75, and 0.74, respectively, for side, and 0.94, 0.85, 0.84, and 0.85, respectively, for brain region. Within the brain region, class labels were similarly high for all regions except the cerebellum, and were similar when considering all three trials to examine metrics by case. Errors were due to both extrinsic and intrinsic causes, including inadequate information in the published cases (extrinsic) and failures of logic or inadequate knowledge base (intrinsic).
“Future clinical use of LLM with fine-tuning and training focused on neurologic diseases and neuroanatomy would particularly assist nonneurologist health care providers with limited training in localization and diagnosis of nervous system disease,” the authors write.
Copyright © 2024 HealthDay. All rights reserved.
