On small data set, model achieved 95.5 percent accuracy in assessment of endoscopic ear images compared with 65.0 percent for clinicians
FRIDAY, Sept. 9, 2022 (HealthDay News) — An artificial intelligence model outperforms clinicians in assessing images of tympanic membranes obtained in an operative setting from children with possible middle ear effusions, according to a study published online Aug. 16 in Otolaryngology-Head and Neck Surgery.
Matthew G. Crowson, M.D., from Massachusetts Eye & Ear in Boston, and colleagues conducted a retrospective cohort study at a tertiary academic medical center from 2018 to 2021. A training set of 639 images of tympanic membranes representing normal, otitis media with effusion, and acute otitis media was used to train a neural network and a proprietary commercial image classifier from Google. Based on classification accuracy, model diagnostic prediction performance was scored for differentiating normal, nonpurulent effusion, and purulent effusion. Clinicians’ diagnostic accuracy was tested on a novel image set and was compared against the current model.
The researchers found that a mean prediction accuracy of 80.8 percent was achieved by the model, while the model from Google achieved 85.4 percent prediction accuracy. In a validation survey of 39 clinicians who analyzed 22 endoscopic ear images, the average diagnostic accuracy was 65.0 percent. The model developed achieved an accuracy of 95.5 percent on the same data set.
“Our result supports prior work that human performance in diagnosing ear infections via direct visual inspection is highly variable and generally poor. The possibility that machine learning systems are superior to humans in the classification task provides motivation for further exploration and external validation of these models,” the authors write.
One author disclosed financial ties to the pharmaceutical, medical device, and health technology industries.
Copyright © 2022 HealthDay. All rights reserved.
