External validation study shows poor performance for proprietary sepsis prediction model in hospitalized patients
TUESDAY, June 22, 2021 (HealthDay News) — The Epic Sepsis Model (ESM) has poor discrimination and calibration for predicting sepsis, according to a study published online June 21 in JAMA Internal Medicine.
Andrew Wong, M.D., from the University of Michigan Medical School in Ann Arbor, and colleagues externally validated the ESM in the prediction of sepsis and assessed its potential clinical value versus usual care in a retrospective cohort study involving 27,697 patients with 38,455 hospitalizations between Dec. 6, 2018, and Oct. 20, 2019. The potential clinical benefit associated with the ESM was assessed based on the added benefit of the ESM score versus contemporary clinical practice (timely administration of antibiotics).
The researchers found that sepsis occurred in 7 percent of the hospitalized patients (2,552 patients). The hospitalization-level area under the receiver operating characteristic curve was 0.63 for the ESM. The ESM identified 7 percent of patients with sepsis who did not receive timely antibiotic administration, highlighting low sensitivity of the ESM relative to contemporary clinical practice. In addition, the ESM did not identify 67 percent of patients with sepsis, despite generating alerts for an ESM score or 6 or higher for 18 percent of all hospitalized patients, thereby creating a considerable burden of alert fatigue.
“Our study has important national implications,” the authors write. “The increase and growth in deployment of proprietary models has led to an underbelly of confidential, non-peer-reviewed model performance documents that may not accurately reflect real-world model performance.”
Editorial (subscription or payment may be required)
Copyright © 2021 HealthDay. All rights reserved.
