Construction and validation of machine learning models based on bedside parameters for identifying sepsis in acute pancreatitis patients

Acute pancreatitis (AP) with sepsis is a severe and potentially fatal complication. Current predictive systems for identifying high-risk sepsis in AP patients often lack specificity and timeliness, resulting in delays in diagnosis and intervention. This study retrospectively collected data from emergency departments in three tertiary comprehensive hospitals to develop a machine learning (ML) model for the rapid identification of high-risk sepsis in patients with AP. Patients were randomly divided into training and testing datasets (7:3 ratio). In the training dataset, we employed 10 ML algorithms to analyze bedside parameters of patients with AP upon admission. The 10-fold cross-validation was used to find the best parameter and model. The model was then applied to the testing dataset without modifying the model parameters to obtain unbiased classification performance. The performance of the ML model was assessed using the receiver operating characteristic curve (ROC) and compared to scoring systems using the DeLong test. In this study, 771 AP patients were assessed. During hospitalization, 559 patients were diagnosed with sepsis within the first 24 hours, while 212 were not. A Random Forest (RF) model containing 8 features demonstrated the highest area under the curve (AUC) on the cross-validation dataset (AUC: 0.877, accuracy: 0.772), with the AUC of 0.947 and accuracy of 0.836 on the testing dataset. Compared to the Acute Physiology and Chronic Health Evaluation II (AUC 0.708), quick Sequential Organ Failure Assessment (AUC 0.672), and Bedside Index of Severity in Acute Pancreatitis (AUC 0.680), the RF model showed superior performance in predicting sepsis occurrence in patients with AP. This study constructed and validated ML models for the early prediction of sepsis in patients with AP. The RF model provides clinicians with a rapid and useful tool to guide the level of patient care and implement early intervention strategies.

Machine learning; Random forest; Acute pancreatitis; Sepsis

[1] Johnson CD, Besselink MG, Carter R. Acute pancreatitis. BMJ. 2014; 349: g4859.

[2] Petrov MS, Yadav D. Global epidemiology and holistic prevention of pancreatitis. Nature Reviews Gastroenterology & Hepatology. 2019; 16: 175–184.

[3] Iannuzzi JP, King JA, Leong JH, Quan J, Windsor JW, Tanyingoh D, et al. Global incidence of acute pancreatitis is increasing over time: a systematic review and meta-analysis. Gastroenterology. 2022; 162: 122–134.

[4] Susak YM, Dirda OO, Fedorchuk OG, Tkachenko OA, Skivka LM. Infectious complications of acute pancreatitis is associated with peripheral blood phagocyte functional exhaustion. Digestive Diseases and Sciences. 2021; 66: 121–130.

[5] Mifkovic A, Pindak D, Daniel I, Pechan J. Septic complications of acute pancreatitis. Bratislava Medical Journal. 2006; 107: 296–313.

[6] Jaber S, Garnier M, Asehnoune K, Bounes F, Buscail L, Chevaux J, et al. Guidelines for the management of patients with severe acute pancreatitis, 2021. Anaesthesia Critical Care & Pain Medicine. 2022; 41: 101060.

[7] Herwanto V, Shetty A, Nalos M, Chakraborty M, McLean A, Eslick GD, et al. Accuracy of quick sequential organ failure assessment score to predict sepsis mortality in 121 studies including 1,716,017 individuals. Critical Care Explorations. 2019; 1: e0043.

[8] Wagner J, Hernández Blanco YY, Yu A, Garcia-Rodriguez V, Mohajir W, Goodman C, et al. The quick sepsis-related organ failure assessment score is prognostic of pancreatitis severity in patients with alcohol-induced pancreatitis. Pancreas. 2022; 51: 694–699.

[9] Rasch S, Pichlmeier E, Phillip V, Mayr U, Schmid RM, Huber W, et al. Prediction of outcome in acute pancreatitis by the qSOFA and the new ERAP score. Digestive Diseases and Sciences. 2022; 67: 1371–1378.

[10] Hallac A, Puri N, Applebury D, Myers K, Dhumal P, Thatte A, et al. The value of quick sepsis-related organ failure assessment scores in patients with acute pancreatitis who present to emergency departments: a three-year cohort study. Gastroenterology Research. 2019; 12: 67–71.

[11] Mederos MA, Reber HA, Girgis MD. Acute pancreatitis: a review. JAMA. 2021; 325: 382–390.

[12] Cazacu SM, Parscoveanu M, Cartu D, Moraru E, Rogoveanu I, Ungureanu BS, et al. NLR48 is better than CRP, and mCTSI, and similar to BISAP and SOFA scores for mortality prediction in acute pancreatitis: a comparison of 6 scores. Journal of Inflammation Research. 2023; 16: 4793–4804.

[13] Harshit Kumar A, Singh Griwan M. A comparison of APACHE II, BISAP, Ranson’s score and modified CTSI in predicting the severity of acute pancreatitis based on the 2012 revised Atlanta Classification. Gastroenterology Report. 2018; 6: 127–131.

[14] Capurso G, Ponz de Leon Pisani R, Lauri G, Archibugi L, Hegyi P, Papachristou GI, et al. Clinical usefulness of scoring systems to predict severe acute pancreatitis: A systematic review and meta-analysis with pre and post-test probability assessment. United European Gastroenterol J. 2023; 11: 825–836.

[15] Öder M, Eraslan S, Yesilada Y. Automatically classifying familiar web users from eye-tracking data: a machine learning approach. Turkish Journal of Electrical Engineering and Computer Sciences. 2022; 30: 233–248.

[16] Nikzad S, Ebrahimi A. Two person interaction recognition based on a dual-coded modified metacognitive (DCMMC) extreme learning machine. Turkish Journal of Electrical Engineering and Computer Sciences. 2022; 30: 1621–1636.

[17] Agnello L, Vidali M, Padoan A, Lucis R, Mancini A, Guerranti R, et al. Machine learning algorithms in sepsis. Clinica Chimica Acta. 2024; 553: 117738.

[18] Goh KH, Wang L, Yeow AYK, Poh H, Li K, Yeow JJL, et al. Artificial intelligence in sepsis early prediction and diagnosis using unstructured data in healthcare. Nature Communications. 2021; 12: 711.

[19] Yin M, Zhang R, Zhou Z, Liu L, Gao J, Xu W, et al. Automated machine learning for the early prediction of the severity of acute pancreatitis in hospitals. Frontiers in Cellular and Infection Microbiology. 2022; 12: 886935.

[20] Thapa R, Iqbal Z, Garikipati A, Siefkas A, Hoffman J, Mao Q, et al. Early prediction of severe acute pancreatitis using machine learning. Pancreatology. 2022; 22: 43–50.

[21] Liu F, Yao J, Liu C, Shou S. Construction and validation of machine learning models for sepsis prediction in patients with acute pancreatitis. BMC Surgery. 2023; 23: 267.

[22] Banks PA, Bollen TL, Dervenis C, Gooszen HG, Johnson CD, Sarr MG, et al. Classification of acute pancreatitis—2012: revision of the Atlanta classification and definitions by international consensus. Gut. 2013; 62: 102–111.

[23] Song Y, Zhang J, Zhang YD, Hou Y, Yan X, Wang Y, et al. FeAture explorer (FAE): a tool for developing and comparing radiomics models. PLOS ONE. 2020; 15: e0237587.

[24] Li Q, Song Z, Li X, Zhang D, Yu J, Li Z, et al. Development of a CT radiomics nomogram for preoperative prediction of Ki-67 index in pancreatic ductal adenocarcinoma: a two-center retrospective study. To be published in European Radiology. 2023. [Preprint].

[25] Li X, Zhang C, Li T, Lin X, Wu D, Yang G, et al. Early acquired resistance to EGFR-TKIs in lung adenocarcinomas before radiographic advanced identified by CT radiomic delta model based on two central studies. Scientific Reports. 2023; 13: 15586.

[26] Becker T, Rousseau A, Geubbelmans M, Burzykowski T, Valkenborg D. Decision trees and random forests. American Journal of Orthodontics and Dentofacial Orthopedics. 2023; 164: 894–897.

[27] Giannini HM, Ginestra JC, Chivers C, Draugelis M, Hanish A, Schweickert WD, et al. A machine learning algorithm to predict severe sepsis and septic shock: development, implementation, and impact on clinical practice. Critical Care Medicine. 2019; 47: 1485–1492.

[28] Wang D, Li J, Sun Y, Ding X, Zhang X, Liu S, et al. A machine learning model for accurate prediction of sepsis in ICU patients. Frontiers in Public Health. 2021; 9: 754348.

[29] Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016; 315: 801–810.

[30] Kui B, Pintér J, Molontay R, Nagy M, Farkas N, Gede N, et al. EASY-APP: an artificial intelligence model and application for early and easy prediction of severity in acute pancreatitis. Clinical and Translational Medicine. 2022; 12: e842.

[31] Dauphine C, Kovar J, Stabile BE, Haukoos JS, de Virgilio C. Identification of admission values predictive of complicated acute alcoholic pancreatitis. Archives of Surgery. 2004; 139: 978–982.

[32] Bank S, Singh P, Pooran N, Stark B. Evaluation of factors that have reduced mortality from acute pancreatitis over the past 20 years. Journal of Clinical Gastroenterology. 2002; 35: 50–60.

[33] Knapp S. Diabetes and infection: is there a link?—A mini-review. Gerontology. 2013; 59: 99–104.

[34] Hu L, Zhu Y, Chen M, Li X, Lu X, Liang Y, et al. Development and validation of a disease severity scoring model for pediatric sepsis. Iranian Journal of Public Health. 2016; 45: 875–884.

[35] Walter EJ, Hanna-Jumma S, Carraretto M, Forni L. The pathophysiological basis and consequences of fever. Critical Care. 2016; 20: 200.

[36] Feng A, Ao X, Zhou N, Huang T, Li L, Zeng M, et al. A novel risk-prediction scoring system for sepsis among patients with acute pancreatitis: a retrospective analysis of a large clinical database. International Journal of Clinical Practice. 2022; 2022: 5435656.

[37] Thomas-Rüddel DO, Hoffmann P, Schwarzkopf D, Scheer C, Bach F, Komann M, et al. Fever and hypothermia represent two populations of sepsis patients and are associated with outside temperature. Critical Care. 2021; 25: 368.

[38] Mao Q, Jay M, Hoffman JL, Calvert J, Barton C, Shimabukuro D, et al. Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU. BMJ Open. 2018; 8: e017833.

[39] Kraut JA, Madias NE. Lactic acidosis. New England Journal of Medicine. 2014; 371: 2309–2319.

[40] Evans L, Rhodes A, Alhazzani W, Antonelli M, Coopersmith CM, French C, et al. Surviving sepsis campaign: international guidelines for management of sepsis and Septic Shock 2021. Intensive Care Medicine. 2021; 47: 1181–1247.

[41] Calente TJN, Albino LB, Oliveira JG, Delfrate G, Sordi R, Santos FA, et al. Early blood lactate as a biomarker for cardiovascular collapse in experimental sepsis. Shock. 2024; 61: 142–149.

[42] Park H, Lee J, Oh DK, Park MH, Lim CM, Lee SM, et al. Serial evaluation of the serum lactate level with the SOFA score to predict mortality in patients with sepsis. Scientific Reports. 2023; 13: 6351.

[43] Liu J, Huang L, Luo M, Xia X. Bacterial translocation in acute pancreatitis. Critical Reviews in Microbiology. 2019; 45: 539–547.

[44] Glaubitz J, Wilden A, Frost F, Ameling S, Homuth G, Mazloum H, et al. Activated regulatory T-cells promote duodenal bacterial translocation into necrotic areas in severe acute pancreatitis. Gut. 2023; 72: 1355–1369.

[45] Forsmark CE, Vege SS, Wilcox CM. Acute pancreatitis. New England Journal of Medicine. 2016; 375: 1972–1981.