The development and validation of a novel deep-learning algorithm to predict in-hospital cardiac arrest in ED-ICU (emergency department-based intensive care units): a single center retrospective cohort study

Over recent years, the escalation of patient volumes in emergency departments (ED) worldwide has posed to the delivery of timely critical care. Intensive Care Unit (ICU) services became essential due to increasing acuity in EDs, and previous studies revealed a strong association between prolonged boarding times and unfavorable outcomes. Innovative strategies such as Emergency Department-based Intensive Care Units (ED-ICUs) have been introduced to optimize critical care delivery. Given the higher acuity and mortality rates in ED-ICU patients, the prediction of certain events, such as In-Hospital Cardiac Arrest (IHCA), has become abstruse. Conventional Early Warning Scores (EWSs) were developed to stratify the risk of conventional ICUs, but have never been validated in ED-ICU patients with higher acuity. Moreover, EWSs are predominantly focused on forecasting mortality and lack capability for real-time prediction. Our study aimed to develop and validate a deep-learning-based model to predict IHCA within 24 h in ED-ICU. We included 1975 patients admitted to ED-ICU. The study period was from 01 January 2019 to 31 December 2020. Our model, the Deep-ICU CMS (Central Monitoring System), uses four classic vital signs (blood pressure, heart rate, respiratory rate, and body temperature) as input. The model outperformed conventional EWSs in predicting IHCA and maintained performance even with extended prediction windows; it provided robust prediction within a 24-h window, setting it apart from models with restricted prediction horizons. It achieved notably high sensitivity and specificity, overcoming the alarm fatigue issue that is common in EWSs. This study pioneered IHCA risk stratification in ED-ICU and showcases Deep-ICU CMS as a robust prediction tool that overcomes the limitations of conventional EWSs. Prospective and external validation are now warranted to confirm the impact of Deep-ICU CMS in real-world practice. Given the scarcity of research in ED-ICU, our findings contribute valuable insights to optimizing critical care delivery.

In-hospital cardiac arrest (IHCA); Emergency department-based intensive care unit (ED-ICU); Early warning score (EWS); Cardiac arrest (CA) prediction; Clinical deterioration; Machine learning; Deep learning; DeepCars

[1] Sartini M, Carbone A, Demartini A, Giribone L, Oliva M, Spagnolo AM, et al. Overcrowding in emergency department: causes, consequences, and solutions-a narrative review. Healthcare. 2022; 10: 1625.

[2] Herring AA, Ginde AA, Fahimi J, Alter HJ, Maselli JH, Espinola JA, et al. Increasing critical care admissions from U.S. emergency departments, 2001–2009. Critical Care Medicine. 2013; 41: 1197–1204.

[3] Singer AJ, Thode Jr HC, Viccellio P, Pines JM. The association between length of emergency department boarding and mortality. Academic Emergency Medicine. 2011; 18: 1324–1329.

[4] Bhat R, Goyal M, Graf S, Bhooshan A, Teferra E, Dubin J, et al. Impact of post-intubation interventions on mortality in patients boarding in the emergency department. Western Journal of Emergency Medicine. 2014; 15: 708–711.

[5] Gunnerson KJ, Bassin BS, Havey RA, Haas NL, Sozener CB, Medlin RP, et al. Association of an emergency department-based intensive care unit with survival and inpatient intensive care unit admissions. JAMA Network Open. 2019; 2: e197584.

[6] Verma A, Vishen A, Haldar M, Jaiswal S, Ahuja R, Sheikh WR, et al. Increased length of stay of critically ill patients in the emergency department associated with higher in-hospital mortality. Indian Journal of Critical Care Medicine. 2021; 25: 1221–1225.

[7] Asplin BR, Magid DJ, Rhodes KV, Solberg LI, Lurie N, Camargo CA. A conceptual model of emergency department crowding. Annals of Emergency Medicine. 2003; 42: 173–180.

[8] McKenna P, Heslin SM, Viccellio P, Mallon WK, Hernandez C, Morley EJ. Emergency department and hospital crowding: causes, consequences, and cures. Clinical and Experimental Emergency Medicine. 2019; 6: 189–195.

[9] Savioli G, Ceresa IF, Guarnone R, Muzzi A, Novelli V, Ricevuti G, et al. Impact of coronavirus disease 2019 pandemic on crowding: a call to action for effective solutions to “access block”. The Western Journal of Emergency Medicine. 2021; 22: 860–870.

[10] Pearce S, Marchand T, Shannon T, Ganshorn H, Lang E. Emergency department crowding: an overview of reviews describing measures causes, and harms. Internal and Emergency Medicine. 2023; 18: 1137–1158.

[11] Jones S, Moulton C, Swift S, Molyneux P, Black S, Mason N, et al. Association between delays to patient admission from the emergency department and all-cause 30-day mortality. Emergency Medicine Journal. 2022; 39: 168–173.

[12] Tej Prakash S, Brunda RL, Sakshi Y, Sanjeev B. Practice Changing Innovations for Emergency Care during the COVID-19 Pandemic in Resource Limited Settings. In: Vijay K, editor. SARS-CoV-2 Origin and COVID-19 Pandemic Across the Globe (pp. 195–208). IntechOpen: Rijeka. 2021.

[13] Tedesco D, Capodici A, Gribaudo G, Di Valerio Z, Montalti M, Salussolia A, et al. Innovative health technologies to improve emergency department performance. European Journal of Public Health. 2022; 32; 131–169.

[14] Weingart SD, Sherwin RL, Emlet LL, Tawil I, Mayglothling J, Rittenberger JC. ED intensivists and ED intensive care units. The American Journal of Emergency Medicine. 2013; 31: 617–620.

[15] Jeong H, Jung YS, Suh GJ, Kwon WY, Kim KS, Kim T, et al. Emergency physician-based intensive care unit for critically ill patients visiting emergency department. The American Journal of Emergency Medicine. 2020; 38: 2277–2282.

[16] Korean National Emergency Medical Center. Statistical yearbook of emergency medical service, 2021. 2021. Available at: https://www.e-gen.or.kr/nemc/statistics_annual_report.do (Accessed: 15 October 2023).

[17] Kim JH, Kim J, Bae S, Lee T, Ahn JJ, Kang BJ. Intensivists’ direct management without residents may improve the survival rate compared to high-intensity intensivist staffing in academic intensive care units: retrospective and crossover study design. Journal of Korean Medical Science. 2020; 35: e19.

[18] Tsao CW, Aday AW, Almarzooq ZI, Anderson CAM, Arora P, Avery CL, et al. Heart disease and stroke statistics-2023 update: a report from the american heart association. Circulation. 2023; 147: e93–e621.

[19] Nijor S, Rallis G, Lad N, Gokcen E. Patient safety issues from information overload in electronic medical records. Journal of Patient Safety. 2022; 18: e999–e1003.

[20] Cox EGM, Wiersema R, Eck RJ, Kaufmann T, Granholm A, Vaara ST, et al. External validation of mortality prediction models for critical illness reveals preserved discrimination but poor calibration. Critical Care Medicine. 2023; 51: 80–90.

[21] Le Gall JR. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA. 1993; 270: 2957–2963.

[22] Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Critical Care Medicine. 1985; 13: 818–829.

[23] Kwon J, Lee Y, Lee Y, Lee S, Park J. An algorithm based on deep learning for predicting in-hospital cardiac arrest. Journal of the American Heart Association. 2018; 7: e008678.

[24] Pandya S, Gadekallu TR, Reddy PK, Wang W, Alazab M. InfusedHeart: a novel knowledge-infused learning framework for diagnosis of cardiovascular events. IEEE Transactions on Computational Social Systems. 2022; 1–10.

[25] Liu S, Wang X, Xiang Y, Xu H, Wang H, Tang B. Multi-channel fusion LSTM for medical event prediction using EHRs. Journal of Biomedical Informatics. 2022; 127: 104011.

[26] Park SJ, Cho K, Kwon O, Park H, Lee Y, Shim WH, et al. Development and validation of a deep-learning-based pediatric early warning system: a single-center study. Biomedical Journal. 2022; 45: 155–168.

[27] Lee YJ, Cho K, Kwon O, Park H, Lee Y, Kwon J, et al. A multicentre validation study of the deep learning-based early warning score for predicting in-hospital cardiac arrest in patients admitted to general wards. Resuscitation. 2021; 163: 78–85.

[28] Lee Y, Kwon J, Lee Y, Park H, Cho H, Park J. Deep learning in the medical domain: predicting cardiac arrest using deep learning. Acute and Critical Care. 2018; 33: 117–120.

[29] Kang D, Cho K, Kwon O, Kwon J, Jeon K, Park H, et al. Artificial intelligence algorithm to predict the need for critical care in prehospital emergency medical services. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine. 2020; 28: 17.

[30] Cho K, Kwon O, Kwon J, Lee Y, Park H, Jeon K, et al. Detecting patient deterioration using artificial intelligence in a rapid response system. Critical Care Medicine. 2020; 48: e285–e289.

[31] Chandriah KK, Naraganahalli RV. RNN/LSTM with modified Adam optimizer in deep learning approach for automobile spare parts demand forecasting. Multimedia Tools and Applications. 2021; 80: 26145–26159.

[32] Jacobs I, Nadkarni V, Bahr J, Berg RA, Billi JE, Bossaert L, et al. Cardiac arrest and cardiopulmonary resuscitation outcome reports. Circulation. 2004; 110: 3385–3397.

[33] Song MJ, Lee YJ. Strategies for successful implementation and permanent maintenance of a rapid response system. The Korean Journal of Internal Medicine. 2021; 36: 1031–1039.

[34] Smith GB, Prytherch DR, Schmidt PE, Featherstone PI. Review and performance evaluation of aggregate weighted ‘track and trigger’ systems. Resuscitation. 2008; 77: 170–179.

[35] Cvach M. Monitor alarm fatigue: an integrative review. Biomedical Instrumentation & Technology. 2012; 46: 268–277.

[36] Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW. Calibration: the achilles heel of predictive analytics. BMC Medicine. 2019; 17: 230.

[37] Fang AHS, Lim WT, Balakrishnan T. Early warning score validation methodologies and performance metrics: a systematic review. BMC Medical Informatics and Decision Making. 2020; 20: 111.

[38] Xie Y, Chen J, Xu J, Shen B, Liao J, Teng J, et al. Early goal-directed renal replacement therapy in acute decompensated heart failure patients with cardiorenal syndrome. Blood Purification. 2022; 51: 251–259.

[39] Jessen MK, Vallentin MF, Holmberg MJ, Bolther M, Hansen FB, Holst JM, et al. Goal-directed haemodynamic therapy during general anaesthesia for noncardiac surgery: a systematic review and meta-analysis. British Journal of Anaesthesia. 2022; 128: 416–433.

[40] Rivers E, Nguyen B, Havstad S, Ressler J, Muzzin A, Knoblich B, et al. Early goal-directed therapy in the treatment of severe sepsis and septic shock. New England Journal of Medicine. 2001; 345: 1368–1377.

[41] Poncette AS, Spies C, Mosch L, Schieler M, Weber-Carstens S, Krampe H, et al. Clinical requirements of future patient monitoring in the intensive care unit: qualitative study. JMIR Medical Informatics. 2019; 7: e13064.

[42] Romare C, Anderberg P, Sanmartin Berglund J, Skär L. Burden of care related to monitoring patient vital signs during intensive care; a descriptive retrospective database study. Intensive and Critical Care Nursing. 2022; 71: 103213.

[43] Gasciauskaite G, Lunkiewicz J, Roche TR, Spahn DR, Nöthiger CB, Tscholl DW. Human-centered visualization technologies for patient monitoring are the future: a narrative review. Critical Care. 2023; 27: 254.

[44] Allyn J, Devineau M, Oliver M, Descombes G, Allou N, Ferdynus C. A descriptive study of routine laboratory testing in intensive care unit in nearly 140,000 patient stays. Scientific Reports. 2022; 12: 21526.

[45] Kim J, Chae M, Chang HJ, Kim YA, Park E. Predicting cardiac arrest and respiratory failure using feasible artificial intelligence with simple trajectories of patient data. Journal of Clinical Medicine. 2019; 8: 1336.

[46] Collins GS, Reitsma JB, Altman DG, Moons K. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMC Medicine. 2015; 13: 1.

[47] Sung M, Hahn S, Han CH, Lee JM, Lee J, Yoo J, et al. Event prediction model considering time and input error using electronic medical records in the intensive care unit: retrospective study. JMIR Medical Informatics. 2021; 9: e26426.

[48] Kim J, Park YR, Lee JH, Lee JH, Kim YH, Huh JW. Development of a real-time risk prediction model for in-hospital cardiac arrest in critically Ill patients using deep learning: retrospective study. JMIR Medical Informatics. 2020; 8: e16349.

[49] Yijing L, Wenyu Y, Kang Y, Shengyu Z, Xianliang H, Xingliang J, et al. Prediction of cardiac arrest in critically ill patients based on bedside vital signs monitoring. Computer Methods and Programs in Biomedicine. 2022; 214: 106568.

[50] Smith SK, Sincich T. An empirical analysis of the effect of length of forecast horizon on population forecast errors. Demography. 1991; 28: 261–274.

[51] Bounoua Z, Mechaqrane A. Hourly and sub-hourly ahead global horizontal solar irradiation forecasting via a novel deep learning approach: a case study. Sustainable Materials and Technologies. 2023; 36: e00599.

[52] Chandra R, Goyal S, Gupta R. Evaluation of deep learning models for multi-step ahead time series prediction. IEEE Access. 2021; 9: 83105–83123.

[53] Rothman MJ, Rothman SI, Beals J. Development and validation of a continuous measure of patient condition using the electronic medical record. Journal of Biomedical Informatics. 2013; 46: 837–848.

[54] Bell D, Baker J, Williams C, Bassin L. A trend-based early warning score can be implemented in a hospital electronic medical record to effectively predict inpatient deterioration. Critical Care Medicine. 2021; 49: e961–e967.

[55] Lee KJ, Tilling KM, Cornish RP, Little RJA, Bell ML, Goetghebeur E, et al. Framework for the treatment and reporting of missing data in observational studies: the treatment and reporting of missing data in observational studies framework. Journal of Clinical Epidemiology. 2021; 134: 79–88.

[56] Ghaferi AA, Schwartz TA, Pawlik TM. STROBE reporting guidelines for observational studies. JAMA Surgery. 2021; 156: 577–578.

[57] Combe B. SP0183 2016 update of the EULAR recommendations for the management of early arthritis. Annals of the Rheumatic Diseases. 2016; 75: 44–45.

[58] Mitchell OJL, Dewan M, Wolfe HA, Roberts KJ, Neefe S, Lighthall G, et al. Defining physiological decompensation: an expert consensus and retrospective outcome validation. Critical Care Explorations. 2022; 4: e0677.

[59] Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G, F, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. The Lancet. 2005; 365: 2091–2097.

[60] Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the national early warning score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013; 84: 465–470.

[61] Pimentel MAF, Redfern OC, Malycha J, Meredith P, Prytherch D, Briggs J, et al. Detecting deteriorating patients in the hospital: development and validation of a novel scoring system. American Journal of Respiratory and Critical Care Medicine. 2021; 204: 44–52.

[62] Zimmerman JE, Kramer AA, Knaus WA. Changes in hospital mortality for United States intensive care unit admissions from 1988 to 2012. Critical Care. 2013; 17: R81.

[63] Capuzzo M, Volta CA, Tassinati T, Moreno RP, Valentin A, Guidet B, et al. Hospital mortality of adults admitted to intensive care units in hospitals with and without intermediate care units: a multicentre european cohort study. Critical Care. 2014; 18: 551.

[64] Halpern NA, Pastores SM. Critical care medicine beds, use, occupancy, and costs in the united states. Critical Care Medicine. 2015; 43: 2452–2459.

[65] Pronovost PJ, Needham DM, Waters H, Birkmeyer CM, Calinawan JR, Birkmeyer JD, et al. Intensive care unit physician staffing: financial modeling of the Leapfrog standard. Critical Care Medicine. 2004; 32: 1247–1253.

[66] Halpern NA, Tan KS, DeWitt M, Pastores SM. Intensivists in U.S. acute care hospitals. Critical Care Medicine. 2019; 47: 517–525.

[67] Jatoi NN, Awan S, Abbasi M, Marufi MM, Ahmed M, Memon SF, et al. Intensivist and COVID-19 in the United States of America: a narrative review of clinical roles, current workforce, and future direction. The Pan African Medical Journal. 2022; 41: 210.