Effect of an artificial-intelligent chest radiographs reporting system in an emergency department

Though chest radiography is a first-line diagnostic tool in the emergency department (ED), interpretation has a high error rate. We aimed to evaluate the usability and acceptability of deep learning-based computer-aided detection for chest radiography (DeepCADCR) in an ED environment. We conducted a single-institution survey of emergency physicians (EPs) who had used DeepCADCR (Lunit INSIGHT Chest Xray (CXR), version 3.1.4.1) as part of their ED workflow for at least three months. We developed 22 questions that assessed the subscales of effectiveness, efficiency, safety, satisfaction, and reliability. A seven-point Likert agreement scale was used to rate the responses. A total of 23 EPs who completed the survey was enrolled in the study. When averaged by subscale, satisfaction scores were highest (mean 4.71, standard deviation (SD) 1.43), and safety scores were lowest (mean 4.3, SD 0.72). When scores were converted to acceptability, the total average acceptance of DeepCADCR was 86.0%, with higher scores in ED residents than ED specialists for all subscales. Use of DeepCADCR in the ED workflow was well accepted by EPs.

Artificial intelligence; Deep learning; Chest radiography; Emergency department; Survey; Computer-aided detection

[1] Aronchick J, Epstein D, Gefter WB, Miller WT. Evaluation of the chest radiograph in the emergency department patient. Emergency Medicine Clinics of North America. 1985; 3: 491–505.

[2] Chung JH, Duszak R, Hemingway J, Hughes DR, Rosenkrantz AB. Increasing utilization of chest imaging in us emergency departments from 1994 to 2015. Journal of the American College of Radiology. 2019; 16: 674–682.

[3] Donald JJ, Barnard SA. Common patterns in 558 diagnostic radiology errors. Journal of Medical Imaging and Radiation Oncology. 2012; 56: 173–178.

[4] Al aseri Z. Accuracy of chest radiograph interpretation by emergency physicians. Emergency Radiology. 2009; 16: 111–114.

[5] Kim JH, Kim JY, Kim GH, Kang D, Kim IJ, Seo J, et al. Clinical validation of a deep learning algorithm for detection of pneumonia on chest radiographs in emergency department patients with acute febrile respiratory illness. Journal of Clinical Medicine. 2020; 9: 1981.

[6] Hwang EJ, Nam JG, Lim WH, Park SJ, Jeong YS, Kang JH, et al. Deep learning for chest radiograph diagnosis in the emergency department. Radiology. 2019; 293: 573–580.

[7] Jin KN, Jae HJ, Shin CI, Chai JW, Chun SR, Shin SD, et al. Overnight preliminary interpretations of CT and MR images by radiology residents in ER: how accurate are they? Journal of The Korean Society of Emergency Medicine. 2008; 19: 205–210.

[8] Sellers A, Hillman BJ, Wintermark M. Survey of after-hours coverage of emergency department imaging studies by US academic radiology departments. Journal of the American College of Radiology. 2014; 11: 725–730.

[9] Hwang EJ, Park S, Jin K, Kim JI, Choi SY, Lee JH, et al. Development and validation of a deep learning—based automatic detection algorithm for active pulmonary tuberculosis on chest radiographs. Clinical Infectious Diseases. 2019; 69: 739–747.

[10] Nam JG, Park S, Hwang EJ, Lee JH, Jin K, Lim KY, et al. Development and validation of deep learning-based automatic detection algorithm for malignant pulmonary nodules on chest radiographs. Radiology. 2019; 290: 218–228.

[11] Nam JG, Kim M, Park J, Hwang EJ, Lee JH, Hong JH, et al. Development and validation of a deep learning algorithm detecting 10 common abnormalities on chest radiographs. The European Respiratory Journal. 2021; 57: 2003061.

[12] Kozuka T, Matsukubo Y, Kadoba T, Oda T, Suzuki A, Hyodo T, et al. Efficiency of a computer-aided diagnosis (CAD) system with deep learning in detection of pulmonary nodules on 1-mm-thick images of computed tomography. Japanese Journal of Radiology. 2020; 38: 1052–1061.

[13] Summers RM. Improving the accuracy of CTC interpretation: computer-aided detection. Gastrointestinal Endoscopy Clinics of North America. 2010; 20: 245–257.

[14] van Zelst JC, Tan T, Mann RM, Karssemeijer N. Validation of radiologists’ findings by computer-aided detection (CAD) software in breast cancer detection with automated 3D breast ultrasound: a concept study in implementation of artificial intelligence software. Acta Radiologica. 2020; 61: 312–320.

[15] Wani IM, Arora S. Computer-aided diagnosis systems for osteoporosis detection: a comprehensive survey. Medical & Biological Engineering & Computing. 2020; 58: 1873–1917.

[16] Fujita R, Iwasawa T, Aoki T, Iwao Y, Ogura T, Utsunomiya D. Detection of the usual interstitial pneumonia pattern in chest CT: effect of computer-aided diagnosis on radiologist diagnostic performance. To be published in Acta Radiologica. 2020. [Preprint].

[17] Seah JCY, Tang CHM, Buchlak QD, Holt XG, Wardman JB, Aimoldin A, et al. Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study. The Lancet Digital Health. 2021; 3: e496–e506.

[18] Jones CM, Danaher L, Milne MR, Tang C, Seah J, Oakden-Rayner L, et al. Assessment of the effect of a comprehensive chest radiograph deep learning model on radiologist reports and patient outcomes: a real-world observational study. BMJ Open. 2021; 11: e052902.

[19] Kim JH, Han SG, Cho A, Shin HJ, Baek S. Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: a prospective interventional simulation-based study. BMC Medical Informatics and Decision Making. 2021; 21: 311.

[20] Pinto A, Reginelli A, Pinto F, Lo Re G, Midiri F, Muzj C, et al. Errors in imaging patients in the emergency setting. The British Journal of Radiology. 2016; 89: 20150914.

[21] Thakkalpalli M. Reducing diagnostic errors in emergency department with the help of radiographers. Journal of Medical Radiation Sciences. 2019; 66: 152–153.

[22] Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ digital medicine. 2020; 3: 17.

[23] Yoo H, Kim EY, Kim H, Choi YR, Kim MY, Hwang SH, et al. Artificial intelligence-based identification of normal chest radiographs: a simulation study in a multicenter health screening cohort. Korean Journal of Radiology. 2022; 23: 1009–1018.

[24] Chabi M, Borget I, Ardiles R, Aboud G, Boussouar S, Vilar V, et al. Evaluation of the accuracy of a computer-aided diagnosis (CAD) system in breast ultrasound according to the radiologist’s experience. Academic Radiology. 2012; 19: 311–319.

[25] Seagull FJ, Bailey JE, Trout A, Cohan RH, Lypson ML. Residents’ ability to interpret radiology images. Academic Radiology. 2014; 21: 909–915.

[26] Civaner MM, Uncu Y, Bulut F, Chalil EG, Tatli A. Artificial intelligence in medical education: a cross-sectional needs assessment. BMC Medical Education. 2022; 22: 772.

[27] Stead WW, Haynes RB, Fuller S, Friedman CP, Travis LE, Beck JR, et al. Designing medical informatics research and library—resource projects to increase what is learned. Journal of the American Medical Informatics Association. 1994; 1: 28–33.