A deep-learning model for intracranial aneurysm detection on CT angiography images in China: a stepwise, multicentre, early-stage clinical validation study

作者全名：Hu, Bin; Shi, Zhao; Lu, Li; Miao, Zhongchang; Wang, Hao; Zhou, Zhen; Zhang, Fandong; Wang, Rongpin; Luo, Xiao; Xu, Feng; Li, Sheng; Fang, Xiangming; Wang, Xiaodong; Yan, Ge; Lv, Fajin; Zhang, Meng; Sun, Qiu; Cui, Guangbin; Liu, Yubao; Zhang, Shu; Pan, Chengwei; Hou, Zhibo; Liang, Huiying; Pan, Yuning; Chen, Xiaoxia; Li, Xiaorong; Zhou, Fei; Schoepf, U. Joseph; Varga-Szemes, Akos; Moore, W. Garrison; Yu, Yizhou; Hu, Chunfeng; Zhang, Long Jiang

作者地址：[Hu, Bin; Shi, Zhao; Zhang, Long Jiang] Nanjing Univ, Jinling Hosp, Affiliated Hosp, Dept Radiol,Med Sch, Nanjing, Peoples R China; [Lu, Li; Hu, Chunfeng] Xuzhou Med Univ, Dept Radiol, Affiliated Hosp, Xuzhou, Jiangsu, Peoples R China; [Miao, Zhongchang] First Peoples Hosp Guannan Cty, Dept Med Imaging, Lianyungang, Jiangsu, Peoples R China; [Wang, Hao; Zhou, Zhen; Zhang, Fandong; Zhang, Shu] Deepwise Artificial Intelligence AI Lab, Beijing, Peoples R China; [Wang, Rongpin] Guizhou Prov Peoples Hosp, Dept Med Imaging, Guiyang, Guizhou, Peoples R China; [Luo, Xiao] Maanshan Peoples Hosp, Dept Radiol, Maanshan, Anhui, Peoples R China; [Xu, Feng] Nanjing Med Univ, Dept Med Imaging, Affiliated Suqian First Peoples Hosp, Suqian, Jiangsu, Peoples R China; [Li, Sheng] Hubei Univ Med, Taihe Hosp, Dept Radiol, Shiyan, Hubei, Peoples R China; [Fang, Xiangming] Nanjing Med Univ, Affiliated Wuxi Peoples Hosp, Dept Med Imaging, Wuxi, Jiangsu, Peoples R China; [Wang, Xiaodong] Ningxia Med Univ, Gen Hosp, Dept Radiol, Yinchuan, Ningxia, Peoples R China; [Yan, Ge] Xi An Jiao Tong Univ, Dept Med Imaging, Affiliated Hosp 1, Xian, Peoples R China; [Lv, Fajin] Chongqing Med Univ, Dept Radiol, Affiliated Hosp 1, Chongqing, Peoples R China; [Zhang, Meng] Sanya Peoples Hosp, Dept Radiol, Sanya, Hainan, Peoples R China; [Sun, Qiu] Lanzhou Univ Second Hosp, Dept Radiol, Lanzhou, Gansu, Peoples R China; [Cui, Guangbin] Fourth Mil Med Univ, Mil Med Univ 4, Tangdu Hosp, Dept Radiol, Xian, Shaanxi, Peoples R China; [Liu, Yubao] Southern Med Univ, Shenzhen Hosp, Med Imaging Ctr, Shenzhen, Guangdong, Peoples R China; [Pan, Chengwei] Beihang Univ, Inst Artificial Intelligence, Beijing, Peoples R China; [Hou, Zhibo] Peking Univ Shougang Hosp, Med Imaging Ctr, Dept Radiol, Beijing, Peoples R China; [Liang, Huiying] Guangdong Prov Peoples Hosp, Med Big Data Ctr, Guangzhou 510317, Peoples R China; [Pan, Yuning] Ningbo First Hosp, Dept Radiol, Ningbo, Zhejiang, Peoples R China; [Chen, Xiaoxia] Chinese Peoples Liberat Army Gen Hosp, Ctr Med Ctr 3, Dept Radiol, Beijing, Peoples R China; [Li, Xiaorong] Gen Hosp Southern Theater Command, Dept Radiol, PLA, Guangzhou, Guangdong, Peoples R China; [Zhou, Fei] Cent Hosp Jilin City, Dept Radiol, Jilin, Peoples R China; [Schoepf, U. Joseph; Varga-Szemes, Akos; Moore, W. Garrison] Med Univ South Carolina, Dept Radiol & Radiol Sci, Div Cardiovasc Imaging, Charleston, SC USA; [Yu, Yizhou] Univ Hong Kong, Dept Comp Sci, Hong Kong, Peoples R China; [Zhang, Long Jiang] Nanjing Univ, Jinling Hosp, Affiliated Hosp, Dept Radiol,Med Sch, Nanjing 210002, Peoples R China

通信作者：Zhang, LJ (通讯作者)，Nanjing Univ, Jinling Hosp, Affiliated Hosp, Dept Radiol,Med Sch, Nanjing 210002, Peoples R China.

来源：LANCET DIGITAL HEALTH

ESI学科分类：　

WOS号：WOS:001228530300001

JCR分区：Q1

影响因子：23.8

年份：2024

卷号：6

期号：4

开始页：e261

结束页：e271

文献类型：Article

关键词：　

摘要：Background Artificial intelligence (AI) models in real -world implementation are scarce. Our study aimed to develop a CT angiography (CTA)-based AI model for intracranial aneurysm detection, assess how it helps clinicians improve diagnostic performance, and validate its application in real -world clinical implementation. Methods We developed a deep-learning model using 16 546 head and neck CTA examination images from 14 517 patients at eight Chinese hospitals. Using an adapted, stepwise implementation and evaluation, 120 certified clinicians from 15 geographically different hospitals were recruited. Initially, the AI model was externally validated with images of 900 digital subtraction angiography-verified CTA cases (examinations) and compared with the performance of 24 clinicians who each viewed 300 of these cases (stage 1). Next, as a further external validation a multi-reader multi -case study enrolled 48 clinicians to individually review 298 digital subtraction angiography-verified CTA cases (stage 2). The clinicians reviewed each CTA examination twice (ie, with and without the AI model), separated by a 4-week washout period. Then, a randomised open -label comparison study enrolled 48 clinicians to assess the acceptance and performance of this AI model (stage 3). Finally, the model was prospectively deployed and validated in 1562 real -world clinical CTA cases. Findings The AI model in the internal dataset achieved a patient-level diagnostic sensitivity of 0<middle dot>957 (95% CI 0<middle dot>939-0<middle dot>971) and a higher patient-level diagnostic sensitivity than clinicians (0<middle dot>943 [0<middle dot>921-0<middle dot>961] vs 0<middle dot>658 [0<middle dot>644-0<middle dot>672]; p<0<middle dot>0001) in the external dataset. In the multi-reader multi -case study, the AI-assisted strategy improved clinicians' diagnostic performance both on a per -patient basis (the area under the receiver operating characteristic curves [AUCs]; 0<middle dot>795 [0<middle dot>761-0<middle dot>830] without AI vs 0<middle dot>878 [0<middle dot>850-0<middle dot>906] with AI; p<0<middle dot>0001) and a peraneurysm basis (the area under the weighted alternative free-response receiver operating characteristic curves; 0<middle dot>765 [0<middle dot>732-0<middle dot>799] vs 0<middle dot>865 [0<middle dot>839-0<middle dot>891]; p<0<middle dot>0001). Reading time decreased with the aid of the AI model (87<middle dot>5 s vs 82<middle dot>7 s, p<0<middle dot>0001). In the randomised open -label comparison study, clinicians in the AI-assisted group had a high acceptance of the AI model (92<middle dot>6% adoption rate), and a higher AUC when compared with the control group (0<middle dot>858 [95% CI 0<middle dot>850-0<middle dot>866] vs 0<middle dot>789 [0<middle dot>780-0<middle dot>799]; p<0<middle dot>0001). In the prospective study, the AI model had a 0<middle dot>51% (8/1570) error rate due to poor-quality CTA images and recognition failure. The model had a high negative predictive value of 0<middle dot>998 (0<middle dot>994-1<middle dot>000) and significantly improved the diagnostic performance of clinicians; AUC improved from 0<middle dot>787 (95% CI 0<middle dot>766-0<middle dot>808) to 0<middle dot>909 (0<middle dot>894-0<middle dot>923; p<0<middle dot>0001) and patient-level sensitivity improved from 0<middle dot>590 (0<middle dot>511-0<middle dot>666) to 0<middle dot>825 (0<middle dot>759-0<middle dot>880; p<0<middle dot>0001). Interpretation This AI model demonstrated strong clinical potential for intracranial aneurysm detection with improved clinician diagnostic performance, high acceptance, and practical implementation in real -world clinical cases.

基金机构：National Natural Science Foundation of China

基金资助正文：Funding National Natural Science Foundation of China.