Development and validation of a transformer-based CAD model for improving the consistency of BI-RADS category 3-5 nodule classification among radiologists: a multiple center study

作者全名："Ji, Hongtao; Zhu, Qiang; Ma, Teng; Cheng, Yun; Zhou, Shuai; Ren, Wei; Huang, Huilian; He, Wen; Ran, Haitao; Ruan, Litao; Guo, Yanli; Tian, Jiawei; Chen, Wu; Chen, Luzeng; Wang, Zhiyuan; Zhou, Qi; Niu, Lijuan; Zhang, Wei; Yang, Ruimin; Chen, Qin; Zhang, Ruifang; Wang, Hui; Li, Li; Liu, Minghui; Nie, Fang; Zhou, Aiyun"

作者地址："[Ran, Haitao] Capital Med Univ, Beijing Tongren Hosp, Dept Diagnost Ultrasound, Beijing, Peoples R China; [He, Wen] Capital Med Univ, Beijing Tiantan Hosp, Dept Ultrasonog, Beijing, Peoples R China; [Ran, Haitao] Chongqing Med Univ, Affiliated Hosp 2, Dept Ultrasound, Chongqing, Peoples R China; [Ruan, Litao] Xi An Jiao Tong Univ, Affiliated Hosp 1, Dept Med Ultrasound, Xian, Peoples R China; [Guo, Yanli] Army Med Univ, Southwest Hosp, Dept Ultrasound, Chongqing, Peoples R China; [Tian, Jiawei] Harbin Med Univ, Affiliated Hosp 2, Dept Ultrasound, Harbin, Peoples R China; [Chen, Wu] Shanxi Med Univ, Hosp 1, Dept Ultrasound, Taiyuan, Peoples R China; [Chen, Luzeng] Peking Univ, Hosp 1, Dept Ultrasound, Beijing, Peoples R China; [Wang, Zhiyuan] Hunan Prov Canc Hosp, Diag Ctr Ultrasound, Dept Ultrasound, Changsha, Peoples R China; [Zhou, Qi] Xi An Jiao Tong Univ, Affiliated Hosp 2, Dept Ultrasound, Xian, Peoples R China; [Niu, Lijuan] Canc Hosp, Chinese Acad Med Sci & Peking Union Med Coll, Natl Canc Ctr, Beijing, Peoples R China; [Zhang, Wei] Guangxi Med Univ, Affiliated Hosp 3, Dept Ultrasonog, Nanning, Peoples R China; [Yang, Ruimin] Hebei North Univ, Dept Ultrasound, Affiliated Hosp 1, Zhangjiakou, Peoples R China; [Chen, Qin] Univ Elect Sci & Technol China, Sichuan Prov Peoples Hosp, Dept Ultrasound, Chengdu, Peoples R China; [Zhang, Ruifang] Zhengzhou Univ, Affiliated Hosp 1, Dept Ultrasound, Zhengzhou, Peoples R China; [Wang, Hui] Jilin Univ, China Japan Union Hosp, Dept Ultrasound, Changchun, Peoples R China; [Li, Li] Qilu Hosp Shandong Univ, Dept Ultrasound, Qingdao, Peoples R China; [Liu, Minghui] Cent South Univ, Xiangya Hosp 2, Dept Ultrasound Diag, Changsha, Peoples R China; [Nie, Fang] Lanzhou Univ, Dept Ultrasound, Hosp 2, Lanzhou, Peoples R China; [Zhou, Aiyun] Nanchang Univ, Affiliated Hosp 1, Dept Ultrasound, Nanchang, Peoples R China; [Zhu, Qiang] Capital Med Univ, Beijing Tiantan Hosp, Dept Ultrasonog, 119 West Section of South 4th Ring Rd, Beijing 100070, Peoples R China; [He, Wen] Capital Medial Univ, Beijing Tongren Hosp, Dept Diagnost Ultrasound, Beijing 100730, Peoples R China"

通信作者："Zhu, Q (通讯作者)，Capital Med Univ, Beijing Tiantan Hosp, Dept Ultrasonog, 119 West Section of South 4th Ring Rd, Beijing 100070, Peoples R China.; He, W (通讯作者)，Capital Medial Univ, Beijing Tongren Hosp, Dept Diagnost Ultrasound, Beijing 100730, Peoples R China."

来源：QUANTITATIVE IMAGING IN MEDICINE AND SURGERY

ESI学科分类：CLINICAL MEDICINE

WOS号：WOS:000982615300001

JCR分区：Q2

影响因子：2.9

年份：2023

卷号：　

期号：　

开始页：　

结束页：　

文献类型：Article; Early Access

关键词：Breast Imaging Reporting and Data Systems (BI-RADS); computer-aided diagnosis (CAD); ultrasound; transformers

摘要："Background: Significant differences exist in the classification outcomes for radiologists using ultrasonography-based Breast Imaging Reporting and Data Systems for diagnosing category 3-5 (BI-RADS 3-5) breast nodules, due to a lack of clear and distinguishing image features. Consequently, this retrospective study investigated the improvement of BI-RADS 3-5 classification consistency using a transformer-based computer-aided diagnosis (CAD) model.Methods: Independently, 5 radiologists performed BI-RADS annotations on 21,332 breast ultrasonographic images collected from 3,978 female patients from 20 clinical centers in China. All images were divided into training, validation, testing, and sampling sets. The trained transformer-based CAD model was then used to classify test images, for which sensitivity (SEN), specificity (SPE), accuracy (ACC), area under the curve (AUC), and calibration curve were evaluated. Variations in these metrics among the 5 radiologists were analyzed by referencing BI-RADS classification results for the sampling test set provided by CAD to determine whether classification consistency (the k value), SEN, SPE, and ACC could be improved. Results: After the training set (11,238 images) and validation set (2,996 images) were learned by the CAD model, the classification ACC of the CAD model applied to the test set (7,098 images) was 94.89% in category 3, 96.90% in category 4A, 95.49% in category 4B, 92.28% in category 4C, and 95.45% in category 5 nodules. Based on pathological results, the AUC of the CAD model was 0.924 and the predicted probability of CAD was a little higher than the actual probability in the calibration curve. After referencing BI-RADS classification results, the adjustments were made to 1,583 nodules, of which 905 were classified to a lower category and 678 to a higher category in the sampling test set. As a result, the ACC (72.41-82.65%), SEN (32.73-56.98%), and SPE (82.46-89.26%) of the classification by each radiologist were significantly improved on average, with the consistency (k values) in almost all of them increasing to >0.6.Conclusions: The radiologist's classification consistency was markedly improved with almost all the k values increasing by a value greater than 0.6, and the diagnostic efficiency was also improved by approximately 24% (32.73% to 56.98%) and 7% (82.46% to 89.26%) for SEN and SPE, respectively, of the total classification on average. The transformer-based CAD model can help to improve the radiologist's diagnostic efficacy and consistency with others in the classification of BI-RADS 3-5 nodules."

基金机构：National Key Research and Development Plan of China [2016YFC0104803]

基金资助正文：Funding: This project was supported by the National Key Research and Development Plan of China (No. 2016YFC0104803) .