Evaluating the performance of the language model ChatGPT in responding to common questions of people with epilepsy

作者全名："Wu, Yuxin; Zhang, Zaiyu; Dong, Xinyu; Hong, Siqi; Hu, Yue; Liang, Ping; Li, Lusheng; Zou, Bin; Wu, Xuanxuan; Wang, Difei; Chen, Hui; Qiu, Hanli; Tang, Haotian; Kang, Kaiyi; Li, Qinling; Zhai, Xuan"

作者地址："[Wu, Yuxin; Zhang, Zaiyu; Dong, Xinyu; Hong, Siqi; Hu, Yue; Liang, Ping; Li, Lusheng; Zou, Bin; Wu, Xuanxuan; Wang, Difei] Chongqing Med Univ, Natl Clin Res Ctr Child Hlth & Disorders, Dept Neurosurg, Childrens Hosp,Minist Educ,Key Lab Child Dev & Dis, Chongqing, Peoples R China; [Wu, Yuxin; Zhang, Zaiyu; Dong, Xinyu; Hong, Siqi; Hu, Yue; Liang, Ping; Li, Lusheng; Zou, Bin; Wu, Xuanxuan; Wang, Difei] Chongqing Key Lab Translat Med Res Cognit Dev & Le, Chongqing, Peoples R China; [Qiu, Hanli] Univ Tubingen, MEG Ctr, Tubingen, Germany; [Qiu, Hanli] Univ Tubingen, Hertie Inst Klin Hirnforschung, Abt Neuronale Dynam & Magnetoenzephalog, Tubingen, Germany; [Li, Qinling; Zhai, Xuan] Chongqing Med Univ, Childrens Hosp, 136 Zhongshan 2nd Rd, Chongqing 400010, Peoples R China"

通信作者："Li, QL; Zhai, X (通讯作者)，Chongqing Med Univ, Childrens Hosp, 136 Zhongshan 2nd Rd, Chongqing 400010, Peoples R China."

来源：EPILEPSY & BEHAVIOR

ESI学科分类：NEUROSCIENCE & BEHAVIOR

WOS号：WOS:001174013600001

JCR分区：Q2

影响因子：2.3

年份：2024

卷号：151

期号：　

开始页：　

结束页：　

文献类型：Article

关键词：Artificial intelligence; ChatGPT; Epilepsy; Patient education; Language learning models

摘要："Objective: People with epilepsy desire to acquire accurate information about epilepsy and actively engage in its management throughout the long journey of living with seizures. ChatGPT is a large language model and we aimed to assess the accuracy and consistency of ChatGPT in responding to the common concerns of people with epilepsy and to evaluate its ability to provide emotional support. Methods: Questions were collected from the International League against Epilepsy and the China Association against Epilepsy. The responses were independently assessed by two board-certified epileptologists from the China Association against Epilepsy, and a third reviewer resolved disagreements. The reviewers assessed its ability to provide emotional support subjectively. Results: A total of 378 questions related to epilepsy and 5 questions related to emotional support were included. ChatGPT provided ""correct and comprehensive"" answers to 68.4% of the questions. The model provided reproducible answers for 82.3% questions. The model performed poorly in answering prognostic questions, with only 46.8% of the answers rated as comprehensive. When faced with questions requiring emotional support, the model can generate natural and understandable responses. Significance: ChatGPT provides accurate and reliable answers to patients with epilepsy and is a valuable source of information. It also provides partial emotional support, potentially assisting those experiencing emotional distress. However, ChatGPT may provide incorrect responses, leading users to inadvertently accept incorrect and potentially dangerous advice. Therefore, the direct use of ChatGPT for medical guidance is not recommended and its primary use at present is in patients education."

基金机构：National Natural Science Founda- tion [81971217]

基金资助正文：<BOLD>Funding</BOLD> This study was supported by the National Natural Science Founda- tion (Grant No. 81971217) .