التفاصيل البيبلوغرافية
| العنوان: |
A critical assessment of using ChatGPT for extracting structured data from clinical notes. |
| المؤلفون: |
Huang, Jingwei, Yang, Donghan M., Rong, Ruichen, Nezafati, Kuroush, Treager, Colin, Chi, Zhikai, Wang, Shidan, Cheng, Xian, Guo, Yujia, Klesse, Laura J., Xiao, Guanghua, Peterson, Eric D., Zhan, Xiaowei, Xie, Yang |
| المصدر: |
NPJ Digital Medicine; 5/1/2024, Vol. 7 Issue 1, p1-13, 13p |
| مصطلحات موضوعية: |
ARTIFICIAL intelligence tests, OSTEOSARCOMA, DATABASES, MEDICAL information storage & retrieval systems, TERMS & phrases, ARTIFICIAL intelligence, RESEARCH evaluation, EVALUATION of organizational effectiveness, DATA curation, NATURAL language processing, DECISION making in clinical medicine, PROBLEM solving, DESCRIPTIVE statistics, WORKFLOW, INFORMATION retrieval, MEDICAL records, CONCEPTUAL structures, LUNG tumors, TUMOR classification, ALGORITHMS |
| مستخلص: |
Existing natural language processing (NLP) methods to convert free-text clinical notes into structured data often require problem-specific annotations and model training. This study aims to evaluate ChatGPT's capacity to extract information from free-text medical notes efficiently and comprehensively. We developed a large language model (LLM)-based workflow, utilizing systems engineering methodology and spiral "prompt engineering" process, leveraging OpenAI's API for batch querying ChatGPT. We evaluated the effectiveness of this method using a dataset of more than 1000 lung cancer pathology reports and a dataset of 191 pediatric osteosarcoma pathology reports, comparing the ChatGPT-3.5 (gpt-3.5-turbo-16k) outputs with expert-curated structured data. ChatGPT-3.5 demonstrated the ability to extract pathological classifications with an overall accuracy of 89%, in lung cancer dataset, outperforming the performance of two traditional NLP methods. The performance is influenced by the design of the instructive prompt. Our case analysis shows that most misclassifications were due to the lack of highly specialized pathology terminology, and erroneous interpretation of TNM staging rules. Reproducibility shows the relatively stable performance of ChatGPT-3.5 over time. In pediatric osteosarcoma dataset, ChatGPT-3.5 accurately classified both grades and margin status with accuracy of 98.6% and 100% respectively. Our study shows the feasibility of using ChatGPT to process large volumes of clinical notes for structured information extraction without requiring extensive task-specific human annotation and model training. The results underscore the potential role of LLMs in transforming unstructured healthcare data into structured formats, thereby supporting research and aiding clinical decision-making. [ABSTRACT FROM AUTHOR] |
|
Copyright of NPJ Digital Medicine is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) |
| قاعدة البيانات: |
Complementary Index |
Full Text Finder