Exploring ChatGPT's Efficacy in Pathophysiological Analysis: A Comparative Study of Ischemic Heart Disease and Anaphylactic Shock Cases

Abstract

Physiopathology is one of the basic disciplines which helps to understand the mechanisms of diseases and, accessing effective and personalized therapies. The purpose of this work was to compare the answers provided by human intelligence with experience and professional qualification with the answers provided by ChatGPT, regarding the targeted questions addressed to two clinical cases, to decipher the key pathophysiological mechanisms. Two cases published in an applied pathophysiology book represented the input data. ChatGPT-3.5 was used in our study to retrieve responses to five specific questions per case and GPTZero to score the responses. The input and output text were in Romanian language, and we collected the answers on August 7, 2023. Two researchers independently did the classification of the answers provided by ChatGPT as correct, partially correct, or incorrect. The ChatGPT generated answers were partially correct in 4 out of 5 questions and incorrect in one question in both cases. The number of words generated by ChatGPT was similar to the response given by professionals and the perplexity, burstiness, readability and simplicity have smaller scores. GPTZero did not identify the responses generated by ChatGPT as generated by a large language model. ChatGPT-3.5 generated satisfactory responses but not complete in most of the cases, but must be supervised by experts to ensure the validity of the responses and explanations.