Insights from Self-Reported Student Experiences on using ChatBots

Abstract

Background and Aim: The development of conversational artificial intelligence tools (ChatBots) provide new ways to access information. Our study aimed to explore if and how medical students utilize chatbots to understand the role of artificial intelligence-powered tools in medical education and their implications for future training and practice. Materials and Methods: The target population was represented by medical students, regardless of specialization, who study at the Iuliu Haţieganu university of Medicine and Pharmacy Cluj-Napoca, Romania. An online questionnaire was developed, and students were invited to participate. The questionnaire was open between 10 October 2024 and 31 January 2025. Results: Ninety-two students, half up to 21 years, 74% female filled the survey. Sixteen percentage of respondents does not use ChatBots, with similar frequency used for learning (45%) or searching medical information (39%). The top three ChatBots are ChatGPT (84%), Gemini (11%) and Copilot (5%). Half of those who use ChatBots (47, 51.1%) considered that ChatBots help them much or very much to understand medical concepts and pathologies. Most participants (61, 79%) considered that the used of ChatBots positively influenced their abilities to ask educational questions. While majority of respondents acknowledged that the retrieved information is sometimes accurate (72, 94%), only 41.3% of respondents check the accuracy at least in 75% of cases; 62% did not check the accuracy at the last interrogation. ChatBots are seen as tools with impact on approaching academic tasks, providing clear and quick explanations to questions (60.9%), organizing information more efficiently (50.0%), or managing time for academic tasks (40.2%). Most respondents (59%) would published the results generated by ChatBots in personal projects or academic papers. Most respondents seen the ChatBots as useful “as long as we do not take over the content word for word, verifying the data and through other reliable sources”. Conclusion: Our results showed that undergraduate medical students used ChatBots, recognize that the retrieved information is only sometimes accurate but unfortunately, they verify the accuracy in less than half interrogations. The main acknowledge benefit is clear and quick explanations to specific questions.