Harnessing and Misapplying Artificial Intelligence in Medical Evidence Search

Abstract

The 6S pyramid, a conceptual model that systematically categorizes evidence-based resources according to their type, to facilitate rapid and efficient clinical decision-making, is a cornerstone of evidence-based practice. Two main area can be seen in the 6S pyramid with the original studies evidence at the bottom of pyramid and five classes of summaries of evidence (pre-appraisal secondary sources) at the top (Systems, Computerised Decision Support Systems – Summaries, Clinical practice guidelines – Synopsis of Synthesis, summarises findings from high quality systematic reviews – Synthesis, systematic review - Synopsis of Studies, critically appraised brief summaries of original studies) [1].  Artificial intelligence (AI) has emerged as a transformative agent in the retrieval of medical evidence, offering both benefits and pitfalls. Artificial intelligence-driven information retrieval tools offer timely access to information, but misapplication can lead to biased or erroneous retrieval outcomes (for example, biased AI-recommendations in an image-based diagnosis simulation experiment [2]). Considering the amount of evidence that is published yearly in medical scientific literature, AI-tools could be useful in searching and retrieving medical evidence. Healthcare professionals access evidence summaries through dedicated professional platforms such as ClinicalKey, UpToDate, and guidelines (e.g., NICE, NHMRC Clinical Practice Guidelines Portal, National Clinical Guidelines Clearinghouse, etc.). Dedicated AI-assisted search platforms have been developed to support rapid access to medical evidence. Some examples of such platforms are OpenEvidence and EvidenceHunt. OpenEvidence (https://www.openevidence.com/) is platform develope “to aggregate, synthesize, and visualize clinically relevant evidence in understandable, accessible formats that can be used to make more evidenced-based decisions and improve patient outcomes”. OpenScience signed a content agreement with The New England Journal of Medicine, a highly prestigious, often ranked at or near the top regarding imapact factor and influence. EvidenceHunt (https://evidencehunt.com) is an AI-powered search engine able to streamline clinical evidence searches that allow users to customizable e-alerts on the topic of interest. In this presentation, simulations based on case vignettes are presented to highlight potential and challenges of AI-driven retrieval of medical evidence compared to traditional access to medical evidence.