Factor Analysis in Assessing the Research Methodology Quality of Systematic Reviews

Abstract

Introduction: Many high quality systematic reviews available from medical journals, data bases and other electronic sources differ in quality and provide different answers to the same question. The literature recommended the use of a checklist type approach, which exceeds many of the problems associated with measurements. Aim: This study proposes to identify in a checklist type approach the most commonly used factors (from a methodological point of view) in assessing the quality of systematic reviews, and then mirror the actual stage of medical writing. We want to analyze the factors’ occurrence and / or their development in the text and in the abstract of systematic reviews published in 2011. Methods: The present study randomly selected only free full text systematic reviews published in 2011, systematic reviews found in Pubmed and in Cochrane Database. The most commonly used factors were identified in PRISMA statement and quality measurement tools. Results: The evaluated systematic reviews mentioned or developed several of the factors studied. Only 78% of the papers surveyed have used the correct IMRAD format and 59% of them have mentioned the sample size used. The correspondence between the content of the paper and its abstract is summarized in the proportion of 54.63% and 51.85% for the two sets of factors, and it can lead to scarce appreciation of the article provided that only abstracts are read. Conclusions: Researchers do not properly take into consideration scientific articles and assessment tools used for quality evaluation. They should place more value over methodological factors which help assess systematic review quality, while journals form the only party who can enforce quality standards in medical writing.