Beyond Medline: reducing bias through extended systematic review search.

TitleBeyond Medline: reducing bias through extended systematic review search.
Publication TypeJournal Article
Year of Publication2003
AuthorsSavoie I, Helmer D, Green CJ, Kazanjian A
JournalInternational journal of technology assessment in health care
Volume19
Issue1
Pagination168-78
Date Published2003 Winter
ISSN0266-4623
KeywordsBias (Epidemiology); Bibliometrics; Databases, Bibliographic; Humans; Information Storage and Retrieval; MEDLINE; Randomized Controlled Trials as Topic; Review Literature as Topic; Sensitivity and Specificity
Abstract

OBJECTIVE: To evaluate the sensitivity and precision of various extended search methods in identifying randomized controlled trials (RCTs) for systematic reviews.

METHOD: Prospective analysis of extended search methods (specialized databases or trial registries, reference lists, hand-searching, personal communication, and Internet) used in two systematic reviews of RCTs. The gold standard was the total number of RCTs identified by major databases (MEDLINE, EMBASE, etc.) and extended search strategies combined. Sensitivity was the proportion of all known RCTs identified by any extended search method. Precision reflected the proportion of all items uncovered by any extended search method that actually were RCTs.

RESULTS: The extended search identified 94 additional RCTs for the systematic reviews beyond those identified with the major databases. Specialized databases and trial registries had the highest sensitivity and precision for the lipid-lowering project (13.6% and 52.7%, respectively; p < .05) followed by scanning of reference lists (7.2% sensitivity and 41.9% precision; p <.05). Hand-searching was more effective than personal communication and Internet searching (1.7% sensitivity and 12.2% precision; p < .05). The acupuncture project had slightly different results, with the specialized databases and trial registries tied with the review of reference lists for highest sensitivity (14.2%). The precision followed the same trend as the lipid-lowering project (17.6% specialized databases; 8.3% reference lists; p < .05). A post-hoc analysis showed that 75 of the 94 RCTs were indexed in the major databases but missed by the major database search.

CONCLUSIONS: Extended searching identified additional RCTs for the systematic reviews beyond those found in major databases. Specialized databases and trial registries were most effective. An important number of RCTs were missed by the major database search. Timing and accuracy of indexing may explain this finding. The definitive measure, whether there is an association between the method used to uncover RCTs, the quality of the items uncovered and their impact on systematic review results, is yet to be determined.

Alternate JournalInt J Technol Assess Health Care