Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography.

TitleTools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography.
Publication TypeJournal Article
Year of Publication2007
AuthorsSanderson S, Tatt ID, Higgins JPT
JournalInternational journal of epidemiology
Volume36
Issue3
Pagination666-76
Date Published2007 Jun
ISSN0300-5771
KeywordsBias (Epidemiology); Epidemiologic Studies; Evidence-Based Medicine; Humans; Observation; Research Design
Abstract

BACKGROUND: Assessing quality and susceptibility to bias is essential when interpreting primary research and conducting systematic reviews and meta-analyses. Tools for assessing quality in clinical trials are well-described but much less attention has been given to similar tools for observational epidemiological studies.

METHODS: Tools were identified from a search of three electronic databases, bibliographies and an Internet search using Google. Two reviewers extracted data using a pre-piloted extraction form and strict inclusion criteria. Tool content was evaluated for domains potentially related to bias and was informed by the STROBE guidelines for reporting observational epidemiological studies.

RESULTS: A total of 86 tools were reviewed, comprising 41 simple checklists, 12 checklists with additional summary judgements and 33 scales. The number of items ranged from 3 to 36 (mean 13.7). One-third of tools were designed for single use in a specific review and one-third for critical appraisal. Half of the tools provided development details, although most were proposed for future use in other contexts. Most tools included items for selection methods (92%), measurement of study variables (86%), design-specific sources of bias (86%), control of confounding (78%) and use of statistics (78%); only 4% addressed conflict of interest. The distribution and weighting of domains across tools was variable and inconsistent.

CONCLUSION: A number of useful assessment tools have been identified by this report. Tools should be rigorously developed, evidence-based, valid, reliable and easy to use. There is a need to agree on critical elements for assessing susceptibility to bias in observational epidemiology and to develop appropriate evaluation tools.

DOI10.1093/ije/dym018
Alternate JournalInt J Epidemiol