6.3. Checklists

We have found four papers (Reynolds, 2011; Walsh, 2006; Cohen, 2008; Côté and Turgeon, 2005) reviewing the literature on quality criteria or guidelines for qualitative research. One of them (Walsh, 2006) provides us with a synthesis of eight existing checklists and summary frameworks (see Table 7). This checklist is quite detailed and is designed in function of meta-synthesis, which is a kind of systematic review of qualitative research papers.

The list of criteria was built in order to rigorously appraise studies first before submitting them to the meta-synthesis technique. Agreement on criteria to judge rigor was necessary in order to decide which studies to include in the meta-synthesis. Walsh and Downe (Walsh, 2006) tabulated the characteristics mentioned in each of the papers in their review. Then they mapped together the characteristics given in all the included papers, sorting them by the number of checklists in which they appeared. In the next step both authors independently attempted a synthesis before coming together to discuss. Redundant criteria were excluded if both authors agreed that the exclusion would not change the final judgment on the meaningfulness and applicability of a piece of qualitative research. Finally the table below was constructed, structured into three columns, namely stages, essential criteria and specific prompts. Although some criteria may seem self-evident, others are less obviously fundamental (Walsh, 2006). This list of criteria is very detailed. In some studies, especially those with short time frame, a shorter and more pragmatic hands-on list could be practical. Therefore we also added the grid of  Côté and Turgeon [c] (Table 8) which is shorter, adapted to the specific context of heath care and easier to use for researchers who are less familiar with qualitative research. Other checklists are described in Appendix 1.

The use of a checklist may improve qualitative research, however they should be used critically: not every criterion is appropriate to every research context (Barbour, 2001). For example the list of Coté and Turgeon mentions interpretation of results in an innovative way as a quality criterion (point 10, Table 8), while this is not necessarily the case. Most important is a systematic approach during research process. For example the credibility of data analysis could encompass the use of software (Table 7), triangulation and/or member checking (point 7, Table 8), whereas a systematic approach with a detailed description of each step in the research process could have been sufficient.

 

Table 7 – Summary criteria for appraising qualitative research studies

Stages

Essential criteria

Specific prompts

Scope and purpose

Clear statement of, and rationale for, research question / aims / purposes

  • Clarity of focus demonstrated
  • Explicit purpose given, such as descriptive/explanatory intent, theory building, hypothesis testing
  • Link between research and existing knowledge demonstrated

 

Study thoroughly contextualized by existing literature

  • Evidence of systematic approach to literature review, location of literature to contextualise the findings, or both

Design

Method/design apparent, and consistent with research intent

  • Rationale given for use of qualitative design
  • Discussion of epistemological/ontological grounding
  • Rationale explored for specific qualitative method (e.g. ethnography, grounded theory, phenomenology)
  • Discussion of why particular method chosen is most appropriate/sensitive/relevant for research question/aims
  • Setting appropriate

 

Data collection strategy apparent and appropriate

  • Were data collection methods appropriate for type of data required and for specific qualitative method?
  • Were they likely to capture the complexity/diversity of expereince and illuminate context in sufficient detail?
  • Was triangulation of data sources used if appropriate?

Sampling strategy

Sample and sampling method appropriate

  • Selection criteria detailed, and description of how sampling was undertaken
  • Justification for sampling strategy given
  • Thickness of description likely to be achieved from sampling 
  • Any disparity between planned and actual sample explained 

Analysis

Analytic approach appropriate

  • Approach made explicit (e.g. thematic distillation, constant comparative method, grounded theory)
  • Was it appropriate for the qualitative method chosen?
  • Was data managed by software package of by hand and why?
  • Discussion of how coding systems/conceptual frameworks evolved
  • How was context of data retained during analysis
  • Evidence that the subjective meanings of participants were portrayed
  • Evidence of more than one researcher involved in stages if appropriate to epistemological/theoretical stance
  • Did research participants have any involvement in analysis (e.g. member checking)
  • Evidence provided that data reached saturation or discussion/rationale if it did not
  • Evidence that deviant data was sought, or discussion/rationale if it was not

Interpretation

Context described and taken account of in interpretation

  • Description of social/physical and interpersonal contexts of data collection
  • Evidence that researcher spent time ‘dwelling with the data’, interrogating it for competing/alternative explanations of phenomena

 

Clear audit trail given

  • Sufficient discussion of research processes such that others can follow ‘decision trail’

 

Data used to support interpretation

  • Extensive use of field notes entries/verbatim interview quotes in discussion of findings
  • Clear exposition of how interpretation led to conclusions

Reflexivity

Researcher reflexivity demonstrated

  • Discussion of relationship between researcher and participants during fieldwork
  • Demonstration of researcher’s influence on stages of research process
  • Evidence of self-awareness/insight
  • Documentation of effects of the research on researcher
  • Evidence of how problems/complications met were dealt with

Ethical dimensions

Demonstration of sensitivity to ethical concerns

  • Ethical committee approval granted
  • Clear commitment to integrity, honesty, transparency, equality and mutual respect in relationships with participants
  • Evidence of fair dealing with all research participants
  • Recording of dilemmas met and how resolved in relation to ethical issues
  • Documentation of how autonomy, consent, confidentiality, anonymity were managed

Relevance and transferability

Relevance and transferability evident

  • Sufficient evidence for typicality specificity to be assessed
  • Analysis interwoven with existing theories and other relevant