What is it?
- a method for studying complex interventions in response to the perceived limitations of conventional systematic review methodology
- Involves identification of Contexts, Mechanisms and Outcomes for individual programs to explain differences, intended or unintended, between them
- "traditional methods of review focus on measuring and reporting on program effectiveness, often find that the evidence is mixed or conflicting, and provide little or no clue as to why the intervention worked or did not work when applied in different contexts or circumstances, deployed by different stakeholders, or used for different purposes"
(p. 21)
Pawson et al. (2005)
Outline of Stages
Timeframe:
Question:
- Answers complex questions in a manner that attempts to explain rather than pass judgement.
- Expect question focusing to be a time-consuming and ongoing task as you learn more about the topic over the course of the review
Sources and searches:
- These four steps can be a general outline on how to think about the searching process, but keep in mind that your process may be quite circuitous:
- A background search to see what's out there
- Focus the scope of the review by identifying "the administrative thinking, policy history, legislative background, key points of contention in respect of the intervention, and so on." (Pawson et al., 2005)
- Formally search for primary studies to back up the theories (this search should be properly documented)
- One final search near the end of evidence compilation to find additional studies that add to the theories
Selection:
- Since the method of searching is purposeful rather than comprehensive, compiling resources is more about actively seeking relevant materials rather than systematically gathering a large number of articles and excluding irrelevant ones
Appraisal:
- Realist reviews rejects the hierarchy of evidence (i.e. with RCTs being above non-RCTs)
- Complex service interventions are influenced by other programs
- Policies are rarely adopted instantly in its entirety thereby defining a cut-off between control and intervention near impossible
- Appraisal checklists are usually not used in realist reviews
- Realist reviews which favors judgement by the reviewer to appraise articles
Synthesis:
- Involves taking the preliminary understanding of a theory and applying the evidence to refine the theory.
- A realist review tries to provide more explanation and detail to a theory or intervention
(Pawson et al., 2005)
Limitations
-
Finding balance: According to Rycroft-Malone et al. (2012), "the challenge of developing a framework for a realist synthesis is in finding a level of abstraction that allows reviewers to stand back from the detail and variation in the evidence, but that is also specific enough to meet the purpose of the review" (p. 3).
-
Rycroft-Malone et al. (2012) state that conducting a realist review "is not an easy option. Realist review[s] demands much of the reviewer, including an ability to think flexibly and deal with complexity" (p. 9).
-
Pawson et al. (2005) also state that realist reviews can't be applied in a wide variety of contexts, explaining that "realist review[s] cannot be used as a formulaic, protocol-driven approach. Realist review[s] [are] more about principles that guide than rules that regularize" (p. 32).
-
Pawson et al. (2005) also add that realist reviews are "not standarizable or reproducible the same sense as a conventional Cochrane review" (p. 32) and that realist reviews often only lead to, at the most, tentative recommendations. As well, it is important to note that realist reviews require considerable skill to complete and to date have not been published as much as other review types.
Credit:
Format and content adapted from Temple University Health Science Libraries
