Skip to Main Content

Knowledge Syntheses: A How-To Guide

Overview of systematic review steps and resources to assist researchers conducting reviews

What is a Protocol and Why is it Important?

"Systematic reviews should build on a protocol that describes the rationale, hypothesis, and planned methods of the review. Detailed, well-described protocols can facilitate the understanding and appraisal of the review methods, as well as the detection of modifications to methods and selective reporting in completed reviews" (Moher, 2015).
Protocol registration or publishing

Systematic review standards require prospectively registering the protocol prior to conducting the review. PROSPERO is one option for health-related systematic reviews. Some review teams may also choose to publish their protocol in a topic-related or synthesis methods journal following peer-review. Cochrane, JBI, and some other synthesis organizations have internal peer-review processes for the review protocol. Registration of the protocol in Open Science Framework (OSF) or another topic or institutional research repository are also options for protocols of other types of reviews, such as scoping reviews, or in other disciplines outside of health.

Research Data and Project Management Planning for Reviews

Sections to Include in a Protocol

Title

Identify the report as a protocol for the type of review you're conducting (systematic, scoping, rapid, etc.)

 

Registration

Where did you register your protocol? OSF? PROSPERO? Other?

 

Authors

  • Who is conducting the review? 
  • Who is part of your team?
  • List all contributors and their roles
     
  • What did everyone do?
  • A helpful tool to determine contributions in an unbiased way is to use the CRediT tool

 

Support

  • Who's the review funder/sponsor?
  • What's their role in the review?

 

Introduction

Rationale

  • What's your reason for producing the review?
  • What are you trying to achieve?
  • What gap in knowledge is this covering?

 

Objectives/Review Question

  • What specific question is your review addressing?
  • Identify the participants, interventions, exposure, issue, comparators & outcomes.

Covidence has written a blog post that can help you begin to think about formulating your review question.

 

Method

Eligibility Criteria

  • What criteria are you using to include and exclude studies from your review?
  • E.g. Study design, setting, time frame, date, language, publication status, participants characteristics?
     

 

Source: https://unimelb.libguides.com/c.php?g=492361&p=3368110

 

Information Sources

  • What information sources are you gathering information from?
  • E.g. list the databases, study authors, trial registers and grey literature & the dates of coverage you're searching
  • If you have a draft for your search, provide it in an Appendix

 

Study Selection Process

  • What will your process for selecting studies entail such as having two independent reviewers?
  • Describe the process for EACH phase of your review
    • (e.g. screening, eligibility & inclusion in meta-anaylsis)

 

Data Collection and Management

  • How will you extract data from reports?
  • How will you obtain & confirm data from investigators? 
     
  • How are you going to manage your records & data during your review process?
  • List & define all outcomes for which data will be sought, including prioritization of main & additional outcomes, with rationales

 

Risk of Bias

  • How will you assess risk of bias in individual studies (will this be done at outcome or study level or both)? 

 

Data Synthesis

  • What criteria will the study data be quantitatively synthesized? 
  • Is there any proposed additional analyses (e.g. sensitivity, subgroup analyses, meta-regression)?
  • If not a quantitative synthesis (i.e. qualitative synthesis), describe your method for summarizing data

 

 

Example Protocols

  1. Xiao Z, Baldwin MM, Meinck F, Obsuth I, Murray AL. The impact of childhood psychological maltreatment on mental health outcomes in adulthood: a protocol for a systematic review and meta-analysis Syst Rev. 2021;10(1):224. Published 2021 Aug 12. doi:10.1186/s13643-021-01777-4.
  1. Milne-Ives M, Swancutt D, Burns L, et al. The Effectiveness and Usability of Online, Group-Based Interventions for People With Severe Obesity: Protocol for a Systematic Review. JMIR Res Protoc. 2021;10(6):e26619. Published 2021 Jun 30. doi:10.2196/26619. https://www.researchprotocols.org/2021/7/e28083/

 

Reused with permission from: https://guides.library.utoronto.ca/c.php?g=713309&p=5087816

 

Title

Authors

First author name1 Second author nameThird author name3 Fourth author name4 Fifth author name5 Sixth author name6

1.  Affiliation
2.  Affiliation
3.  Affiliation
4.  Affiliation
5.  Affiliation
6.  Affiliation

Abstract

(Maximum - 250 words for Protocols / 500 words for Scoping Reviews)

Objective:

  • State an overarching review objective structured using the key components of the inclusion criteria
    (approximately one or two sentences).
  • e.g. The objective of this scoping review is to understand the extent and type of evidence in relation to
    (insert field)  

Introduction:

  • Briefly describe the rationale for the review considering what is already known on the topic
  • (approximately two to three sentences)

Inclusion criteria:

  • Summarize the inclusion criteria using the participants, concept, and context (PCC framework) and highlight any relevant exclusions in paragraph format
  • Present the information in one to three sentences – NOT under individual subheadings

Methods:

  • List the key information sources searched/to be searched
    (
    those from which the majority of evidence sources were/will be located)
  • the date (month/year) the search was conducted (for reviews only)
  • any search limits (e.g. language)
  • Briefly describe the approach to study selection, data extraction, analysis of the evidence and presentation of the results
  • Briefly describe any notable deviations to the methodological approach taken (for reviews only)

Results (For Reviews ONLY):

  • The bulk of the abstract should be reserved to convey the main results of the review in relation to the objective/question
  • Report the number and type of included evidence as well as any pertinent study characteristics

Conclusions (For Reviews ONLY):

  • Provide a conclusion based on a general interpretation of the review findings in line with the review’s objective/s and any limitations of the review
  • Briefly convey key implications of the findings for practice and research (if any)

 

Introduction

Guidance for authors:

  • Describe the rationale for the review considering what is already known on the topic, including information that supports and justifies the selection of inclusion criteria
  • Key terms should be defined
  • operational definitions narratively explained
  • provide some indication that there is evidence available that will meet your inclusion criteria
  • A rationale as to why a scoping review was the most appropriate method should also be provided
    (Approximately 1000 words)
  • If there are any existing systematic reviews/scoping reviews, it should be specified how the proposed review will differ
  • Conclude with an overarching review objective, that captures and aligns with the core elements/PCC framework of the inclusion criteria
    • e.g. The objective of this scoping review is to assess the extent of the literature (insert area/field)

Review question

  • Clearly state the review question or questions using the PCC framework
    (i.e. additional or sub-questions) that the review seeks to answer)
  • The review question/s should clearly relate to the objective/s

Keywords

  • List a maximum of five keywords in alphabetical order, separated by a semi-colon and a space
  • Ideally these words should be different to those that appear in the title and abstract
  • These are for the purposes of meta-data and indexing, and not related to the search strategy

Eligibility criteria

Participants

  • Describe/define participants that will be included
  • Specific exclusion criteria based on any participant characteristics should also be stated

Concept

  • Describe and/or define the concept
  • Specific exclusion criteria based on any concept should also be stated

Context

  • Consider:
    • cultural/sub-cultural factors
    • geographic location
    • specific racial or gender-based interests
    • details about the specific setting
  • Specific exclusion criteria based on any context should also be stated.

Types of Sources

 

Methods

Example Text:

The proposed scoping review will be conducted in accordance with the JBI methodology for scoping reviews (insert a citation to the methodology ).

Search strategy

  • Studies published in any language (modify as appropriate) will be included
  • Studies published since insert date will be included as (justify date range and any language limitations)
  • List all information sources (e.g. electronic databases, contact with study authors etc.)

Example Text:

The databases to be searched include (insert databases with platforms as appropriate). Sources of unpublished studies/ gray literature to be searched include (insert text, e.g. trial registers etc).

Study/Source of Evidence selection

Example Text:

Following the search, all identified citations will be collated and uploaded into (insert the name of the bibliographic software or citation management system e.g. EndNote insert version /year (Clarivate Analytics, PA, USA)) and duplicates removed. Following a pilot test, titles and abstracts will then be screened by two or more independent reviewers for assessment against the inclusion criteria for the review. Potentially relevant sources will be retrieved in full and their citation details imported into the JBI System for the Unified Management, Assessment and Review of Information (JBI SUMARI) (JBI, Adelaide, Australia) (insert citation to JBI SUMARI paper). The full text of selected citations will be assessed in detail against the inclusion criteria by two or more independent reviewers. Reasons for exclusion of sources of evidence at full text that do not meet the inclusion criteria will be recorded and reported in the scoping review. Any disagreements that arise between the reviewers at each stage of the selection process will be resolved through discussion, or with an additional reviewer/s. The results of the search and the study inclusion process will be reported in full in the final scoping review and presented in a Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram (insert citation to PRISMA-ScR statement and include in the reference list).

 

Data Extraction

Example Text:

Data will be extracted from papers included in the scoping review by two or more independent reviewers using a data extraction tool developed by the reviewers. The data extracted will include specific details about the participants, concept, context, study methods and key findings relevant to the review question/s.

  • Discuss the planned piloting of the draft extraction form in the protocol, and how this was conducted in the review

Example Text:

A draft extraction form is provided (see Appendix XX). The draft data extraction tool will be modified and revised as necessary during the process of extracting data from each included evidence source. Modifications will be detailed in the scoping review. Any disagreements that arise between the reviewers will be resolved through discussion, or with an additional reviewer/s. If appropriate, authors of papers will be contacted to request missing or additional data, where required. 

  • Critical appraisal of individual sources of evidence is generally not required for scoping reviews
  • If it will be done, provide a rationale as to why and describe the methods, including tools which will be used

 

Data Analysis and Presentation

  • The evidence presented should directly respond to the review objective and question(s)

  • The data is commonly presented graphically or in diagrammatic or tabular form

  • Preparation of the review protocol is the opportunity for authors to pilot and determine how to best present their data or map and provide detailed description for the reader

  • Insert information on data presentation/mapping techniques, if any

  • A narrative summary will accompany the tabulated and/or charted results and will describe how the results relate to the reviews objective and question/s

 

Acknowledgements

  • Insert the full names and precise contributions of individuals, or institutions, who have not already been listed as co-authors
  • Specify if this review is to contribute towards a degree award and for which author (initials)
  • The acknowledgement section must be included in the submission title page to facilitate the double-blind peer-review process
  • For further guidance, please refer to the JBI Evidence Synthesis manuscript style and preparation guidelines

 

Funding

  • Provide details on sources of funding for the review
  • Explicitly describe the role of funders in the review process
  • For further guidance, please refer to the JBI Evidence Synthesis manuscript style and preparation guidelines

 

Conflicts of interest

  • Include a statement that describes a potential conflict of interest or any personal, financial, professional, or intellectual bias for any of the authors listed on the manuscript
  • If no conflict exists, include the following statement: "There is no conflict of interest in this project."

 

References

Appendices

Appendix I: Search strategy

  • For protocols - present a full search strategy for at least one electronic database including planned limits, such that it can be reviewed and repeated
  • For systematic reviews, all search strategies should be presented

 

Appendix II: Data extraction instrument

  • Only append the JBI or non-JBI data extraction instrument if the standardized tool has been modified in any way, otherwise simply cite the tool used in the text
  • Any modifications made to the instrument should also be described in the text

 

 

Example Protocols

  1. Scoping reviews: Lackie, Kelly, Miller, Stephen, Ayn, Caitlyn, et al. Interprofessional collaboration between health professional learners when breaking bad news: a scoping review protocol. JBI Evidence Synthesis. Lippincott Williams & Wilkins: Hagerstown, MD. 2021;19(8):2032-2039. doi:10.11124/JBIES-20-00239. https://sfxhosted.exlibrisgroup.com/dal?sid=Entrez%3APubMed&id=pmid%3A33882558&issn=2689-8381
  1. Cassidy CE, Beck AJ, Conway A, et al. Using an integrated knowledge translation or other research partnership approach in trainee-led research: a scoping review protocol. BMJ Open. 2021;11(5):e043756. Published 2021 May 25. doi:10.1136/bmjopen-2020-043756 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8154946/

Review Management Software

Review management software tools are specifically tailored to the needs of knowledge synthesis teams. In addition to reference management, some of these tools can also help with data extraction, perform meta-analysis, track team progress, and facilitate communication between members. As indicated below, some of these tools are fee-based. You should also bear in mind that not every tool is appropriate for every kind of synthesis or review - be sure to choose the right fit for your project.