Guidance for systematic reviews

This guide aims to round up the most frequently asked questions (FAQs) on conducting systematic reviews

How to start

A systematic review protocol should describe the rationale, hypothesis, and planned methods of the review. Systematic reviews should have a focused and clear research question. The overall protocol which outlines the study methodology should include as follows:

  1. Background
  2. Research question and aims
  3. Eligibility criteria for inclusion and exclusion
  4. Methods:
  • Search strategy: Databases, search terms
  • Selecting studies for inclusion
  • Quality assessment
  • Data extraction & analysis
  • Synthesis of results
  • Dissemination
  1. Time frame

The overall process of a systematic review is illustrated in the Figure below. 

 

Overall process of a systematic review

 

1. Search literature based on a search strategy: You should search relevant studies using keywords on library databases. 

2. All retrieved literature from databases: Once you have run search terms, you should export all of your results to a literature manager (e.g., EndNote), using a separate group storing the results from each database. 

3. Remove duplicates: Some duplicate studies should be removed before the screening. 

4. Title & Abstract screening: You examine titles and abstracts to remove irrelevant studies. 

5. Full-text screening: You examine the full text based on your eligibility criteria.

6. Data extraction & Quality assessment: From the full text, you extract relevant data and assess the risk of bias.

7. Data synthesis: With the extracted data, you synthesise the data. 

Before you run the searches for your systematic review, it is really critical that you do some groundwork. The key part of this groundwork is the development of a protocol, which is like a detailed project plan, that is agreed upon by everyone involved in the systematic review project.

Guidance for protocol development

Choosing an appropriate review type is the first step in resolving your research questions before beginning your review. Although you may want to do a systematic review, not all research questions are suitable for systematic review. We find that many researchers are interested to know what is the difference between a scoping review and a systematic review. 

Both systematic reviews and scoping reviews require a clearly defined research question: however, the depth and breadth of the two review types are quite different. While the breadth of systematic review is very narrow, it has great depth (it takes a deep look at the data), scoping reviews are broad but have less depth. In other words, research questions for scoping reviews have a broader focus. Both review types require transparent and rigorous methodology (having a protocol and eligibility criteria). However, scoping reviews do not require quality assessment, and their data extraction can be iterative, which is the main difference from systematic review.

Below, we list some useful materials that can help you decide whether to do a scoping review or a systematic review.

 

Deciding the type of review

Indications for systematic review are as follows:

  1. Uncover the international evidence
  2. Confirm current practice/address any variation/identify new practices
  3. Identify and inform areas for future research
  4. Identify and investigate conflicting results
  5. Produce statements to guide decision-making

Indications for scoping review are as follows:

  1. To identify the types of available evidence in a given field
  2. To clarify key concepts/definitions in the literature
  3. To examine how research is conducted on a certain topic or field
  4. To identify key characteristics or factors related to a concept
  5. As a precursor to a systematic review
  6. To identify and analyse knowledge gaps

 

References:

  • Booth, A. (2015). EVIDENT Guidance for Reviewing the Evidence: a compendium of methodological literature and websites. Available from: https://edin.ac/38lglJu 
  • Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020. Available from https://synthesismanual.jbi.global
  • Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018). What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol, 18(1), 5. doi:10.1186/s12874-017-0468-4 Available from: https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-017-0468-4
  • Sutton, A., Clowes, M., Preston, L. and Booth, A. (2019), Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J, 36: 202-222. https://doi.org/10.1111/hir.12276
  • Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J, 26(2), 91-108. doi:10.1111/j.1471-1842.2009.00848.x Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1471-1842.2009.00848.x/full
  • Munn, Z., Peters, M.D.J., Stern, C. et al. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 18, 143 (2018). https://doi.org/10.1186/s12874-018-0611-x

All systematic review requires at least 3 researchers to review the potentially included studies.

At least two reviewers should screen, select, assess quality, and collect data independently to maintain transparency and reduce potential bias. The third researcher will resolve any conflicts between the two reviewers.  

For publishing your research, it is essential that more than one person undertake systematic reviews. However, you might be able to do your systematic reviews on your own, which is only for your dissertation but for aiming for publication.

It's very variable depending on the scope and volume of the relevant literature, but estimates of the average time to conduct a systematic review range from 6-18 months.