Guidance for systematic reviews

The Cochrane Collaboration is a leading group in the production of evidence synthesis and systematic reviews.
According to the Cochrane Collaboration, "a systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias in order to produce more reliable findings that can be used to inform decision making."
(Chapter 1: The Cochrane Handbook)

If you are unsure about what is the best type of review for your research questions, contact your Academic Support Librarian team

or have a look at this chapter by Boland et al (2017): Scoping review versus Systematic review

Reference: Boland, A. et al. (2017) Doing a systematic review: a student's guide / edited by Angela Boland, M. Gemma Cherry, Rumona Dickson. 2nd edition. Los Angeles: SAGE.

Systematic review searches need to be comprehensive, transparent, and reproducible, which means they should be of high quality without errors and well structured. The search strategy should be peer-reviewed, and the final search strategy should be tested.

Before finalising your search plan, it will help to read good-quality published systematic reviews on topics related to your own research question to see what methods they employed, and you’ll need to do quite a bit of exploratory testing.

Some questions to ask yourself when considering your search strategy:

 1.  Write out your systematic review research question
 2.  What type(s) of data do you need to address your question?
 3.  What study design(s) would yield those data? Or what types of publications or reports are most likely to contain the data you need to address your question?
 4.  Where are those publications or reports likely to be published? e.g in which journals, or by which organisations?
 5.  If you think there might be unpublished but relevant reports, which organisations or people do you think would know about them?
 6.  What literature databases, research registers, or organisation websites are most likely to contain the records (or full reports) of the publications you need?

More details are available here: LibSmartII module on Literature Searching for Systematic Reviews

Choosing appropriate databases for search is very important as it determines the comprehensiveness of systematic reviews.

The number of databases you use to search will vary depending on the research query, but it is important to use multiple databases to mitigate database bias and publication bias. More important than the number of databases is using the appropriate databases for the subject to find all the relevant data.

 •  "Databases by Subject guides" to help you get the best out of the library with information about finding academic literature, referencing and more.

Including all possible search terms does not necessarily increase the sensitivity of your search. If search terms contain common words (i.e. development, disease) rather than keywords, it will retrieve too many irrelevant records. Therefore, search terms should be peer reviewed before they are run. Peer review of search strategies is highly recommended by many organisations including Cochrane collaboration.

More detailed information is available here: Peer review search strategies

Many researchers are confused with scoping reviews from systematic reviews. Both share common traits but the breadth and depth of information are very different from each other.

 • Firstly, while systematic reviews require well-focused research questions, scoping reviews are appropriate for research questions that are broader in focus.
 • Secondly, both require explicit and rigorous methodologies but they can be iterative in scoping reviews.
 • Thirdly, while quality assessment (risk of bias assessment) is mandatory in systematic reviews, it is unlikely in scoping reviews.

More details are available here:
Scoping review vs systematic review
SAGE research methods

According to the Cochrane Handbook, grey literature usually mean literature not formally published in sources such as books or journal articles. The term ‘grey literature’ is often used to refer to reports published outside of traditional commercial publishing. Review authors should generally search sources such as dissertations and conference abstracts ( see Chapter 6: Cochrane Handbook)

In addition it is important to identify key organisations or professional bodies in your subject and their publications or research.

Sources of grey literature

Grey literature databases by Cochrane Handbook

Advanced search in Google

Domain Searching site in Google:.gov, site:gov.uk or site:who.int in a domain to search reports from national/international organisations

If you have too many/small records, it could be due to a number of reasons. There are several points you might need to check.

1. Did you apply filters?
 • Please do a comparison of the results before/after applying filters.
 • Remove unnecessary filters
 • Reconsider applying filters
2. Your search terms might be too broad or too narrow
 • Include some synonymous keywords
 • Use truncation to include word variations
 • Consider British/American English spellings
 • Consider various locations of a hyphen.
 • Spell out abbreviations and include abbreviations
3. Did you choose appropriate databases?
 •  Check a list of databases by subject ( Subject guides)

Furthermore, it is important to have the search strategies to be peer-reviewed: PRESS Peer Review of Electronic Search Strategies: 2016 Guideline Statement

Yes. Before starting your systematic review, we always advise you to take a online, self-enrol course LibSmart II

Please click this link to see a list of the training: Available systematic review training

Yes, before you run the searches for your systematic review, it is really critical that you do some groundwork. The key part of this groundwork is the development of a protocol, which is like a detailed project plan, that is agreed upon by everyone involved in the systematic review project.

 •  PRISMA-P
 •  Centre for Reviews and Dissemination. (2009). Systematic Reviews: CRD's guidance for undertaking systematic reviews in health care, University of York.
 •  Littell JH, Corcoran J, Pillai V. Formulating a Topic and Developing a Protocol. In: Systematic Reviews and Meta-Analysis

At least 3 reviewers should screen, assess quality, and collect data independently to maintain transparency and reduce potential bias. The third researcher will resolve any conflicts between the two reviewers.

There are many quality assessment tools provided by several research groups. In general, you can choose a tool depending on the study type. It is up to you to choose a tool – there is no gold standard for this

Once you have finished developing the searches for each resource, run the saved searches and export the results to a place you control. This provides a clean cut-off date for your data/results which are the ones you will use for the rest of the process.

Reference managers

 •  EndNote EndNote is a literature manager software supported by the University of Edinburgh. EndNote is available as desktop run software and as a web application
 •  Choosing a reference manager
 •  The library referencing guide

How to export your search results?

 • Use bulk export where possible as it saves time and reduces error.
 • Select export options that give DOIs and other identifiers which can speed up finding full text.
 • Look for the option to export results as a RIS file as these are accepted by all reference manager tools and also Covidence.

Please see the detail. Managing search results

Covidence is a web-based platform designed to support de-duplication and easier collaboration with co-reviewers for independent screening (and more). Covidence is evolving, therefore they provide training for their updated version too.

 •  Covidence
 •  Registration Registration with your UoE email address.
 •  Covidence training Various training supported by Covidence
 •  How to use covidence Getting started with Covidence
 •  Managing Search Results

For the screening process, all the results need to be in the same place for the screening stage. Before screening though, duplicate records/results should be removed. You can remove the duplicates using the following managers:

 •  Polygot
 •  Covidence
 •  Endnote