Skip to Main Content

Evidence Synthesis and Systematic Reviews

What is an evidence synthesis? How do you do a systematic review? And how can the library help?

So what's a systematic review?

In the most restrictive sense, a systematic review is a type of evidence synthesis that is particularly rigorous in the finding, selecting, evaluation, and analyzing of research.

The rigorous nature of a systematic review means that it is not always an appropriate (or possible) methodology for all evidence synthesis. The ideal systematic review includes:

  • a well-documented and reproducible search
  • limited to well-defined research methodologies
  • the selection process includes strict criteria and multiple selectors to avoid bias (ie. you can't do a systematic review with one person. A single person review using this methodology is sometimes called a "systematized"  or "structured" review.)
  • a comprehensive synthesis of the available evidence

Some of the best known systematic reviews are produced in the healthcare field in the Cochrane Library of Systematic Reviews (SCSU login for subscription access). They have an extremely rigorous process and require training, as documented in the Cochrane Handbook.

JBI is another group specializing in systematic reviews and other types of healthcare evidence synthesis. Besides documentation on methodology, they also have several video series on defining and conducting high quality evidence synthesis projects.

A newer group is the Campbell Collaboration, which produces systematic reviews (and other types of evidence syntheses) in Business & Management, Crime & Justice, Disability, Education, International Development (including Nutrition), Knowledge Translation & Implementation, and Social Welfare.

The Institute for Education Science (out of the US Dept of Education) maintains the What Works Clearinghouse, which compiles reviews into educational intervention reports and practice guides following specific protocols.

That said, not all fields differentiate between systematic reviews and other types of evidence syntheses, calling everything systematic reviews. And the definitions have changed even within the healthcare field, where evidence syntheses have been formally used the longest.

Steps in a traditional systematic review

Steps in a Traditional Systematic Review Estimated Time Investment  
1. Assemble systematic review team and select project manager Varies  
2. Identify appropriate review methodology 2 weeks  
3. Define research question 2 weeks  
4. Define inclusion/exclusion criteria 1 week  
5. Select databases 1 week  
6. Select gray literature resources 1 week  
7. Write search strategy for “master” database 1 week  
8. Write and register protocol (written compilation of previous steps) Varies  
9. Translate search strategy to syntax for all databases (including gray literature) 2 weeks  
10. Search and export results into citation management software 2 weeks  
11. De-duplicate results 2-4 weeks  
12. Title and abstract screening 2–3 months*  
13. Retrieve full-text articles 1 month*  
14. Full-text screening 2–3 months*  
15. Risk-of-bias assessment 2–3 months  
16. Data extraction 2–3 months  
17. Meta-analysis or synthesis of results 2–3 months  
18. Write the manuscript 2–3 months  

* Timeframe can vary significantly depending on number of citations identified for screening.

Creative Commons License From: Evidence Synthesis: How Librarians Can Help from Cornell University Library (CC-BY 4.0)

Should I do a systematic review?

Questions to think about:

  1. Do you have a team? Full SRs are best done with mulitple people to reduce bias. Three is the minimum, though more eases the workload.
  2. Do you have enough time? Most full SRs take a minimum of 12 months; 18-36 months is not uncommon. Adding other types of analysis, such as a meta-analysis of data, will add to the time.

If your answers are No, then you may want to do some other type of review. For semester-long projects, a Rapid review or a systematized/structured review (both follow SR techniques but with streamlined procedures.)