Skip to Main Content
Skip to main content
Indiana University School of Medicine

Ruth Lilly Medical Library

Systematic Reviews and other Evidence Syntheses

Extract, Appraise, and Synthesize

During these steps, data is extracted from all relevant studies.  Then, individual studies are evaluated for potential bias, based on study design and reporting. For example, design flaws that raise questions about the validity of findings or an overestimate of the intervention effect. Lastly, studies are synthesized, and conclusions are drawn.

Extract

At this stage, you will develop your evidence tables, which give detailed information for each study (see Table 1 in this article). You can create evidence tables to describe study characteristics, results, or both. 

Data extraction requires a lot of planning. We will review some of the tools you can use for data extraction, the types of information you will want to extract.

 

Data to Extract

What data you extract will depend on your research question and the needs of your review.

Helpful data for an intervention question may include:

  • Information about the article
    (author(s), year of publication, title, DOI)
  • Information about the study
    (study type, participant recruitment / selection / allocation, level of evidence, study quality)
  • Patient demographics
    (age, sex, ethnicity, diseases / conditions, other characteristics related to the intervention / outcome)
  • Intervention
    (quantity, dosage, route of administration, format, duration, time frame, setting)
  • Outcomes
    (quantitative and / or qualitative)

If you are doing a qualitative review, you may want to extract data on the theoretical framework, data collection method, and role of researcher and their potential bias.

 

Data Extraction Tools

 

How to Display Data Extraction Table

Appraise

Individual studies are evaluated for potential bias, based on study design and reporting. When conducting a systematic review, at least one critical appraisal tool should be chosen, a priori, to evaluate the studies.

Consult the following article for general information on risk of bias assessment.

Appraisal Tools: Click on the link to access the tool(s). 
Study Type JBI Critical Appraisal Tools CASP: Critical Appraisal Checklists LEGEND Evidence Evaluation Tools Risk of Bias Tools (formerly Cochrane Risk of Bias Tools) Quality assessment tool for quantitative studies (EPHPP) SIGN Checklists GRADE system

 

(Used with other tools to evaluate intervention outcomes)

 AGREE II AMSTAR 2
Randomized Controlled Trials (RCTs) X X X X X X X   X
Non-randomized Trial     X X X   X   X
Cohort Studies X X X X   X      
Case-Control Studies X X X     X      
Cross-Sectional Studies X   X            
Diagnostic Studies X X       X      
Economic Studies X X       X      
Mixed Methods     X X          
Qualitative Studies X X X X          
Systematic Reviews and Meta-Analyses X X X X   X X X  
Clinical Practice Guidelines     X         X  
Other Study Designs X   X            
More Tools Here:

The following were consulted for this table.  Please consult these resources to find: evaluations of many of these tools, appraisal tools focused on other study designs, and additional critical appraisal tools.

  • Evidence Tools -  assembled by CareSearch (which is funded by the Australian Government Department of Health and Aged Care)
  • Repository of Quality Assessment and Risk of Bias Tools OSF.xlsx (maintained by Duke University's Medical Center Library & Archives)
  • "Select a quality Assessment tool" (University of North Carolina Health Sciences Library LibGuide page, Systematic Reviews: Step 6: Assess Quality of Included Studies)
  • Study Quality Assessment Tools - "In 2013, NHLBI [National Heart, Lung, and Blood Institute] developed a set of tailored quality assessment tools to assist reviewers in focusing on concepts that are key to a study’s internal validity. While these tools have not been independently published and would not be considered standardized, they may be useful to the research community."
  • CATevaluation - Critical Appraisal Tools (CATs) (supported by the Canadian Institutes of Health Research (CIHR-IRSC), le Fonds de recherche du Québec – Santé (FRQS), and the Method Development Component of the Quebec SPOR Support Unit)

Synthesize using Standards & Guidelines

Statistical help available

While librarians do not provide assistance with statistics, the Biostatistics and Health Data Science offers assistance.