How to do a good systematic review of effects in international development: a tool kit

We provide a ‘how to’ guide to undertake systematic reviews of effects in international development, by which we mean, synthesis of literature relating to the effectiveness of particular development interventions. Our remit includes determining the review's questions and scope, literature search, critical appraisal, methods of synthesis including meta-analysis, and assessing the extent to which generalisable conclusions can be drawn using a theory-based approach. Our work draws on the experiences of the International Initiative for Impact Evaluation's (3ie's) systematic reviews programme.

• Publication year

  2012

• Venue

  Journal of Development Effectiveness

• Publication date

  2012-09-01

• Fields of study

  Economics, Political Science

• Identifiers
  DOI 10.1080/19439342.2012.711765
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Farmer Field Schools for Improving Farming Practices and Farmer Outcomes: A Systematic Review

  2014cited by this paper
• The effects of microcredit on women's control over household spending in developing countries : a systematic review and meta-analysis

  2014cited by this paper
• The Impact of Export Processing Zones on Employment , Wages and Labour Conditions in Developing Countries

  2013cited by this paper
• A Comparison of Model-Based and Design-Based Impact Evaluations of Interventions in Developing Countries

  2013cited by this paper
• Systematic Reviews: Questions, Methods and Usage

  2013cited by this paper
• Impact of Private Sector Participation on access and quality in provision of electricity, telecom and water services in developing countries: A systematic review

  2013cited by this paper
• Interventions for promoting reintegration and reducing harmful behaviour and lifestyles in street-connected children and young people.

  2013influential reference
• The Preferred Reporting Items for Systematic Reviews and Meta-analyses

  2013cited by this paper
• Interventions for Promoting Reintegration and Reducing Harmful Behaviour and Lifestyles in Street‐connected Children and Young People: A Systematic Review

  2013cited by this paper
• Finding relevant studies

  2012cited by this paper
• Which providers can bridge the health literacy gap in lifestyle risk factor modification education: a systematic review and narrative synthesis

  2012cited by this paper
• Community-based intervention packages for reducing maternal morbidity and mortality and improving neonatal outcomes 1

  2012cited by this paper
• Addressing attribution of cause and effect in small n impact evaluations: towards an integrated framework

  2012cited by this paper
• What are the economic impacts of conditional cash transfer programmes? A systematic review of the evidence.

  2012cited by this paper
• Effectiveness of agricultural interventions that aim to improve nutritional status of children: systematic review

  2012cited by this paper
• Systematic reviews: from ‘bare bones’ reviews to policy relevance

  2012cited by this paper
• The benefits and challenges of using systematic reviews in international development research

  2012cited by this paper
• Farmer field schools for improving farming practices and farmer outcomes in low- and middle-income countries: a systematic review

  2012cited by this paper
• Assessing ‘what works’ in international development: meta-analysis for sophisticated dummies

  2012cited by this paper
• Purity or pragmatism? Reflecting on the use of systematic review methodology in development

  2012cited by this paper
• The impact of daycare programmes on child health, nutrition and development in developing countries: a systematic review

  2012cited by this paper
• Narrative approaches to systematic review and synthesis of evidence for international development policy and practice

  2012cited by this paper
• How effective are cash transfer programmes at improving nutritional status? A rapid evidence assessment of programmes' effects on anthropometric outcomes.

  2012cited by this paper
• Using logic models to capture complexity in systematic reviews

  2011cited by this paper
• What is the evidence of the impact of tariff reductions on employment and fiscal revenue in developing countries

  2011cited by this paper
• Experimental and Quasi-Experimental Designs

  2011influential reference
• The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials

  2011cited by this paper
• Social franchising evaluations: a scoping review.

  2011influential reference
• Wages and informality in developing countries

  2011influential reference
• The Cochrane Collaboration's tool for assessing risk of bias in randomised trials

  2011influential reference
• Interventions to reduce the prevalence of female genital mutilation/cutting in African countries

  2011influential reference
• Interpretation of random effects meta-analyses

  2011cited by this paper
• Using Natural Experiments to Evaluate Population Health Interventions: Guidance for Producers and Users of Evidence

  2011cited by this paper
• Assessing the Impact of Planned Social Change*

  2010cited by this paper
• Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.

  2010cited by this paper
• What is the impact of microfinance on poor people?: a systematic review of evidence from sub-Saharan Africa

  2010cited by this paper
• Behavior Change Interventions to Prevent HIV Infection among Women Living in Low and Middle Income Countries: A Systematic Review

  2010cited by this paper
• Community-based intervention packages for reducing maternal and neonatal morbidity and mortality and improving neonatal outcomes.

  2010cited by this paper
• SUPPORT Tools for evidence-informed health Policymaking (STP) 10:Taking equity into consideration when assessing the findings of a systematic review

  2010cited by this paper
• Conditional cash transfers and health: unpacking the causal chain

  2010cited by this paper
• Systematic reviews: CRD's guidance for undertaking reviews in health care

  2010cited by this paper
• Impact Evaluation in Practice: Second Edition

  2010cited by this paper
• Cochrane Handbook for Systematic Reviews of Interventions

  2010cited by this paper
• Checklist For Reviewing a Randomized Controlled Trial of a Social Program or Project, To Assess Whether It Produced Valid Evidence

  2010cited by this paper
• Magnesium sulphate and other anticonvulsants for women with pre-eclampsia.

  2010cited by this paper
• Interventions to promote social cohesion in sub-Saharan Africa

  2010cited by this paper
• How Can We Support the Use of Systematic Reviews in Policymaking?

  2009cited by this paper
• Household water treatment in developing countries: comparing different intervention types using meta-regression.

  2009cited by this paper
• Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement

  2009cited by this paper
• The effect of social franchising on access to and quality of health services in low- and middle-income countries.

  2009cited by this paper
• Effectiveness and sustainability of water, sanitation, and hygiene interventions in combating diarrhoea

  2009cited by this paper
• Primary, Secondary, and Meta-Analysis of Research

  2008cited by this paper
• Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within‐study comparisons

  2008cited by this paper
• Asking the right questions: Scoping studies in the commissioning of research on the organisation and delivery of health services

  2008cited by this paper
• Can Nonrandomized Experiments Yield Accurate Answers? A Randomized Experiment Comparing Random and Nonrandom Assignments

  2008cited by this paper
• Statistical Methods for Meta‐Analysis

  2008cited by this paper
• USING RANDOMIZATION IN DEVELOPMENT ECONOMICS RESEARCH: A TOOLKIT

  2008cited by this paper
• Addressing the Unit of Analysis in Medical Care Studies: A Systematic Review

  2008cited by this paper
• A Summative Evaluation of RCT Methodology: & An Alternative Approach to Causal Research

  2008cited by this paper
• A systematic and transparent approach for assessing the methodological quality of intervention effectiveness research: the Study Design and Implementation Assessment Device (Study DIAD).

  2008cited by this paper
• Realist review to understand the efficacy of school feeding programmes

  2007cited by this paper
• Preschool feeding programmes for improving the health of disadvantaged infants and young children

  2007cited by this paper
• School feeding for improving the physical and psychosocial health of disadvantaged students

  2006cited by this paper
• Applicability and transferability of interventions in evidence-based public health.

  2006cited by this paper
• Clinical Epidemiology: How to Do Clinical Practice Research

  2006cited by this paper
• Pathfinding in the Research Forest: The Pearl Harvesting Method for Effective Information Retrieval

  2006cited by this paper
• Systematic Reviews in the Social Sciences: A Practical Guide

  2005influential reference
• Evaluating Anti-Poverty Programs

  2005influential reference
• If evidence-informed policy works in practice, does it matter if it doesn't work in theory?

  2005cited by this paper
• School meals, educational achievement, and school competition : evidence from a randomized evaluation

  2005influential reference
• Introduction to Systematic Reviews

  2004cited by this paper
• A review of the effectiveness of agriculture interventions in improving nutrition outcomes

  2004cited by this paper
• The meta-analysis of partial effect sizes.

  2004cited by this paper
• Systematic reviews of the effectiveness of quality improvement strategies and programmes

  2003cited by this paper
• How meta-analysis increases statistical power.

  2003cited by this paper
• Magnesium sulphate and other anticonvulsants for women with pre-eclampsia.

  2003cited by this paper
• Evaluating non-randomised intervention studies.

  2003cited by this paper
• Experimental and Quasi-Experimental Designs for Generalized Causal Inference

  2001influential reference
• The Campbell Collaboration

  2001cited by this paper
• Practical Meta-Analysis

  2000cited by this paper
• Why we need a broad perspective on meta-analysis

  2000cited by this paper
• A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis

  2000influential reference
• Characterizing Selection Bias Using Experimental Data

  1998cited by this paper
• Searching bibliographic databases effectively.

  1998cited by this paper
• How the Read a Paper: The Basics of Evidence Based Medicine

  1998cited by this paper
• Bias in meta-analysis detected by a simple, graphical test

  1997cited by this paper
• Benefits of a School Breakfast Programme among Andean Children in Huaraz, Peru

  1996influential reference
• The Handbook of Research Synthesis

  1994influential reference
• An introduction to meta-analysis.

  1989cited by this paper
• President's Column: Acting Towards the Future

  1981cited by this paper
• Distribution Theory for Glass's Estimator of Effect size and Related Estimators

  1981cited by this paper
• Vote-counting methods in research synthesis.

  1980cited by this paper
• Estimating causal effects of treatments in randomized and nonrandomized studies.

  1974cited by this paper
• Statistical Methods for Rates and Proportions.

  1973cited by this paper
• Statistical methods for rates and proportions

  1973cited by this paper
• Accumulating Evidence: Procedures for Resolving Contradictions among Different Research Studies.

  1971cited by this paper
• For Personal Use. Only Reproduce with Permission from Elsevier Ltd. Initial Selection Criteria and Data Extraction Water, Sanitation, and Hygiene Interventions to Reduce Diarrhoea in Less Developed Countries: a Systematic Review and Meta-analysis

  year unknowncited by this paper

• What is Financial Inclusion? A Critical Review

  2026cites this paper
• The Effects of Assisted Return Programmes on Sustainable Reintegration: A Review

  2026cites this paper
• Improving product circularity through circular economy strategies for small domestic appliance design in China

  2025cites this paper
• Community-based conserved areas in advancing sustainable development and conservation in Sub-Saharan Africa

  2025cites this paper
• Co-developing practice guides: A researcher-practitioner model for evidence-based practice.

  2025cites this paper
• Finding rare and patched type population variance via systematic analysis using adaptive cluster sampling

  2025cites this paper
• Food price shocks and food supply in fragile states: a scoping review

  2025cites this paper
• Effects of COVID-19 lockdowns on unintended pregnancies among adolescent girls and young women in low- and middle-income countries: a scoping review

  2025cites this paper
• The effect of agricultural productivity on poverty: what does meta-regression analyses reveal?

  2025cites this paper
• The impact of income support interventions on children’s long-term health trajectories: a systematic review

  2025influential citation
• The role of nutrition-sensitive interventions in improving nutritional outcomes: findings from a systematic review and meta-analysis

  2025cites this paper
• The nexus of climate and conflict in the Lake Chad Region: what we know, don’t know and need to know

  2025cites this paper
• PROTOCOL: The Effects of Land Management Policies on the Environment and People in Low‐ and Middle‐Income Countries: A Systematic Review

  2025cites this paper
• Improving Energy Access, Climate and Socio‐Economic Outcomes Through Off‐Grid Electrification Technologies: A Systematic Review

  2025cites this paper
• A review of climate change, human population growth and poverty as potential drivers of human wildlife conflicts in Africa

  2025cites this paper
• Exploring the impact of gamification on students' academic performance: A comprehensive meta-analysis of studies from the year 2008 to 2023

  2024cites this paper
• The effects of agricultural output market access interventions on agricultural, socio‐economic, food security, and nutrition outcomes in low‐ and middle‐income countries: A systematic review

  2024cites this paper
• Research Translation for International Development: A Literature Review and Framework for Evidence Use and Partner Engagement

  2024cites this paper
• Does women's economic empowerment promote human development in low- and middle-income countries? A meta-analysis

  2024cites this paper
• Effects of guaranteed basic income interventions on poverty‐related outcomes in high‐income countries: A systematic review and meta‐analysis

  2024cites this paper
• Blockchain Technology Characteristics Essential for the Agri-food Sector: A Systematic Review

  2024cites this paper
• Protocol: What works to increase the use of evidence for policy decision‐making: A systematic review

  2024cites this paper
• Green coconut shell as energy biomass: a bibliometric and systematic analysis of scientific literature

  2024cites this paper
• Shifting the focus? From individual to collective‐level effects of cash transfers: A systematic review of the impacts on social capital

  2024cites this paper
• Socioeconomic impacts of land restoration in agriculture: a systematic review

  2024influential citation
• Interventions promoting resilience through climate smart agricultural practices for women farmers: A systematic review

  2024cites this paper
• Observations on Status and Trends of Agricultural Extension and Inequality in Uganda

  2024cites this paper
• Locally Led Opportunities for Water, Sanitation and Hygiene, Climate Change and Gender Equality Partnerships in the Blue Pacific

  2024cites this paper
• Social protection and formalization in low- and middle-income countries: A scoping review of the literature

  2024cites this paper
• Innovations in Clay-Based Irrigation Technologies—A Systematic Review

  2024cites this paper
• A systematic review of fish trade and its impact on economic progress: Barriers, opportunities, and outlook for developing countries

  2024cites this paper
• Evaluating InterDev: A FAIR Platform for International Development Data

  2024cites this paper
• Delivering online community service from community perspective: A critical review

  2023cites this paper
• Whose security? Whose justice? Customary authorities in security and justice interventions in the Horn of Africa

  2023cites this paper
• LA INFLUENCIA DEL GÉNERO EN EL EMPRENDIMIENTO TURÍSTICO INNOVADOR: UNA REVISIÓN SISTEMÁTICA DE LA LITERATURA

  2023influential citation
• Telehealth Use and Legal Considerations in Drug Health Services During Pandemics: Systematic Scoping Review

  2023cites this paper
• The effects of democracy and freedom interventions in democratic backsliding contexts: A rapid evidence assessment

  2023cites this paper
• Learning from the Evidence: Impact Evaluations, Ontology and Policy

  2023cites this paper
• Underrepresentation of Local Researchers in Geophysical Studies at the Bosumtwi Impact Crater: Insights from A Systematic Review

  2023cites this paper
• Agriculture-led Growth in Low- and Middle-income Countries: An Evidence Gap Map

  2023cites this paper
• A Systematic Review of Geophysical Studies at the Bosumtwi Impact Crater in Ghana

  2023cites this paper
• The evolution of systematic evidence reviews: Past and future developments and their implications for policy analysis

  2023cites this paper
• The Impact of Fisheries on the Economy: A Systematic Review on the Application of General Equilibrium and Input–Output Methods

  2023cites this paper
• Climate change adaptation based on computable general equilibrium models – a systematic review

  2023cites this paper
• Decisions, options, and actions in the face of uncertainty: a systematic bibliometric and thematic review of climate adaptation pathways

  2023cites this paper
• What Have We Learned about National Development Banks? Evidence from Brazil

  2023influential citation
• PROTOCOL: The effects of agricultural output market access interventions on agricultural, socio‐economic and food and nutrition security outcomes in low‐ and middle‐income countries: A systematic review

  2023cites this paper
• Socioeconomic differences in the impact of prices and taxes on tobacco use in low- and middle-income countries–A systematic review

  2023cites this paper
• Research trends on biochar-based smart fertilizers as an option for the sustainable agricultural land management: Bibliometric analysis and review

  2023cites this paper
• Two systematic literature reviews of scientific research on the environmental impacts of forest certifications and community forest management at a global scale

  2023cites this paper
• Impact on childhood mortality of interventions to improve drinking water, sanitation and hygiene (WASH) to households: systematic review and meta-analysis

  2023cites this paper
• Impact on childhood mortality of interventions to improve drinking water, sanitation, and hygiene (WASH) to households: Systematic review and meta-analysis

  2023cites this paper
• Addressing the systemic causes of malnutrition: The nutrition-sensitive agriculture evidence gap map

  2023cites this paper
• Educational technology for reading instruction in developing countries: A systematic literature review

  2023cites this paper
• Factors affecting consumers’ evaluation of food derived from animals fed insect meal: A systematic review

  2023cites this paper
• The impact of income-support interventions on life course risk factors and health outcomes during childhood: a systematic review in high income countries

  2022influential citation
• Literature review of international empirical research on transfer of vocational education and training

  2022cites this paper
• Use of community engagement interventions to improve child immunisation in low-income and middle-income countries: a systematic review and meta-analysis

  2022cites this paper
• Effects of decentralization on the functionality of health facility governing committees in lower and middle-income countries: a systematic literature review

  2022cites this paper
• Scaling education innovations in complex emergencies: a meta-evaluation of five process and three impact evaluations

  2022cites this paper
• PROTOCOL: Interventions promoting resilience through climate‐smart agricultural practices for women farmers: A systematic review

  2022cites this paper
• PROTOCOL: Gender transformative approaches in agriculture for women's empowerment: A systematic review

  2022cites this paper
• Quantifying the effectiveness and health co-benefits of climate change mitigation actions across sectors: a protocol for an umbrella review

  2022cites this paper
• Why is adoption of micro-irrigation slow in India? a review

  2022cites this paper
• Aid and forgetting the enemy: A systematic review of the unintended consequences of international development in fragile and conflict-affected situations.

  2022cites this paper
• Economic outcomes of rubber-based agroforestry systems: a systematic review and narrative synthesis

  2022cites this paper
• Technology Use for Teacher Professional Development in Low- and Middle-Income Countries: A Systematic Review

  2022cites this paper
• HOW THE QUALITY OF GOVERNANCE INSTITUTIONS MAY BE THE BOOSTER SHOT WE NEED TO REDUCE POVERTY AND INEQUALITY

  2022cites this paper
• Related Work and Our Approach

  2022cites this paper
• Does citizen engagement improve development outcomes? A realist-informed systematic review of participation and accountability mechanisms

  2022cites this paper
• Evidence for the impacts of agroforestry on ecosystem services and human well-being in high-income countries: a systematic map

  2022cites this paper
• PROTOCOL: Water, sanitation and hygiene for reducing childhood mortality in low‐ and middle‐income countries

  2021cites this paper
• The Amazon Forest Preservation by Clarifying Property Rights and Potential Conflicts: How Experiments Using Fit-for-Purpose Can Help

  2021cites this paper
• Mapping energy efficiency interventions

  2021cites this paper
• Technology, Teacher Professional Development and Low- and Middle-Income Countries: Technical report on systematic mapping review

  2021cites this paper
• Soil health and gender: why and how to identify the linkages

  2021influential citation
• Do anti-poverty policies sway voters? Evidence from a meta-analysis of Conditional Cash Transfers

  2021cites this paper
• Aquaculture for improving productivity, income, nutrition and women's empowerment in low‐ and middle‐income countries: A systematic review and meta‐analysis

  2021cites this paper
• Impact pathways of photovoltaic poverty alleviation in China: Evidence from a systematic review

  2021cites this paper
• Strengthening women's empowerment and gender equality in fragile contexts towards peaceful and inclusive societies: A systematic review and meta‐analysis

  2021cites this paper
• RIGOROUS IMPACT EVALUATION: EVIDENCE GENERATION AND TAKE-UP IN GERMAN DEVELOPMENT COOPERATION

  2021cites this paper
• O REÚSO DE ÁGUA E SEUS RESULTADOS RELACIONADOS AO TRIPLE BOTTON LINE

  2021cites this paper
• Community Development Financial Institution (CDFI) program evaluation: a luxury but not a necessity?

  2021cites this paper
• A Human Machine Hybrid Approach for Systematic Reviews and Maps in International Development and Social Impact Sectors

  2021cites this paper
• PROTOCOL: Residential energy efficiency interventions: An effectiveness systematic review

  2021cites this paper
• ERCT: An Ontology for Describing Randomised Controlled Trials in the Social Sciences

  2021influential citation
• Research in International Transfer of Vocational Education and Training – A Systematic Literature Review

  2021cites this paper
• PROTOCOL: Aquaculture for improving productivity, income, nutrition and women's empowerment in low‐ and middle‐income countries: A systematic review and meta‐analysis

  2021cites this paper
• Residential energy efficiency interventions: A meta‐analysis of effectiveness studies

  2021cites this paper
• Key factors for operational performance in manufacturing systems: Conceptual model, systematic literature review and implications

  2021cites this paper
• Review of factors affecting consumer acceptance of cultured meat.

  2021cites this paper
• Interventions promoting uptake of water, sanitation and hygiene (WASH) technologies in low‐ and middle‐income countries: An evidence and gap map of effectiveness studies

  2021cites this paper
• What stimulates the demand for grid-based electrification in low-and middle-income countries?

  2021cites this paper
• The Role of Stomach Infrastructures on Children’s Work and Child Labour in Africa: Systematic Review

  2021cites this paper
• PROTOCOL: Strengthening women's empowerment and gender equality in fragile contexts towards peaceful and inclusive societies: A systematic review and meta‐analysis

  2021cites this paper
• Making qualitative data more visible in policy: a critical appraisal of meta-synthesis

  2020cites this paper
• Quantitative Evidence Synthesis Methods for the Assessment of the Effectiveness of Treatment Sequences for Clinical and Economic Decision Making: A Review and Taxonomy of Simplifying Assumptions

  2020cites this paper
• A systematic review of EPDS cultural suitability with Indigenous mothers: a global perspective

  2020cites this paper
• Influence of land tenure interventions on human well-being and environmental outcomes

  2020cites this paper
• Environmental regulation, governance, and policy instruments, 20 years after the stick, carrot, and sermon typology

  2020cites this paper