Replication

We set up our Replication Programme to address the need for a freely available global public good that helps improve the quality and reliability of impact evaluation evidence used for development decision-making. Replication is the most established method of research validation in science, yet it has not been fully embraced by the research community or development donors, leading to this gap.

DB singh

What is replication?

The 3ie programme highlights the benefits of replicating impact evaluations of development studies to incentivise replication of selected influential, innovative or controversial impact evaluations. Since 2012, we have funded more than 20 internal replications. This type of replication uses data from the original study and possibly existing secondary datasets from the same location to check the validity and robustness of the estimations and recommendations.

Where 3ie-funded replication studies confirm that the impact evaluation findings are valid and robust, they lend additional credibility to these findings for use in policy and programme decision-making.

For impact evaluations revealed to have invalid or non-robust findings, the replication results caution decision makers in the use of those findings. The overall benefit of 3ie’s programme, though, is to improve the incentives for all impact evaluators to conduct careful analysis leading to credible findings in the first place.

This selected bibliography attests to the fact that there is a growing number of replications in economics. For a multidisciplinary list of previous research on the practice of replication, see our replication methodologies bibliography. A comprehensive database of replication studies can also be found on the University of Göttingen's replication wiki website.

For a list of frequently asked questions about the 3ie Impact Evaluation Replication Programme, download the FAQ document.

  • Replication programme: We provide funding for researchers to conduct replication studies of impact evaluations in low- and middle-income countries. We currently have two ongoing sector-focused programmes:
  • Replication study status: We track the progress of each completed and ongoing study and those that could not be replicated.
  • In-house replication: 3ie’s specialist staff also conduct replication studies and develop guidance on replication methodology.
  • Data Preparation and Release Window: 3ie provides funding to original authors of 3ie-funded replication studies, published before September 2013, for preparing their raw datasets and accompanying codebooks for public release.
  • Replication Paper Series: We publish replication studies funded through the replication programme, we may also include qualifying external submissions to the series.
  • Push Button Replication: 3ie conducts push-button replications of all data sets emerging from 3ie-funded studies (see our research transparency page). We also completed a project to confirm the validity of published results using both the original data and the programming code from a study. Learn more about the project here

3ie’s Replication Paper Series offers a defined space for and open access to replication studies of development impact evaluations.

These studies are published regardless of whether the findings support or question the results of the original paper. By providing a reputable publication outlet, we are encouraging researchers to undertake the replication of research in general and to contribute to the public good of improved impact evaluation research practices.

The latest publications in 3ie's Replication Paper Series can be found here. To learn more about the submission process, click here.

3ie is committed to encouraging open and constructive dialogue between replication researchers and original authors. To support this goal, we have created the 3ie replication programme notification and communication policy. We would like to thank everyone who helped strengthen this policy through their comments and suggestions. It is a living document that we update as appropriate.

For more information, read Quality evidence for policymaking: I’ll believe it when I see the replication, 3ie Replication Paper 1

RP14

When to start ART? A replication study of timing of antiretroviral therapy for HIV-1-associated Tuberculosis

Replication paper 3ie 2018
In this paper, the research team conducted a replication study of Havlir and others’ 2011 study, “Timing of antiretroviral therapy for HIV-1 infection and tuberculosis.”

RP13

Stretching HIV treatment: A replication study of task shifting in South Africa

Replication paper 3ie 2017
Baojiang Chen and Morshed Alam conducted a replication study of Fairall and other’s influential paper, “Task shifting of antiretroviral treatment from doctors to primary-care nurses in South Africa (STRETCH)”.

Cash transfers and HIV/HSV-2 prevalence: a replication of a cluster randomized trial in Malawi

Cash transfers and HIV/HSV-2 prevalence: a replication of a cluster randomized trial in Malawi

Replication paper 3ie 2017

In this paper, Lynette Smith, Nick Hein and Danstan Bagenda conduct a replication study of the influential 2012 publication, 

Power to the people?: a replication study of a community-based monitoring programme in Uganda

Power to the people?: a replication study of a community-based monitoring programme in Uganda

Replication paper 3ie 2016

In this replication study, Katherine Donato and Adrian Garcia Mosqueira re-examine the results of an influential 2009 study, 

Fighting corruption does improve schooling: a replication study of a newspaper campaign in Uganda

Fighting corruption does improve schooling: a replication study of a newspaper campaign in Uganda

Replication paper 3ie 2015

Supply chain inefficiencies can hinder public service delivery. Resource capture via corruption is one such leakage.

The effects of land titling on the urban poor: a replication of property rights for the poor

The effects of land titling on the urban poor: a replication of property rights for the poor

Replication paper 3ie 2015
Drew B. Cameron, Edward M. Whitney, and Paul C. Winters replicated the results of an influential 2010 study, Property rights for the poor: Effects of land titling by Sebastian Galiani and Ernesto Schargrodsky.

23
January 2019
London

CEDIL lecture: stakeholder engagement for development impact evaluation and evidence synthesis

CEDIL 23 January 2019 London Professor Sandy Oliver (UCL Institute of Education) will deliver a lecture on engaging stakeholders in the generation and use of evidence on 23 January 2019.

Ollivier GirardCIFOR

First conference of the West Africa Capacity-building and Impact Evaluation (WACIE) Program | La première conférence du Programme de renforcement des capacités et d'évaluation de l'impact en Afrique de l'Ouest (WACIE)

Conference 28 January 2019 Benin 3ie and the Government of Benin, in collaboration with other partners, are organizing a two-day conference to present the WACIE program activities, and share findings from a scoping study on evaluation capacity and opportunities in the region.

12
December 2018
London

To boldly go where no evaluator has gone before: the CEDIL evaluation agenda

Seminar 12 December 2018 London Edoardo Masset will introduce the newly established Centre of Excellence on Development Impact and Learning (CEDIL), which was set up by DFID to develop new evaluation methods and to commission evaluation and synthesis studies in neglected areas of international development.

dialogue on transparency

A dialogue on transparency, open access and ethics in development research

Conferences 04 December 2018 New Delhi This joint-event, organized by the Sehgal Foundation and 3ie, was a platform for over 110 policymakers, including government representatives, researchers and students to come together to discuss transparency, open data, ethical values and issues related to development research in India.

Video lecture 13: Evaluating the impact of an education programme

Speaker: Radhika Menon, Senior Policy, Advocacy and Communication Officer

Many education programmes have helped in increasing the enrolment and attendance of children in schools but there is less evidence on what works in improving learning. In this video, RadhikaMenon uses an example from India to show how an impact evaluation can provide rigorous evidence on what works in improving child learning.

Video lecture 14: Impact evaluations of social protection

Speaker: Heather Lanthorn, Evaluation Specialist, 3ie 

Conditional cash transfers are one of the most studied programmes in development economics but there is still a lot we need to learn about whether and how they work. In this video, Heather Lanthorn illustrates how an evaluation using regression discontinuity design assesses the impact of a conditional cash transfer programme on educational outcomes in Cambodia.

Video lecture 15: Impact evaluations in microfinance

Speaker: Stuti Tripathi, Senior Policy Officer, 3ie 

Impact evaluations can provide practitioners with useful guidance on what an intervention can realistically accomplish. Stuti Tripathishows how impact evaluations have helped the microfinance sector introspect on where microcredit was falling short in fulfilling its mandate and contributed to innovation in programme design.