Transparency and reproducibility of impact evaluation: a turning point for the evaluation community
Speaker: Arnaud Vaganay, director of Meta-Lab, Catalyst (Fellow) of the Berkeley Initiative for Transparency in Social Sciences (BITSS)
Discussant: Kiran Bhatty, senior fellow, Centre for Policy Research, New Delhi
Chair: Diana Milena Lopez Avila, Evaluation Specialist, 3ie
Date: 28 April, 2017
Abstract
Transparency and reproducibility of impact evaluation is paramount to advancing evidence-informed policymaking in lower and middle-income countries. Recent examples demonstrate that pivotal scientific findings cannot be replicated due to poor documentation or methodological bias, sparking debate across scientific and regulatory communities. However, there is general agreement that we need improvements in communicating and documenting research and risk assessment methods.
In this 3ie Delhi seminar, Arnaud Vanagay presented his research in meta-research. He covered measuring the transparency and the scientific credibility of applied social sciences (e.g. evaluations of employment policies) and the influence of scientific norms, political institutions and financial incentives on the transparency and credibility of this research.
About the speaker
Arnaud Vanagay is the founder and director of Meta-Lab, a consultancy that develops, implements and evaluates new tools to make research and teaching more cost-effective. He is also a visiting lecturer at the London School of Economics and Political Science. He is interested in defining what makes ‘good’ research decisions. Arnaud studies the economic, political, psychological and philosophical factors driving these decisions. His interest in meta-research stems from his experience as a policy and programme evaluator. Arnaud holds a PhD in Social Research Methods from the London School of Economics and Political Science and a MSc in political science from Science Po Grenoble, France.