3ie London Evidence Week 2016

Policy deliberation and voter persuasion: estimating intrinsic causal effects of town hall meetings

Prof Leonard Wantchekon: Founder, Institute for Empirical Research in Political Economy, Benin; and Professor of Politics and affiliate, Economics Department at Princeton University

This year’s 3ie London Evidence Week began with the Howard White Lecture 2016 by Leonard Wantchekon who spoke on his work around extending policy evaluation into political economy. Wantchekon drew attention to the lack of rigorous analysis of institutions involved in the political process of policymaking and made a strong case for more impact evaluations in the political sphere. He elaborated on his research on the impacts of town hall meetings on the electoral process in Benin and in The Philippines. Wantchekon described the voter engagement interventions, where people deliberated over platforms proposed by candidates in party-endorsed town hall meetings. These types of meetings proved to have an impact on voter turnout and vote share. Wantchekon concluded his talk with a few critical observations: deliberative campaigns can limit the electoral appeal of clientelism, such campaign strategies seemed to find favour with voters during polls, and it is better to choose interventions that are a part of local traditions to change the behaviour patterns of politicians rather than adopt trends and practices adopted from other countries or regions.

Watch the third Howard White Lecture by professor Leonard Wantchekon

Some reflections on impact evaluation, as practised in epidemiology and economics 

In this talk, James Hargreaves, director of the Centre for Evaluation at LSHTM provided some observations from recent work on the ways in which his own discipline, epidemiology, and the discipline of economics differently conceptualised, designed and interpreted impact evaluation studies, particularly ones using randomisation. The talk drew on several examples, including a recent cross-disciplinary replication study of deworming interventions, which appeared to highlight some areas of difference. He also spoke about areas of convergence between the disciplines in recent years and the potential for advances in the future.

This event was not recorded. The presentation and a report on the talk will soon be uploaded.

Meeting local and global development goals: how rigorous evidence can help

This one-day conference looked at the role for high-quality evidence in policymaking and programming, discussed the evidence synthesised from a major new 3ie systematic review on education effectiveness, took a critical look at current and emerging ethical issues in impact evaluation and presented findings from recent studies, reviews and a gap map. 

Keynote address, Dr Alison Evans, chief commissioner, Independent Commission for Aid Impact

The UK’s Independent Commission for Aid Impact’s chief commissioner Alison Evans kickstarted the one-day LEW conference with a thought-provoking speech on making evidence and scrutiny matter in constantly changing development contexts. She stressed the importance of scrutiny of aid to ensure evidence informed judgements that can improve outcomes for everyone. She spoke about the key challenge to scrutiny rising from the need to move from focusing on evidence to evidence-informed judgement. 

Using rigorous evidence, the role for impact evaluations and systematic reviews

The first panel discussion, chaired by Penny Hawkins, head of evaluation at DFID, focused on the role and use of evidence from SRs for policymaking. The observation by Paul Garner (Liverpool School of Tropical Medicine; Cochrane Infectious Diseases Group) that evidence needs to be separated from decisions, kept transparent and used to engage with policymakers at various levels regularly and not just as a fait accompli proved to be another popular point for discussion. DFID’s Matt Harvey questioned the usefulness of systematic reviews as evidence for policymaking since they were time-consuming and costly, and did not help policymakers in operational decision-making.  Other panellists, including David Gough (EPPI-Centre, UCL) and Phil Davies (former 3ie SRO head) spoke about the relevance of SRs as great tools for mid- to long-term strategic decision-making.

Education effectiveness and the SDGs: what we can learn from a new 3ie systematic review

This session started with 3ie’s evaluation specialist, Birte Snilstveit presenting the findings from 3ie’s new systematic review on education effectiveness. The other panellists, Chris Berry (DFID) and Elizabeth King (3ie Board of Commissioners) lauded the exhaustive review that looked at over 216 impact evaluations in 52 low- and middle-income countries. Beth King raised a critical question about how to make a systematic review of over 900 pages useful for a policymaker, for e.g. a minister of education. She stressed on the importance of ensuring the review presented policy-relevant recommendations in a manner that is understandable and useful to policymakers. The panellists also spoke about tailoring educational materials to local contexts and the relevance of considering contextual factors when making policy recommendations.

Ethics in impact evaluation

Ethics in impact evaluation proved to be another popular topic among stakeholders considering the near-full conference room and rich audience inputs. 3ie’s deputy director for Policy, Advocacy and Communication, Beryl Leach moderated the main presentations and questions in the panel of Penny Hawkins (DFID), Chris Barnett (Centre for Development Impact) and Heather Lanthorn (senior manager, IDinsight). Penny Hawkins shed light on DFID’s recent exercise to review and update their 2011 ethics, principles and guidance in evaluation and research, outlining where they have already taken steps to improve processes and accountability, especially during implementation. Heather Lanthorn presented the perspective of ethics through the lens of an individual evaluator for a private firm, with many examples of the conundrums. She also recommended more transparency around data collection methods and around the conduct of studies. Panellists also discussed the need to recognise that institutional review boards developed for medical research are often not well-suited or capable of reviewing social science research or evaluations.

Innovation in evidence production and synthesis

This final session was a platform for 3ie to present a range of their completed and ongoing work. Jo Puri and Jen Stevenson presented on a new land use change and forestry evidence gap map. Thomas de Hoop (American Institutes for Research) and Martina Vojtkova (NatCen) presented the findings from the 3ie systematic review on whether self-help groups empower women. There were also presentations on 3ie-funded studies, including one on the long-term impact of conditional cash transfers in Malawi and an upcoming systematic review on youth employment.