3ie at the What Works Global Summit 2019

14 October

Designing and using policy briefs effectively

This interactive workshop led by Beryl Leach helped participants develop a critical appreciation of the role of briefs in effectively and accessibly communicating with decision makers. Workshop participants set the tone for the discussions by sharing how they struggled to use briefs to simplify messages and communicate with diverse actors. Following a discussion about what briefs can and cannot do and how they can work better, the participants teamed up to assess a set of existing policy briefs for their effectiveness.

Strategic communication for evidence-informed decision-making

This four-hour workshop covered diverse aspects of strategic communication using an engaging format that included polling and role-playing. Radhika Menon and Beryl Leach led the discussions. Participants brought real-life examples to the discussion, sounding out how they could engage with decision makers, convince internal and external stakeholders to take monitoring and evaluation evidence into account and package and deliver bad news effectively.

CEDIL side event: frontiers in evaluating causality

CEDIL, a DFID-supported international research collaboration to identify and test innovative approaches to impact evaluation and evidence synthesis, hosted a workshop as a one-day side event before the main summit. 3ie is a CEDIL consortium member.
At the event, CEDIL and Campbell Collaboration’s Howard White provided an update on the organisation’s vision and progress to date, with an overview of upcoming work. Other speakers, including 3ie staff, outlined the key work areas supported by CEDIL, including evaluating complexity, mid-level theory in evaluation, new technologies for data collection and stakeholder engagement.

15 October

Rapid response and systematic reviews: a practical application to international development

3ie and the Africa Centre for Evidence at the University of Johannesburg co-hosted a workshop where participants worked together to conduct a rapid systematic review in response to an evidence request on intimate partner violence. They used an existing evidence gap map and focused on co-producing a rapid synthesis and response document. In the run up to the WWGS workshop, 3ie and ACE hosted three webinars to help inform the production process of the rapid systematic review. The evidence request was made by the Colombia’s health ministry and National Planning Department (Sinergia), a 3ie member.   

Promoting, monitoring and reporting on evidence uptake and use: improving your evidence impact

Beryl Leach, Radhika Menon and Kirthi Rao facilitated a well-attended workshop on the importance of stakeholder engagement and the most effective methods to monitor, measure and report evidence impact. The four-hour workshop saw lively discussion around improving engagement with stakeholders through capacity development and incentives, knowledge brokering, monitoring what matters and adapting contribution tracing for measuring impact with confidence.

16 October

What evidence exists for building peaceful and inclusive societies? An evidence gap map

Ada Sonnenfeld presented the peacebuilding evidence gap map that identifies and catalogues the existing evidence and ongoing studies on the effects of interventions to build peaceful and inclusive societies. The map conceptualises peacebuilding in a broad way but is limited to fragile states. She also highlighted the trends in the evidence base and identified key areas for future research.

Transparency and accountability in the extractives sector: a narrative synthesis of what works and what does not

Francis Rathinam highlighted the findings of our new evidence synthesis paper on transparency and accountability in the extractives sector. He also highlighted the key lessons from impact evaluations of seven initiatives under 3ie’s Transparency and Accountability Evidence Programme. He also talked about how impact evaluations could help fill critical knowledge gaps in the natural resource governance sector.

17 October

Funding research for policy impact

In this plenary session, Emmanuel Jimenez (Manny) shared 3ie’s approach to ensure that the evidence generated by grantees meets the demands of decision makers. He emphasised 3ie’s stand that if the purpose of evaluation is learning, evaluators must engage and communicate with decision makers throughout the evaluation process. He talked about all the ways in which 3ie supports research for impact: promoting active stakeholder engagement, presenting high-quality evidence through open and accessible repositories and developing capacity for evidence use among developing country institutions through 3ie’s membership model. He also talked about our newest experiment to make evidence available just in time through rapid response to queries from decision makers.

Jane Mariara from the Partnership for Economic Policy and Sheree Bennett, research and evidence advisor at the International Rescue Committee were the other panellists at this plenary session. While Jane spoke of the Partnership’s capacity development support, Sheree spoke how IRC was drawing on high quality evidence into routine internal decision-making.

Taking stock of systematic reviews in international development: a study of methodological quality, policy relevance and usefulness

Birte Snilstveit’s presentation summarised the findings of two studies designed to assess the reliability of systematic reviews in international development and the extent to which they provide policy relevant and useful findings. A systematic review of the findings and methods of systematic reviews of international development interventions found that over 70 per cent of systematic reviews suffer from major and important limitations. Stressing the need for more and better training of researchers, Birte emphasised the need to critically apply methodologies to reviews of complex interventions. She also discussed how bigger and better teams with a mix of content, statistical, information specialists and trained research assistants can enhance the systematic review production process.

The future of evidence curation and translation in international development: opportunities and challenges

This all-women panel chaired by 3ie’s Marie Gaarder brought together Harsha Dayal from South Africa’s Department for Planning, Monitoring and Evaluation, Rhona Mijumbi-Deve from the African Centre for Systematic Reviews and Knowledge Translation, Laura Daniela Uribe Gomez of Colombia’s National Planning Department (Sinergia) and 3ie’s Birte Snilstveit.

The panellists emphasised the importance of transforming evidence to knowledge for informing decisions. Panellists highlighted rapid synthesis and proactive evidence curation as opportunities for providers. The discussions emphasised the collaborative aspect of evidence curation and translation which involves conversations and engagement between evidence generators, users and knowledge intermediaries. The panellists shared processes which helped evidence curation and translation work, speaking of the necessity of being responsive, considering resources and navigating politics while building demand and ownership for curated evidence.

18 October

Integrating evidence into policymaking: the role and use of impact evaluations

This panel discussed the findings and implications of a 3ie-funded impact evaluation of the Indian government’s flagship National Rural Livelihood Mission (NRLM). 3ie’s ambitious impact evaluation covered nine states and approximately 27,000 households. Nita Kejriwal from India’s Ministry of Rural Development presented why it was essential to understand the quality of community institutions before the next phase of the programme. Although there were state-level studies, a national-level evaluation was missing. Anjini Kochar from the University of Stanford and principal investigator of the study pointed out how important it is to have an innovative impact evaluation for a large programme such as NRLM.

Efficient, timely and living evidence synthesis: opportunities in international development

Birte’s presentation outlined the opportunities for more efficient and timely review production in international development, including, but not limited to ‘living systematic reviews.’ Quite often, a systematic review production timeline does not align well with a decision maker’s needs. Systematic reviews are labour intensive and could be inefficient if there is a lack of coordination in production. Based on a combination of literature review, feasibility testing and modelling approaches, she summarised the opportunities that are currently available for systematic review teams wanting to produce studies more efficiently in six months, which is considered a sweet spot in systematic review production timeline.

Transparency and accountability in the extractives sector: an evidence gap map

Francis Rathinam presented the latest evidence gap map on transparency and accountability in the extractives sector. He highlighted some of the main gaps, for instance, how there were few studies on interventions in environment and financial audits. He also pointed out gaps in research on long-term development outcomes.    

Payment for environmental services: do we get what we pay for?

Birte summarised the results of a Campbell mixed-methods systematic review on the effects of payments for environmental services (PES) on environmental and socio-economic outcomes in L&MICs. Findings suggested that there is limited high quality evidence to assess the effects of PES programmes on deforestation and poverty. The study suggested that randomised designs are feasible and should be prioritised in future studies. Based on the study recommendations, Birte also suggested how future studies should assess both the environmental and human welfare outcomes of PES to allow an assessment of potential synergies or trade-offs between different programme objectives.

Citizen engagement in public services through participation, inclusion, transparency and accountability

Ada Sonnenfeld presented a 3ie systematic review that looked at whether citizen engagement interventions in L&MICs improve outcomes compared to standard public service delivery along the results chain from immediate outcomes (active citizenship and provider actions), intermediate outcomes (access, quality and use of public services) to final outcomes (citizen wellbeing and relations between the state and citizens). The review also analysed the mechanisms operating in citizen engagement interventions.

Immunisation: how formative evaluations can help us design better programmes

Stuti Tripathi highlighted the findings of an evidence synthesis of six formative evaluations of community-based pilot programmes delivering immunisation services in Ethiopia, Nigeria, Myanmar and Pakistan. The synthesis looked at factors affecting programme reach and take-up among targeted participants; the feasibility of delivering the intervention; and contextual barriers and facilitators that matter for programme design and implementation.

Video interviews from the event