3ie Delhi Evidence Week 2018

3ie also organised a two-day conference, where more than 200 people representing donors, implementing partners, grantees and evaluation and development experts attended eight panel discussions. Our Executive director, Emmanuel (Manny) Jimenez started the event by recounting 3ie’s journey over the last decade. He highlighted three complex challenges that faced the evidence community: filling knowledge gaps, effectively using evidence and ensuring that policymakers used high-quality evidence in decision-making.

Varun Gauri spoke on the importance of using behavioural science to inform development programmes and policies. Recognising that people make decisions based on systematic biases and the socio-economic environment, among others, he emphasised the need to understand the context in which development programmes and policies operate. In addition to understanding biases that beneficiaries may have, he explained how behavioural science could help understand biases among policymakers, improve hiring processes and encourage knowledge sharing.

Panellists in this session shared various behavioural science approaches adopted by their organisations, for instance the World Bank’s CRI²SP framework (Communication, resources, incentives and information, social and psychological factors) to inform project design. They also discussed considering mindsets as outcome measures, moving beyond methods that had no effects, and focus on pilot tests or simulated decision making to understand how people think. One panelist explained how she identified barriers and selected appropriate behavior change techniques to address each one in her 3ie-funded study on promoting toilet use in Odisha.

3ie staff led this panel, sharing how EGMs have and can be used by decision makers. One of the panelists, Ann Flanagan, World Bank IEG, explained how they used the mapping methodology to understand support services in the health sector, how EGMs could be used to explain what intervention and outcome areas could be amenable to scale-up and where potential programmatic overlaps may lie. In a discussion on 3ie’s upcoming EGM on NRLM, Yamini Atmavilas (Gates Foundation), talked about how they were using the EGM in real-time to inform programme implementation on the ground. She commended EGMs for being a strong platform where evidence is accessible and easy to understand.

Panelists in this session discussed how technology is used in conducting evaluations, specifically: satellite data for policy evaluations; electronic tablets an application to collect for immunisation data; and, machine learning algorithms to assess crop-losses for insurance. Panelists agreed that there were challenges to using technology, such as improved technical knowledge within research teams, reduced face-to-face interactions with beneficiaries, natural aversions to technology-use, ethical considerations of data collection and the cost implications of introducing technologies in high volumes.

Representatives from four L&MIC country governments in this session shared ideas on strengthening linkages and partnerships among researchers, implementers and governments to facilitate evidence-based decision-making. Carlos Abad Santos (Philippines National Economic and Development Authority) shared learning from the 3ie-supproted Philippines evidence programme, and emphasised the need to strengthen the culture of evidence-informed decision-making.  Damit Amany (West African Development Bank, said there was a need to build government staff capacity and necessary systems to better understand and use evidence. Gonzalo Hernández Licona (CONEVAL) stressed the importance of building relationships with decision makers, and getting buy-in from programme staff for evaluations. Sunita Krishnan (Gates Foundation) highlighted push and pull factors influencing research uptake: generating relevant evidence through engagement, curating and packaging evidence in accessible ways, the need for researchers to identify the types of evidence that can be most influential in decision-making and the importance for researchers to understand uptake.

Radhika Menon (3ie), co-author of 3ie’s evidence synthesis report on community-driven development programmes presented the main findings of the study, while Vinay Kumar Vutukuru (World Bank) shared his experiences with working on CDD programmes in India. Manny Jimenez (3ie) moderated the discussion. The 3ie study found that CDD programmes had no effect on social cohesion, but did have a positive impact on public infrastructure development. Other topics discussed in this session included the factors affecting the sustainability of CDD programmes, and the need to undertake qualitative studies for a better understanding of the effects of these programmes.

Panelists, including 3ie grantees from various evidence programmes, talked about the challenges and lessons learned from formative research and evaluations. All three panelists agreed that the formative phase was crucial to testing the right design. It also allowed time for understanding the context, building relationships with key stakeholders, understanding the evidence base and addressing equity issues in their interventions.

Panelists discussed findings from systematic reviews on women’s empowerment outcomes. Panelists agreed that theories of change needed to look at barriers and facilitators of adoption, along with programme design to encourage gender and equity-sensitive evaluations that are able to identify gendered social norms and address issues of inequality. Panellists identified two main challenges in evaluating outcomes for women: (1) the incomparability of outcome measures across similar interventions; and (2) the need for better empowerment indicators so that interventions can be designed to address gendered and inequitable community norms that affect social outcomes.

Timothy Lubanga (Ugandan Office of the Prime Minister) highlighted three key things researchers should do to ensure the production of more evidence-informed policies: develop research agendas informed by policy processes, translate findings into sector-specific formats, and invest more time into capturing the effects of how evidence recommendations inform decision making. Ronald Abraham (IDInsight) added to the lively discussion by highlighting that often the most impactful policy decisions (such as the shocking and instant demonetization in India) are not evidence informed. He attributed it to researchers not having the necessary skills to have influence in the political sphere and policymakers simply not caring about evidence. Shagun Sabarwal (J-PAL, South Asia) agreed that more policy advocacy institutions help bridge the gap between evidence and its use in high-level decision-making.

In her closing remarks, Ruth Levine (Hewlett Foundation, 3ie board chair) called on the evidence community to anchor their work in the moral quest for truth, justice, the pursuit of an inclusive future and a dedication to contributing towards human progress. She reminded the participants that truth is fundamental not only on the ground in field research, but across all development research work and distributive justice is at the heart of the development agenda.