African Evaluation Association conference

3ie had a strong and effective presence throughout the event.  Our specialists organised five methods skills-building workshops, five panel discussions and presented three posters covering our impact evaluation, systematic reviews and mapping work in health, education, humanitarian assistance, and transparency and accountability. We awarded ten bursaries to African evaluators.  Through our regular grant programmes, we sponsored 30 panellists, primarily from developing countries.

3ie’s co-sponsorship and active participation at the AfrEA conference were well mentioned in the AfrEA newsletter post the event. Dedicated pages in the newsletter acknowledge 3ie’s contribution and an excerpt from the keynote address by 3ie’s executive director.Read more

Resources from the conference

3ie-sponsored workshops

3ie organised three skill-building workshops and two peer-learning workshops that offered a great platform for sharing expertise and experiences in various sectors. Researchers and implementers from Humanitarian Assistance Thematic Window and the Transparency and Accountability Thematic Window talked about the progress made on impact evaluations and the challenges they have encountered. They also shared preliminary results of their studies apart from an overview of the interventions, the relevance of particular research, evaluation design and the challenges they have been encountering. External experts and other research team members provided feedback on how teams could improve their analysis or explore ways to understand the value for money and effectiveness of these interventions in real world settings. 3ie representatives presented on key lessons around stakeholder engagement and the use of evidence. Participants discussed ways to address challenges and the role that researchers play in translating and communicating their findings for different decision makers

Using theory of change and mixed methods in impact evaluation
3ie requires its grantees to use an intervention’s theory of change and mixed methods in order to measure along a causal chain and answer more than whether or not programme is working or not. 3ie also wants evaluations to answer for whom, why and how. Discussions underscored the importance of evaluators understanding the difference between theory of change and logical frameworks. Participants also thought that evaluators need to have experience in programmes to design theory-based evaluations well. 

Designing and implementing high-quality, policy-relevant impact evaluations in humanitarian contexts: learning from 3ie-supported impact evaluations
This day-long event included presentations by 3ie’s grantees under its humanitarian assistance window. They discussed progress made on their studies, including preliminary and final results. They also spoke about the policy relevance of the study, evaluation design, methodologies, capacity-building efforts, stakeholder engagement, uptake of early evidence, challenges and lessons. Researchers discussed communicating timely and relevant evidence for humanitarian programming and how decision makers, including policymakers and donors, use the evidence to inform their work. Discussions also focused on the role of civil society organisations, UN networks and movements such as Scaling Up Nutrition movement to drive the evidence-informed agenda.

Designing and implementing high-quality, policy-relevant impact evaluations in transparency and accountability in extractives sector: learning from 3ie-supported impact evaluations
At this workshop, there were rich discussions on the relevance of producing more evidence on transparency and accountability in the extractive resources sector, especially against the local backdrop of Uganda’s burgeoning oil sector. Participants emphasised the need to figure out how to allocate revenue from extractives to different sectors, such as health and education. Grantees presented early baseline results and an overview of the interventions, the relevance of their research, evaluation design and the challenges they have been encountering. External experts and other research team members provided feedback on how teams could improve their analysis or explore ways to understand the value for money and effectiveness of these interventions in real world settings.  

Briefs are important, they just cannot do everything
3ie is a leader in understanding how to promote evidence-informed decision making.  Briefs are usually short (two to four-pages) summaries of research or evaluation designed to be relevant for decision makers and written in an accessible style.  Participants learned about the importance of identifying the main audience and addressing their information needs. Briefs complement stakeholder engagement and other forms of communication. Everyone learned about the main elements of an effective brief.  They applied their new understanding about effective briefs by appraising a selection of briefs and discussing what worked well and did not.

Capacity building to produce impact evaluations and evidence synthesis in Africa
3ie wants to know directly from producers what they think works and does not in terms of evaluation or synthesis training in the region. Ten participants from four African countries discussed the value of training that is wholly or partially university-based evaluation coursework. Project-based training is prevalent in the region, resulting in a very narrow, donor-focused skill base geared towards producing M&E for a given project. One of the key takeaways from this session, was that short courses work best if the participant already has a broad enough skills base and knowledge of evaluation. Evaluation training works best when offered with programme design, implementation and management skills building.

3ie sponsored panel discussions

3ie was one of the major contributors to the impact evaluation strand at AfrEA 2017. 3ie organised panel discussions to facilitate conversation on the generation and use of impact evaluations, systematic reviews and evidence gap maps in sectors, such as education, environment, agriculture, financial inclusion and humanitarian assistance. These are the highlights from the panel sessions.

How can evidence help us achieve the sustainable development goal on education?
Panellists explored the different types of evidence that can be used for supporting decision-making and programme implementation. 3ie shared the findings of its systematic review on education effectiveness, which examines the impact of 216 programmes in 52 low- and middle-income countries. Researchers from South Africa’s Department of Basic Education presented the preliminary findings of a 3ie-funded early grade reading impact evaluation which shows that structured pedagogy has positive effects on learning outcomes.

Evaluating agricultural innovation
3ie is currently supporting 14 high-quality policy-relevant impact evaluations aimed at improving farmer well-being through promotion of innovative technologies, creation of markets, strengthening of value chains and use of effective communication. 3ie grantees working on these evaluation and agriculture experts discussed evaluation challenges and lessons learned. Panellists agreed that agricultural interventions are complex and challenging because research has to be coordinated with the agricultural season. They also emphasised the need to think innovatively about research methodologies that factor in the realities of agricultural programme implementation and focus on outcomes that are of interest to stakeholders.

Using evidence to improve humanitarian action
The discussions in this panel focused on the value of evidence from different types of evaluations and contexts, including real-time evaluations and impact evaluations. Panellists focused on the kinds of questions these evaluations can or cannot answer, including those on efficiency and effectiveness of humanitarian action. The panellists acknowledged the ongoing tension between the need for rapid responses, collecting data and having it be available at the right time, in the right way for the right people. Participants also flagged the importance of internalising and operationalising lessons from these evaluations, and building the evidence base to synthesise findings and lessons from these evaluations. The important step is to learn the lessons and translate the evidence into useful forms to be ready for the next crisis. Many organisations also stressed the need for translating that evidence for different users, particularly frontline aid workers, who need evidence-informed guidelines and manuals instead of long evaluation reports.

Challenges and opportunities for evidence mapping - lessons from an international context
This session started with presentations on how 3ie conceives of and defines evidence gap maps (EGMs) and the ways in which they can be used. 3ie staff also presented  on EGMs on land-use change and forestry and adolescent sexual and reproductive health.  Both of them are examples of EGMs that have clear findings on where we need more evidence and where to direct future research. Participants discussed the advantages and limitations of restricting gap maps to impact evaluations and systematic reviews.  

Examining the impact of financial inclusion interventions
The panellists discussed various challenges, including arriving at consensus on what should constitute financial inclusion, understanding beneficiaries’ perspective of how poverty alleviation can be defined as and also having indicators that are context specific. Economic indicators alone may give a lopsided picture, and one needs to look at overall well-being of the beneficiaries. The panellists also highlighted innovative ways of understanding people’s economic behaviour through a lab approach that affects uptake of financial inclusion programmes.

3ie executive director’s closing keynote address

Closing plenary keynote
3ie’s executive director Emmanuel (Manny) Jimenez commended the organisers for effectively using the sustainable development goals as a reference point for the sessions, especially on hard-to-evaluate goals, including climate change, sustainable cities and peace and justice. He also praised the increased visibility of and focus on gender-responsiveness and equity and on evaluating complex interventions.