gLOCAL Evaluation Week 2019

Webinar: Systèmes nationaux d'évaluation et Evaluation d'impact: Où en est l'UEMOA?

Date: 6 June 2019
Time: 2.30 – 4 pm (West Africa Standard Time)
Language: French
Speakers: Deo-Gracias Houndolo, Coordonnateur Régional Programme WACIE, Intiative Internationale pour l'Evaluation d'Impact - 3ie; Nadege Djossou, Coordonnateur Pays, Twende Mbele

Deo-Gracias Houndolo (3ie) gave an hour-long presentation, ‘National Evaluation Systems and Impact Evaluation in West Africa’, together with Dr. Nadege Djossou (Twende Mbele). He presented the results of scoping study on impact evaluation in West African Economic and Monetary Union (WAEMU) countries. The results indicate that there had been, on average each year, 13 impact evaluation studies conducted in these countries between 2009 and 2018. In addition to the limited number of studies, governments contributed very little to funding them. Context analysis suggested that the lack of national evaluation systems in most of the countries is one of the key factors that could explain these findings. Nadege explained how Twende Mbele works in Benin, Uganda and South Africa to support and strengthen national evaluation systems, and how it is gradually expanding to WAEMU countries. In conclusion, while more investments are required to undertake impact evaluations in the region, it is equally important to build national evaluation systems that would increase sustainable production and uptake of evidence that informs public policies.

Webinar: Evidence and gap maps: méthodes et utilisation

Date: 6 June 2019
Time: 4 – 5pm (West Africa Standard Time)
Language: French
Speakers: Bertrand Bio Mama, Assistant de Recherche, 3ie-WACIE; Dislène Sossou, Assistant de Recherche, 3ie-WACIE

Bertrand Bio Mama and Dislene Soussou presented on evidence gap map methodology. They included a case study illustrating the importance of evidence gap maps, and their recent work in mapping impact evaluation studies conducted in WAEMU countries from 1990 to 2018. The discussions indicated that evidence gap maps are quite new to West Africa francophone countries. More should be done to promote the use of the tool to inform research priorities and policy decisions. According to an undergraduate student who was participating, ‘This is the first time I have participated in a webinar. I must say that it was beyond my expectations. Knowing that a tool like the evidence gap map can be made available to development actors for better use of evidence is reassuring.’

Seminar on evaluating group-based livelihood programmes: opportunities and challenges from the Indian experience

Date: 3 June 2019
Time: 3– 5 pm (Indian Standard Time)
Venue: Lecture Hall 1, India International Centre, New Delhi
Speakers: Bidisha Barooah, senior evaluation specialist, 3ie; Gayatri Acharya, lead rural development economist, World Bank; Leena Johri, Joint secretary, Ministry of Rural Development; and Rohan Shah, research associate, Catalyst Management Services
Chair: Emmanuel Jimenez, executive director, 3ie

As a part of gLOCAL Evaluation Week, 3ie hosted a Delhi Seminar to discuss the opportunities and challenges in evaluating group-based livelihoods programmes in India.

Approximately 65 people from New Delhi attended the seminar, including researchers, government officials, and students. The seminar started with presentations by Bidisha Barooah (3ie) and Rohan Shah (CMS) on their experiences with the evaluation of the National Rural Livelihoods Mission (NRLM), followed by a panel discussion. The discussion included Leena Johri (joint secretary, Ministry of Rural Development) and Gayatri Acharya (World Bank), who brought in perspectives of the government as well as financiers of research respectively.

Detailed highlights

Bidisha Barooah (3ie) shared lessons learnt from the implementation of a large scale impact evaluation of NRLM, under 3ie’s wider livelihoods evidence program. She stressed the importance of forming partnerships with key stakeholders for the design and implementation of the evaluation. She also discussed how formative evaluations involving intensive fieldwork and engagement with implementers at all levels could go a long way toward strengthening evaluation design. She emphasised that field verified theories of changes should be used to map outputs and outcomes of interest and highlighted of the need for using good quality data in evaluations to help ensure robust results.

Rohan Shah (CMS) discussed the sampling strategy used for the evaluation. There was a heavy reliance on the national MIS data for the evaluation. Data from states and districts was also used, and data on clusterisation had to be collected from blocks. To verify the data, there were rigorous discussions and interviews with state, district, and block level staff on programme implementation. One of his key takeaways was that investing time in building rapport with project staff at various levels laid a strong base for the evaluation. He also said it was important to improve process documentation and understand that using large-scale administrative data requires patience and verification, as well as contextualization.

Leena Johri (Indian Ministry of Rural Development) stressed the importance of the evaluation for the national level and the nine selected states. She praised the study for its rigour, saying it made it more credible. She also stated that it is important to consider the costs to the beneficiaries while conducting evaluations, especially the opportunity cost of time, such as the lengthy four-hour survey interviews
Gayatri Acharya (World Bank) brought the global partner’s perspectives, identifying the opportunity and challenges from this evaluation. She explained how the process of conducting the evaluation can be more impactful than the results and demonstrating the intent to evaluate can help create an enabling environment for planned and disciplined program implementation and future evaluations. There is a creative tension between robustness and nimbleness of an evaluation, and, according to her, we must make this line between the two clear and question this dynamic.