Uganda Evaluation Week 2019: from evidence generation to utilisation
3ie staff participated in the 7th Uganda Evaluation Week conference in Kampala from 11-15 February. This event brought together policymakers, government officials, civil society members, development partners, monitoring and evaluation practitioners and researchers to share evaluation experiences. 3ie has also provided six bursaries for this event, which was organised by the Uganda Evaluation Association in partnership with Uganda’s Office of the Prime Minister.
Venue: Hotel Africana, Kampala, Uganda
Highlights from the conference
In his inaugural address, Ruhakana Rugunda, the Ugandan prime minister referred to the 3ie-funded evaluation of Uganda’s community-based public information programme, which has been immensely successful in providing feedback from citizens on the utilisation of public funds. Marie Gaarder (3ie) delivered the keynote speech on how we are entering an age of ‘evaluation for grown-ups’, and the need to be increasingly careful in our interpretation of evaluation findings.
Click on the session titles to read some of the key takeaways from 3ie’s sessions at the conference.
Panellists shared their experience in evaluating the impact of different community engagement approaches for improving governance and development programming in Uganda.
Emmanuel Manyindo (Maendeleo ya Jamii) talked about the effectiveness of multi-stakeholder forums in Western Uganda in improving development outcomes for local communities through improved transparency (knowledge on oil sector matters) and demand for accountability (civic actions). While these forums have increased respondents’ inclination to pursue information and take civic actions, they have not lead to changes in policy outcomes related to service delivery.
Fredrick Bagamba (Makerere University) shared key findings from the evaluation of the creative capacity building initiative, a hands-on training for community members to harness local creativity and indigenous knowledge, on household welfare, and behavioural and attitudinal change. He asserted that it has a great potential to reduce household poverty among beneficiaries, as labour could be invested in other income generating activities in addition to increased crop income.
Fredrick Bagamba (Makerere University) shared key findings from the evaluation of the creative capacity building initiative, a hands-on training for community members to harness local creativity and indigenous knowledge, on household welfare, and behavioural and attitudinal change. He asserted that it has a great potential to reduce household poverty among beneficiaries, as labour could be invested in other income generating activities in addition to increased crop income.
Nassul Kabunga (IFPRI) shared a few emerging lessons from an ongoing impact evaluation of Uganda’s Community Advocacy Forum (Baraza), an initiative to engage citizens in planning and monitoring of government services in their communities, on citizens’ demand for accountability, as well as service delivery in agriculture, education, water and sanitation, health, and roads.
Panellists shared evaluation findings from two studies supported through 3ie’s Uganda Evidence Programme.
Badru Bukenya (Makerere University) shared findings from the evaluation of Uganda’s youth livelihoods programme. While the evaluation did not show statistically significant employment, especially occupational changes, but showed marginal reduction in income. This reduction could be due to the project gestation period, and learning curve for the newly employed youth. He observed that the qualitative study found several implementation challenges such as inadequate training for youth and lack of adequate operational funds.
Paul Onapa (Ministry of Gender, Labour and Social Development) talked about the ministry’s involvement in the evaluation. He also explained how findings and recommendations from the evaluation (and a previous process evaluation) led to key policy actions, for example, a cabinet memo has been developed to increase operational funds, adequate training with the help of local trainers, among others.
Narathius Asingwire (Socio-Economic Data Centre) presented findings from the family planning evaluation which evaluated the impact of centres that provided youth-friendly family planning services such as product distribution, health camps, counselling, and testing and treatment of HIV. He shared that randomised encouragement design did not lead to any increase in the uptake of the programme and had no effect on key outcomes. Based on insights from the qualitative study, he suggested that this may be due to supply-side constraints. He also pointed out that following the team’s suggestion, the implementing agency has carried out a field assessment to assess staff requirements and stocks.
Abdul Muwanika (OPM) and Francis Rathinam (3ie) gave an overview of our work on capacity development. Rose Namara (Uganda Management Institute) spoke about working with the education and sports ministry, who were willing to develop capacity, but not always able to invest time in implementing their action plans. According to her, ministries needed to be held accountable for making capacity development projects part of their work plans and delivering on them. Tonnie Stieve Luyimbazi (Ministry of Gender, Labour and Social Development) shared that participation in 3ie’s capacity development project had improved appreciation of data and evidence for decision-making at different levels. He also said that the duration of these projects needed to be realistic and factor in the amount of time the change process takes.
James Wokadala (Measure Africa (U) Limited) presented the findings of the 3ie-supported impact evaluation of the Ugandan government’s School Facilities Grant programme. Carthbert Msulyalya (Ministry of Education and Sports) shared the implementing agency’s perspective of being part of an impact evaluation. Mulyalya said ongoing engagement between the research team and the ministry helped in identifying policy-relevant questions. Although randomisation was seen as an ethical and political issue in the beginning, the challenge was overcome with continuous sensitisation of local governments. He emphasised that impact evaluations were key in guiding policy direction.
Beryl Leach and Radhika Menon (3ie) conducted a workshop on promoting evidence use through stakeholder engagement and policy briefs. Participants included officials from the Office of the Prime Minister, Ministry of Finance, and staff from the African Union, GIZ, Uganda Management Institute and other academic institutions. Participants were very engaged and gave thoughtful and useful feedback as part of the group exercises.
In her keynote, 3ie’s Marie Gaarder urged evaluators to be more careful about the interpretations of the findings to avoid arriving at wrong conclusions or policy recommendations. Misdiagnoses, confounding implementation and design issues as well as imperfect understanding about the effectiveness production function are major challenges to arriving at the correct interpretation of the findings. She also gave an overview about 3ie’s holistic approach to evaluation that aspires to use the best available data to answer the questions most relevant to policymakers, the need to involve local researchers in the evaluation efforts and the importance given for extensive stakeholder engagement throughout the evaluation.