Collaborations key to improved impact evaluation designs

Do funding agencies distort impact evaluations? A session organised by BetterEvaluation on choosing and using evaluation methods, at the recent South Asian Conclave of Evaluators in Kathmandu, focused on this issue. Participants were quite candid about funding agencies dictating terms to researchers. “The terms of reference often define the log frame of evaluation (i.e. the approach to designing, executing and assessing projects) and grants are awarded on the basis of budgets that applicants submit.

Using the causal chain to make sense of the numbers

At 3ie, we stress the need for a good theory of change to underpin evaluation designs. Many 3ie-supported study teams illustrate the theory of change through some sort of flow chart linking inputs to outcomes. They lay out the assumptions behind their little arrows to a varying extent. But what they almost invariably fail to do is to collect data along the causal chain. Or, in the rare cases where they do have indicators across the causal chain, they don’t present them as such.

Of rumours related to blood, poison and researchers

The attempt to collect blood samples of children for a malaria treatment intervention in Kenya met with stiff opposition from the study community. There were rumours of blood stealing, covert HIV testing and suspicion about the safety of the study drugs. It may be quite easy to attribute this rumour to ignorance and superstition. But these rumours do not come out of the blue. Historical, anthropological and sociological accounts can trace the roots of such distrust and suspicion.

Delivering Global Public Goods

3ie not only funds studies, it also sets international standards for impact evaluation. For the studies we fund, we do this through our review process. For others, we offer quality assurance services and issue conceptual papers and guidelines. We have also launched a replication programme to test the robustness of study findings, and are preparing a registry of planned impact evaluations in low and middle income countries.

Matching policymakers and researchers

The chasm between policymakers and researchers is frequently observed but seldom addressed. A little over two weeks ago, 3ie organized a matchmaking market place to bridge this gap between research and policy. The setting for this was the Dhaka Colloquium for Systematic Reviews in International Development. The people who came together for this innovative matchmaking exercise were both ‘users’ and ‘doers’ of systematic reviews.

Using impact evaluation to improve policies and programmes

Community-level water supply does not have health benefits. There is emerging evidence that community-driven development programmes do not increase social cohesion. These statements can be made with confidence based on the considerable body of evidence from impact evaluations undertaken to answer the question of what works in development. 3ie is now adding this body of evidence as more completed studies are becoming available.

How useful are systematic reviews in international development?

Systematic reviews summarise all the evidence on a particular intervention or programme and were first developed in the health sector.  The health reviews have a specific audience: doctors, nurses and health practitioners. The audience is also easily able to find the systematic reviews. But there seems to be a big difference in the accessibility of evidence between the health and development sectors.

Special feature for World AIDS Day 2012

There has been only a small decline in the prevalence of HIV in the last decade, dropping from 5.9 percent to 5 percent between 2001 and 2009 for those aged 15-49 (UNAIDS, 2010). This decrease, whilst important, does not seem impressive compared to over US$5 billion spent fighting AIDS in low and middle income countries each year (the latest available figure is US$5.1 billion in 2008).

Field notes on implementing impact evaluations

3ie is currently funding 100 impact evaluations in low and middle-income countries spread across Africa, Asia and Latin America. We are now in a unique position to learn a lot about what’s working well in designing and conducting impact evaluations and what can be done better to ensure that research produces reliable and actionable findings.

Early stimulation and micronutrients interventions

Why should we put more money into early childhood development interventions? Does this help children in secondary education? Should we invest in preschool programmes or more in home stimulation or parenting classes? What is most cost-effective?  These are key questions that policymakers are grappling with at a time when early childhood development is emerging as a priority issue for many developing countries.

Subscribe to