What is an impact evaluation?
Impact evaluations are designed to answer the question: "What was the effect of an intervention on an outcome?" Measuring such an effect is not as simple as it might initially seem. Consider a training programme for female entrepreneurs, like this one in Kenya we helped evaluate. Simply measuring participants’ business performance before and after the programme does not necessarily measure the programme’s impact, because nothing happens in a vacuum. Many factors affect business performance from month to month, from seasonal variations to technological change to global health pandemics. So how can we separate the effect of the programme from all those other factors? Impact evaluations provide a toolkit of methods to measure the effects of that programme, and that programme alone.
In the Kenya example, the research team used a randomized control trial, a method similar to the studies doctors use to test the effectiveness of new medicines. By randomly assigning some individuals to participate in a given programme while others are not, we can compare the outcomes across the two groups to see if the programme works. Because random assignment produces two groups that are similar except for the presence of the programme, we rule out other factors that might otherwise account for the differences between the two groups, leading to a causal interpretation of impact.
However, not everything can (or should) be randomized. Evaluations of other interventions like food aid to conflict-affected families in Mali or an environmental programme in Mexico rely on other tools: quasi-experimental methods. With careful research designs, advanced statistical techniques, and innovative approaches to data collection, we can identify an appropriate comparison group and measure an intervention's effect.
