Since the 1990s, many multi-lateral and bi-lateral donors have expanded their peace-making and peace-keeping assistance to conflict-affected countries to include peace-building activities. The objective of these interventions is to prevent the conflict from reoccurring and return countries to a stable situation in which the economy can operate.
Some examples are programs to reintegrate ex-combatants into communities, mobilize individuals to work as communities to reduce violence and increase security, address war crimes through justice mechanisms, and support media to inform citizens and encourage dialogue. Over the years, these interventions have shown mixed results and learning has been limited in large part by the lack of rigorous impact evaluations. Policymakers and program managers often argue that impact evaluation is not possible in unstable or conflict-affected environments and that the need for rapid programming precludes a carefully planned study.
Cyrus Samii, Monika Kulma, and I are conducting a review study to demonstrate to policymakers and program managers that impact evaluation of peace-building, or stabilization, interventions is indeed possible. Our paper begins by reviewing the portfolio of stabilization interventions across different categories of programming. To frame our review, we focus on U.S. government funded interventions. While we have identified over 150 current or recent US-funded stabilization interventions, we have found only a few dozen evaluations of such programs, and only one rigorous impact evaluation.
But the US experience is not entirely representative. When we looked more broadly, we found more than 20 completed or ongoing impact evaluations of stabilization programs, with Cyrus serving on the research teams for a handful of those. These evaluations provide some useful insights and lessons for how to conduct impact evaluations of stabilization programs.
One initial finding is that quasi-experimental designs are quite useful in these settings, as often there is not the time or political will to randomize implementation. We also found some good examples of RCTs, so they are possible as well. Another lesson from the experience to date is that a key element of these evaluations is outcome measurement. Stabilization outcomes are not as easily defined, much less measured, as school attendance or diarrhea incidence. In fact, impact evaluation of stabilization interventions is very much a multi-disciplinary field, where both the theories of change and the outcome measures used in the studies come from fields including psychology, sociology, political science, and economics, among others.
We hope that this paper, when complete, will help policymakers and program managers for stabilization interventions be more open to, and creative about, impact evaluation. As USAID Administrative Rajiv Shah said in his Stabilization Guidance, “Every activity is an opportunity to learn what works, what does not, and why.”
View Dr Brown’s presentation made at 3ie’s recent Delhi seminar.