Impact in Global Environmental Evaluations
Speaker: Robert D. van den Berg, Date: November 9, 2012
The Global Environment Facility (GEF) is the largest environmental fund in the world. It brings together 182 countries, international institutions, civil society organisations, and the private sector to address global environmental issues while supporting sustainable development initiatives.
With a view to examine the effectiveness of its grant modalities, GEF has been developing specific impact methodologies for evaluating GEF-supported interventions, strategies and policies. The organisation’s impact assessment toolkits have included methods such as quasi-experimental studies, case studies, network analysis and the like.
Initial studies indicated that quasi-experimental work and randomised controlled trials could only contribute to a limited extent in understanding GEF promoted impact. This is mainly due to the aggregated nature of GEF supported interventions, which consist of a relatively high number of sub-interventions, and are sometimes focused on highly different goals and modalities. Furthermore, many interventions aim to demonstrate innovations and new approaches, for which counterfactual testing seems appropriate, but the uptake of demonstrations in national policies, which is an important element in GEF strategies, needs to be tackled with other evaluation tools. Over the past two years, GEF has fine-tuned the theory-based approach into a full blown “theory of change” approach that aims to identify why policies, strategies and interventions are assumed to work, so that appropriate evaluation methods can be applied.
The theory of change approach has been further tested in two evaluations: an impact evaluation of GEF support to the South China Sea and Adjacent Areas, and an evaluation of the GEF focal area strategies. The first is an impact evaluation focusing on longer term and aggregated trends. The second is a formative evaluation to identify assumptions in the current strategies that could be strengthened during the next phase of GEF, or could lend themselves for testing through, amongst others, RCTs. In addition to this, a new objective is to mainstream impact throughout other evaluation streams in GEF, including performance evaluations.
The presentation focused on the various forms in which impact work is taking shape in GEF, including experimental studies, modeling, negative evidence, contribution analysis, case study work and through interacting with the scientific community. The different roles of these impact assessment tools in the overall impact efforts of the organisation were also discussed.