At 3ie, our mission is to fund the generation and sharing of sound, useful evidence on the impacts of development programmes and policies work. Actually, we’re more curious (or nosy) than that. For impact evaluation that matters, we need to know which bits of a programme worked, which didn’t, why and through which mechanisms, in which contexts and for what costs.

Why do we need all this information? Because we want to keep increasing the quality and relevance of the evidence we fund. For the past five years, we have funded impact evaluations and syntheses of such evaluations. Through this, we’ve learned some important lessons about what it takes to get quality, useful impact evidence. Our experience indicates — and so our working hypothesis is — that requiring and fostering engagement at the beginning of the study design process will improve study quality and uptake of the generated evidence. This engagement is both between researchers and implementing agencies and between researchers and 3ie.

We promote this engagement through a new proposal preparation phase within our grant-making cycle. This involves a small pot of seed money and a lot of dialogue. Teams get vital inputs at the right time to shape questions that matter to key decision-makers and study designs that can provide rigorous answers.

By including implementing and other stakeholders, there are opportunities for more decision-makers to gain ownership in the design and, in turn, the results. We think that this increases the likelihood that the studies will be useful to and used by those who have commissioned them.

This summer, we awarded these preparation grants for the second time, through our Climate Change and Disaster Risk-Reduction Thematic Window, funded by Danida.  At our global conference Making Impact Evaluation Matter in Manila this week – also the mid- point of the preparation phase — we held an inception workshop. This brought together nine teams of researchers and implementers planning studies on REDD/REDD+ programmes and early warning systems across nine countries.

We set some goals for our inception workshop. A central goal was for researchers and implementers to engage on a common understanding of the theory of change under study. Moreover, we wanted them to collaborate on a set of rigorous research questions that could help fill knowledge gaps and drive policy and programmatic decision-making.

Towards this goal, two members of each team (an evaluator and an implementing partner) presented their preparation progress to other teams and 3ie. These included some exciting questions and designs, including grappling with the multi-layer, multi-level and politically sensitive interventions related to REDD+ as well as with the infrequent – but devastating – occurrences of natural disasters such as floods from rising sea-levels and landslides from rising river levels. We were particularly encouraged to see different teams working on the same country looking for ways to collaborate and share information.

For each presentation, the team received constructive feedback on their design and had the chance to benefit from peer-sharing and collegial learning.  3ie had a chance to give its inputs directly, face-to-face and more dynamically than in the past.

Of course, we also allotted time for feedback and evaluation of the preparation process, so we can inform and improve our own organizational processes with evidence. Our REDD+ and disaster risk-reduction system participants told us that they found the preparation phase and the inception workshop to be very worthwhile.

These changes in our grant-making mean that preparation grantees are getting more engagement with and guidance from us. In particular, grantees felt they benefitted having check-in points to dialogue with 3ie and from the requirements to visit the study country, meet with partners and hold at least one stakeholder workshop. These processes helped ground their questions and designs in the physical, institutional and political realities of the countries.

We expect that this approach will help teams submit stronger and richer proposals in mid-October. We’re looking forward to see how much further teams have been able to push their ideas by then towards our goals of generating sound, useful evidence.

Watch interviews of researchers and implementing agency representatives highlighting the need for evaluating early warning systems in Nepal as well as the importance of stakeholder engagementin evaluating the impact of REDD+ activities in the country.