Countries such as Sudan, Democratic Republic of Congo and Nigeria have large reserves of natural resources. They are also countries that have suffered extended periods of political violence, authoritarianism, corruption, inequality and poor growth.

What causes this imbalance of high wealth on one side and extreme poverty on the other? The correlation between the quantity of natural resources reserves and poor economic growth is generally considered to be proof of a natural resource curse.

Are countries blessed with large reserves of minerals, natural gas, petroleum and forests necessarily doomed to fail? Is the resource curse a reality? Can it be reversed? If yes, how?

Are good policies the answer? Can policies that encourage information flow and political participation help in ensuring accountability and better development outcomes?

In recent years, global initiatives to improve transparency in revenue collection from extractives, such as public reporting of taxes and other payments received by the governments, have proliferated. For example, the Extractive Industries Transparency Initiative seeks to improve government revenue disclosure, while Publish What You Pay asks large corporations to publish what they pay to governments.

There are several national and sub-national level initiatives that created independent watchdogs, ring-fence resources to spend on specific sectors and prescribe specific ways in which revenue should be shared among local and federal governments (see here for a couple of examples that are currently being evaluated).

However, the evidence on what works and why in promoting transparency, accountability and achieving better development outcomes is just very sparse (Acosta, 2014; Leavy, 2014)

The evidence gap

There are several reasons for this evidence gap. Establishing a rigorous control group and a counterfactual for attributing the observed changes to transparency and accountability initiatives (TAIs) is often hard. This is mainly because these initiatives are mostly nation-wide standards or soft guidelines that apply to all the players in the sector; they are a bundle of complex activities that engage multiple political actors and watchdogs; and they are targeted and implemented in a non-random way.

Further, the theory of change that outlines how better information disclosure (transparency)  leads to restrained government discretionary spending (accountability), which in turn helps in achieving better development outcomes, is unclear. Estimating impact therefore becomes difficult.

Although there are difficulties in evaluating impact, it’s also clear that the possibilities still exist. At 3ie, we aim to fill this critical evidence gap with rigorous impact evaluations. We recently invited impact evaluation proposals under the DFID-supported Transparency and Accountability Thematic Window. These impact evaluations will use innovative methods to evaluate TAIs in the extractives sector of a selected set of low- and middle-income countries.

What did we do?

Many research teams have expressed interest in undertaking impact evaluations. Additionally agencies working in extracting natural resources in Ecuador, India, Kenya, Mongolia, Mozambique, Myanmar, Peru and Uganda have also come forward for having their initiatives evaluated. At a recently held workshop for 3ie’s Transparency and Accountability Thematic Window, we asked all of them a question: How can we learn more about what works and why in TAIs?

So what did we learn from our dialogue?

This dialogue between research teams and implementing agencies continues. But important insights have emerged from our first impressions.

Broaden the focus: Transparency initiatives shouldn’t only be confined to disclosing information and actions related to revenue collection such as taxes and other payments. We should also be thinking about evaluating policies that relate to contracting, licencing or procurement, distribution of income among central and local governments, and independent oversight of public expenditure and sectoral allocation of revenues that are derived from natural resources.

Engage with people: Focus on engagement mechanisms such as providing customised information, promoting diverse platforms for debate, and assessing whether these are necessary – and sufficient – to improve transparency. See here and here for examples of citizen engagement.

Emphasise cross learning: While there are few impact evaluations of TAIs in the extractives sector, there are several impact evaluations of interventions to improve transparency and accountability in service delivery and budgetary processes. We should use what we have learned from these fields to understand how information for example can be presented in different and more effective ways to inform debate and better accountability.

Be aware of the short time span: We need to be realistic about what can be achieved. It may be more practical to examine intermediate outcomes such as delivering services rather than long term and difficult-to-measure outcomes such as livelihood improvement and poverty reduction.

It is clear that we need to use innovative approaches for impact evaluations of TAIs. This will require researchers and practitioners to come together to and jointly answer questions of how rigorous evidence can be generated.