Carlos Oya and colleagues recently published a systematic review of agricultural certification schemes that stands out for me as useful research for informing policy and programming. Why do I say that? Agricultural certification schemes set and monitor compliance to voluntary standards with the objective of making production socially sustainable and terms of trade fairer for smallholder farmers and workers. The rise of voluntary agricultural production standards and their respective certification schemes has accelerated in recent years. This makes this systematic review both necessary and timely. There have also been several important recent developments in certification standards that increase the review’s relevance. Supermarket chain Sainsbury has launched its own certification scheme, while the Rainforest Alliance and Utz certification standards have merged.

This review is useful for other reasons as well. A single impact evaluation can give us answers that may hold true for a specific context; but the findings may not be more widely generalisable. A quality, mixed-method systematic review, however, draws on a wide variety of evidence to explore whether and how a programme works at each stage of the causal chain. It can also tell us for whom the programme works and why. Carlos’s review took a considered approach to a complex area. It drew on a wide evidence base – 43 quantitative impact evaluations that explored the effectiveness of certification schemes; and 136 qualitative studies that shine the light on the barriers to and enablers of effective certification schemes.

I recently had interesting email exchanges with Carlos, Antonio Capillo (Fairtrade Foundation) and Harveen Kour (Fairtrade International).  I asked them to comment on the review’s findings and give their views on wider implications for agricultural certification schemes. All of them reviewed this blog and approved of its content. Here are highlights.

Daniel: Congratulations, Carlos, on completing your review on the impact of certification systems on well-being. Can you tell us a little about the main findings from the review for policy, certification schemes and for research?

Carlos: Evidence suggests that although certification schemes improve prices and income from agriculture, they do not automatically lead to an increase in household income and wages, nor do they always improve education and health outcomes. This may reflect the fact that certification schemes are more likely to be effective in improving some immediate outcomes, while more indirect outcomes and longer term impacts are affected by a wide array of contextual factors that may make certification less effective than expected.

Certification schemes can be quite diverse. For instance, some schemes implement capacity-building initiatives that focus on empowering producer organisations and strengthening their position in the value chain, while others aim to contribute to yield improvements. Some include direct price interventions while others focus on interventions to improve quality and help producers access more remunerative markets.

The review also highlights the need for certification to extend relevant labour standards to all forms of agricultural production and establish enforceable labour standards. This would of course need to be complemented by strong national and international systems regulating trade and labour conditions in global supply chains.

The review also shows the paucity of rigorous evidence on impact. The number of certification schemes for which evidence is available is limited, reflecting a bias towards a few schemes, especially Fairtrade. More importantly, much can be done in terms of improving the quality of existing impact evaluations through theory-based approaches that ensure outcomes are measured consistently along the causal chain. We also need better reporting of methods and statistical procedures to improve the credibility of the evidence base.

Daniel: What do Fairtrade regard as the most important findings from the review, and how might you use them to inform your policies and programmes, and your monitoring, evaluation and learning framework?

Fairtrade: While it is positive that certification schemes can lead to better prices and increased incomes, the study is a crucial step to inform the way forward. It helps the Fairtrade movement and the sector focus greater attention on challenging the market dynamics that still limit farmers’ and workers’ access to decent livelihoods in global value chains.

It is of great relevance to us that the review did not find rural certification schemes to positively impact primary wages for workers engaged in certified production – winning progress towards living wages is a cornerstone of Fairtrade’s global strategy. We have been strengthening our work in this area through our regular Standards reviews and engagement with partner organisations, companies, producer representatives and trade unions. For example, the recent revision of our standards requirements for flower farms aims to tackle extremely low wages in the sector. But it is important we don’t do this in isolation, and that’s why we are actively involved in the Global Living Wage Coalition.

We wholeheartedly agree with the finding that context is particularly important in achieving sustainable impact. It supports our advocacy efforts and our drive for more tailor-made programmes to complement certification. Fairtrade recognises that the systemic and pervasive nature of certain core challenges in supply chains – such as gender inequalities and poor livelihoods for informal labour force – require context-driven interventions to address these deep rooted issues. We are therefore designing programmatic interventions that aim to achieve sustainable impact, are scalable and address endemic issues within key supply chains.

Last but not the least, the overall conclusion of the review is that there is limited and inconclusive evidence on the effects of certification schemes.  This finding is crucial for the sector. These recommendations are particularly helpful to Fairtrade as we renew our commitment to promote reliable and open-access research and evaluation and strengthen our research protocols and policies. We are a learning organisation. That’s why we will continue to invest in robust monitoring, evaluation and learning systems to assess quality and effectiveness of our interventions and adapt according to what works.

Daniel: The sustainability landscape is changing, with some certification schemes merging. At the same time, major companies are launching their own in-house schemes. What lessons can we draw from the review regarding the rapid developments in this field and how could future research address the new challenges and opportunities they bring?

Carlos: Our review documents the growth and proliferation of certification schemes and standards. This growth has led to instances of producers being certified by different standards systems simultaneously, despite sharing many objectives and requirements in common.

As the demand for ethically-produced products expands, more actors, including retailers, are compelled to certify compliance with ethical standards across a wider range of products. Since certification is costly for both producers and standards systems, pressures to merge or rationalise the adoption of standards are understandable and may potentially lead to less complex options for consumers. This can be a positive development as long as new (merged or joint) standards add to what is already being achieved by existing standards systems.

From the point of view of researchers, overlapping processes and schemes make impact evaluation more challenging as the number of confounding factors multiply, making it harder to attribute outcomes to particular schemes or interventions. A narrower set of standards systems with more clearly defined interventions could facilitate the evaluation of each intervention within a given certification scheme. Clearer distinctions between schemes and types of interventions would make it easier to attribute impacts to certification. Of course, all certification systems must be held to the highest standards of performance and be subject to rigorous and independent assessment.

Fairtrade: Our message to brands, traders and retailers is that to be truly successful in dealing with today’s most urgent issues, such as living incomes and wages for producers, none of us can act alone and everybody has to play their part. Our offer is that of collaboration and partnership.

Whilst we welcome the ambition of companies to do more to take responsibility for sustainability in their supply chains, establishing in-house initiatives requires significant investment and expertise.  We would urge any business to work with the sector and to ensure they have processes in place that mean they can be genuinely accountable for their ethical claims to farmers, producers and consumers. Hence, we would also encourage companies to open their doors to independent researchers to assess and measure the effectiveness of their schemes, too.

We also have common interests with many certification schemes. ISEAL provides an important platform for all of us to exchange ideas, learn from best practices and explore innovative ways to disaggregate impact, which is particularly relevant, as many producers opt for more than one scheme. Given the complexity of the changing landscape we find ourselves in, we need more collaboration between researchers and certifiers. We would also be open to exploring investing in methodologically sound research jointly with others to increase our effectiveness and to deliver learning that is applicable at scale and can be put into practice as soon as possible.

To find out more, readers can explore the full systematic review, a summary or a brief.