Water provision remains high on the global development agenda including political commitments such as the Millennium Development Goals (MDG) and associated post-2015 targets. By 2012, the United Nations declared that governments had met the MDG drinking water target to ‘halve the number without access to safe drinking water (defined as access to water from an improved source within 1 kilometre of the household).’  This suggests that some development efforts are working.

When your target is only 50 per cent, it is not a surprise that the scale of the remaining challenge is great. Seven-fifty million people, many living in Sub-Saharan Africa, still do not have access to safe drinking water. For those who do have nominal access, water supply is often unreliable and frequently takes more than 30 minutes to collect. And the world is long way away from providing the basic sanitation services needed to free communities from open defecation.

Water is central for efforts to improve quality of life for the world’s poorest people. It can support virtuous development cycles and pro-poor growth. In contrast, poor access to water and unhygienic sanitation conditions are likely to explain why some countries such as India have worse child malnutrition outcomes than their income levels alone would predict.

In theory, clean drinking water helps combat diseases such as diarrhoea, which kills almost 2 million children aged under-5 each year – six times more children than global conflict. Water can also enable hygienic washing practices which reduce exposure to respiratory infection and parasites like worm infections (which cause malnutrition) and trachoma (which causes blindness). And water poverty is gendered. It is typically women and girls who are subject to the drudgery of carrying water over long distances to the household. This potentially exposes them to musculoskeletal injury and physical attacks, and takes away time that could be more productively spent. What remains at issue, however, is the extent of evidence supporting such claims and the likely size of these impacts.

3ie has produced an evidence gap map that addresses this issue by consolidating what we know about what works in the water, sanitation and hygiene (WASH) sector. The gap map has been developed as a tool that can help policymakers access rigorous systematic review evidence on WASH and assist researchers and funders for determining priorities for conducting future systematic review and impact evaluation research. The results of the evidence gap map are available online and a report will be published shortly. The findings are summarised here.

We found, appraised and summarised 137 impact evaluations and 26 systematic reviews examining effects of WASH provision.  Evidence from systematic reviews suggests WASH interventions can make a big difference to combating infectious diseases and reducing child malnutrition. Hygiene promotion is probably the most efficacious way to reduce child diarrhoeal disease rates. However, recent impact evaluations, published since the systematic reviews were undertaken, have questioned the scalability of community programmes for promoting hygiene. And reviews of hygiene promotion also need to take into account the role of water supply as an enabling co-intervention more systematically.

In contrast, evidence suggests interventions to treat dirty drinking water do not lead to large sustained improvements in diarrhoea because uptake is not sustained and willingness to pay for water treatment is limited.

However, most systematic reviews do not attain the status of ‘high confidence in the review findings’. This is often due to the limited nature of searches for unpublished literature and the questionable rigor of many of the included studies. There are also concerns about methods of synthesis, for example where studies have used vote-counting rather than statistical meta-analysis, or, where meta-analysis has been used, more attention needs to be given to examining heterogeneity and grouping interventions appropriately. Many WASH systematic reviews are also simply out of date. In particular, systematic reviews of water treatment interventions need to be updated to include the most recent rigorous evidence from blinded studies which have called into question the reliability of self- and carer-reported diarrhoea data.

Although WASH sector researchers have shown long-standing commitment to theory-based impact evaluation, the evidence base remains small, especially for non-health outcomes, as shown in the figure.

The figure shows serious gaps in evidence for non-health outcomes. These outcomes are likely to be at least as, if not more, important to quality of life for programme beneficiaries as health outcomes since they are easily observed. In particular, data on productive sector and gendered outcomes are critically under-collected and under-reported. Not a single rigorous impact evaluation has attempted to measure impacts of water and sanitation improvements on women’s and girls’ safety and very few have examined the drudgery involved in water collection and transportation. More studies collecting data on intermediate outcomes like the time used to collect water or access sanitation are also needed to shed light on these factors.

The quality of the impact evaluation evidence has tended to be quite low in the past due to lack of rigorous methods, particularly for costly interventions like water supply improvements. Rigorous evidence can help us determine how we can most effectively meet these challenges and improve the lives of the most disadvantaged people around the world. Researchers have shown that it is possible to apply high quality evaluation methodologies, including randomized assignment, to evaluate piped water connection and sanitation impacts. 3ie has already supported several such studies.

In partnership with the Water Supply and Sanitation Collaborative Council (WSSCC), 3ie has recently embarked on a programme of research which aims to start filling some of the important research gaps by funding new impact evaluations and systematic reviews. We encourage researchers to look out for a call for rigorous systematic reviews of WASH-sector programmes that will be announced by next week.