Open defecation poses significant health risks for individuals and communities across the globe. The practice affects vulnerable populations through diseases such as diarrhoea, schistomiasis and trachoma, which often lead to stunting and malnutrition in children. It is particularly prevalent in India, which is home to 59 per cent of the 1.1 billion people in the world who practice open defecation. It is a major cause of diarrheal deaths among children under five in India, and constituted 22 percent of the global disease burden in 2015 (UNICEF 2018).                                                                                   

To overcome the challenges associated with open defecation, we need, first, accurate estimates of its prevalence, and second, rigorous evidence on what works to promote latrine use. This information would be imperative to inform sanitation policy within and beyond the purview of the ongoing national sanitation campaign, the Swachh Bharat Mission (SBM, Clean India Mission), which aims at making the country free of open defecation by 2 October 2019.

Responding to the need for rigorous evidence on what works to promote latrine use in rural India, 3ie launched an evidence programme that supports four impact evaluations of contextually informed interventions, rooted in behavioural science, to promote latrine use. These evaluations are underway in Gujarat, Bihar, Karnataka and Odisha and will be completed in 2019.

The challenges of measurement

Measuring sanitation coverage and uptake is imperative to shaping policy in a country that hopes to be open defecation free (ODF) in a short span of time. Recent estimates provided by the Ministry of Drinking Water and Sanitation, and the National Annual Rural Sanitation Survey (NARSS 2017) report that the SBM (rural) has been successful in improving latrine coverage and use by a significant margin since 2014. However, from ongoing projects, we learn that there are several challenges to the accurate measurement of defecation behaviour in India. Some of the key barriers to this include:

1. Proxies for latrine use and social desirability bias

Latrine use has typically been measured through the construction of latrines. This could be because the presence of infrastructure is relatively easy to verify. However, latrine presence does not necessarily amount to use.  (3ie 2017)

Measuring the sustained use of water, sanitation and hygiene (WASH) infrastructure is uniquely challenging in the midst of a nationwide campaign advocating for the importance of ‘correct’ defecation behaviour. More recently, surveys have relied on respondent reported usage of toilet infrastructure, which can be prone to social desirability bias. (Coffey and Spears 2014). Asking questions about inherently private behaviours such as defecation practices could elicit desirable responses, especially in ODF declared and verified areas.

For example, in the surveys that feed into the WHO- UNICEF Joint Monitoring Program for Water Supply and Sanitation (JMP), such as the Census 2011, the National Family Health Survey (NFHS-4) and National Sample Surveys (NSS), open defecation is deduced once all other forms of access to sanitation infrastructure have been negated. In addition, the survey questions may lead to biased measurements of latrine use, as they refer to ‘usual’ usage or ‘always’ usage (see box one).

Box one: Latrine use questions in NFHS-4 and NARSS 2017

In order to tackle inflated responses, surveys have begun to rely on the use of structured observations and spot checks. For example, checking for physical signs of use such as slippers, water containers near the latrine, etc. However, using one or the other may lead to inaccurate estimations. Combining structured observations and spot checks, with self-reported latrine use may be one possible solution. But employing the use of both on a large scale may have significant cost implications.

2. Capturing variations within households

Another challenge to the measurement of latrine use lies in adequately capturing variations within households. Surveys frequently fail to unpack usage to investigate individual latrine use due to resource constraints and logistical timelines. Better understanding individual use is important, and a recent investigation of sanitation coverage, usage and health in India concludes that surveys which aggregated to the household level revealed lower levels of partial usage, while surveys investigating individual use found a higher incidence of partial use (Viswanathan 2017). National surveys such as the Census 2011, the NHFS 4 and the NSS also tend to aggregate measured behaviour either to the household, sub-household or demographic group level, overlooking individual behaviour.

While the latest Census (2011) collected usage data indirectly through latrine ownership, the 69th round of the NSS finally introduced questions on use, in particular asking if “all household members of categories specified are using a latrine, followed by the list of categories: men/women above 15 years of age, and men/women below 15 years of age”.

Learning from experiences in measurement, the NARSS (2017) measured both individual and sub-group latrine use. However, it subsumed latrine use into the categories of always, rarely, sometimes and never, conditional on shared and unshared latrines owned by households. This may lead to inaccurate responses, as the terms are not precisely quantifiable.

3. Seasonality implications

Whether or not a latrine is being used depends on water supply, which differs by season. Furthermore, the rainy season could deter many from defecating in the open. Dry season bias in household survey data collection (Wright et al., 2012) coupled with seasonal shifts in sanitation practices (Routray et al 2015; Sahoo et al 2015) are likely to cause some distortion in reported use. In order to tackle this, questions with response categories such as, “all the time/some of the time/ none of the time” could be better answered if they are framed around the time of day. In case of seasonality, respondents should be asked about their toilet usage in the recent past or within a specific time frame that can be accurately recalled. (Coffey and Spears 2014).

Toward more accurate measurements of latrine use

There is no ‘right question’ to capture latrine use in rural India. However, survey questions can be designed in a manner that allows for triangulation to determine latrine use, where self-reported use may be combined with structured observations in sample areas, or in sub-samples. These questions could also be framed while considering the nuances of seasonality, recall periods, and be administered at the individual level. These designs would have to be informed by evidence, the need for which has spurred a healthy debate among researchers and academics of late.

In response to this need for evidence, 3ie is supporting an independent measurement project, which estimates the prevalence of reported latrine use in a sub-sample of existing project areas in rural India. It compares the prevalence of reported latrine use in the same sample areas by asking two distinct survey questions. One question is taken as is, from the National Family Health Survey (NHFS-4), which draws on the recommendations of the JMP on water and sanitation. The other is a standard 3ie-r.i.c.e question developed through formative research in nine states India. (See box two)

Box 2: 3ie rice standard question on latrine use

In the interim, 3ie is has worked on a compilation of latrine use questions across major studies and national surveys. The compilation lists existing latrine use questions across key national surveys and studies, and hopes to aid researchers designing studies in the future. In the long term, it hopes to set the path for standardised questions for measurement. This is a living document, open to updates by researchers, implementers, policymakers and other stakeholders.

India represents an amalgamation of cultures, identities and social values that influence private behaviour currently under the spotlight. Much depends on researchers, implementers, policy makers and citizens to explore what works, why and in which contexts to reduce open defecation. This relies fundamentally, on objective and balanced measures of the successes and failures of sanitation policy in the country.