Impact evaluation methodologies

'Quantitative versus Qualitative' or the need for Mixed Methods

January 13, 2011

Speaker: Howard White

The use of both qualitative and quantitative methods in an impact evaluation can be crucial to understanding whether a programme will work or not work. Mixed methods can help in outlining the causal chain and firming up a clear identification strategy in impact evaluation.

Dr. Howard White, Executive Director of 3ie cited several examples at the Delhi seminar of how qualitative research can feed into evaluation and contribute to rich and rigorous analysis.
The Bangladesh Integrated Nutrition Project was, for instance, initiated in response to the very high level of malnutrition prevalent in the country. Central to this project was nutritional counseling to expectant mothers. However, the programme was not able to achieve the desired outcomes due to inappropriate targetting. Anthropological studies and focus group discussions in this case revealed that women in rural Bangladesh rarely moved out of their house and that mothers-in-law were in fact often the key decision makers of a household. A successful nutrition programme would therefore have to target mothers-in-law for making any sort of headway with behavior change.

“The failure of the programme showed that there was a knowledge- practice gap. Impact evaluations are about counterfactual analysis. But they also require qualitative and factual research,” said Dr. White.

Dr. White said that a significant challenge to integrating qualitative and quantitative studies was a ‘silo mentality’ that exists among researchers of different disciplines. “While anthropologists, economists and sociologists may conduct parallel studies they do not necessarily work towards integrating findings from their research”

“What we could do for more integrated studies is work on a sound theory of change and have team members from multiple disciplines talk to each other. This can help in cross- fertilization from one study to another,” he added.

Responding to questions on the standardization and credibility of qualitative data, Dr. White said, “Both qualitative and quantitative data can be ‘bad’. It is rigorous collection and analysis that can help capturing patterns in the data”.

Download Dr. White’s presentation (6.2 MB)

Scroll to Top