Cost-effectiveness evidence can guide decision-making by showing which interventions offer the highest value for money, critical in an era of diminished resources for development. But, despite widespread acknowledgment of their importance, value for money analyses remain the exception rather than the rule.
Since 2021, 3ie has been tracking if impact evaluations in its evidence gap maps (EGMs) also report on costs. EGMs map where there is causal evidence on the effectiveness of development interventions, each focused on a specific domain such as nutrition or sustainable energy. Evaluations are tagged if they mention costs—from basic references to program budgets to detailed cost analyses like cost-effectiveness or cost-benefit assessments. Despite a few bright spots, only about 15 percent of impact evaluations in the average development domain even minimally mention costs.
A couple caveats—though, these don’t change the conclusion: To be fair, 11 of the 14 pictured EGMs were last updated in 2023 or earlier. If evidence production has accelerated in the last couple of years, that isn’t captured in our review. And for EGMs, we only search for—and review—a program’s primary impact evaluation. If costs are embedded in a wider ecosystem of papers describing program results, we miss them. But other 3ie efforts, including our systematic reviews, do search more widely. Again and again they come to the same conclusion.
What’s holding up the production and use of value-for-money evidence? Past research suggests that there is not enough incentive for production given that funders and academic journals don’t require it.
Finally, even when evidence is produced, it’s rarely used. Few dedicated efforts meaningfully and routinely translate cost-effectiveness evidence for policy aims.
A narrative-changing investment by USAID to promote impact and learning with cost-effectiveness evidence would have knocked down some of these barriers. Now, with USAID’s demise, the costing community must double down on efforts to elevate the agenda.
At 3ie, we’re working to build a stronger, more informed culture of cost evidence use in development.
- Our recently updated handbook on measuring the costs of development programs provides practical guidance for practitioners.
- The Journal of Development Effectiveness continues to spotlight costing in development, most recently through its 2023 special issue.
- Meanwhile, the launch of our Global Evidence Commitment has united development institutions around a shared pledge to strengthen the culture and practice of evidence use.
Other initiatives are emerging. Brookings has launched a Global Costing Task force for Education and is seeking input on its goal to grow the volume, quality, and use of cost data. CEGA is a standard bearer, producing high-quality cost evidence and releasing tools to help other researchers do the same. And, the World Bank’s Strategic Impact Evaluation Fund hosts a growing cache of guidance, costing tools, and research.
Our news isn’t new. A 2019 World Bank working paper examined a sample of impact evaluations published between 1986 and 2013, finding that few included value-for-money analyses. Today, our review of evidence from 3ie’s Evidence Gap Maps reveals much the same. What will we find tomorrow? Reach out at bhutchinson(at)3ieimpact(dot)org to share the resources your organization is developing, to explore collaboration opportunities, or simply to connect with like-minded colleagues. Read more about our cost evidence work here.
