Despite the key role of costs in determining which development interventions are implemented, cost analyses have not seen the same rise in prominence in the last decade as impact evaluations. To find out why, and to discuss how to bring cost evidence to the forefront, more than 200 people tuned in to the first of 3ie’s expert panels on cost evidence on Thursday, part of our ongoing Virtual Evidence Weeks.

The problem is not a lack of demand for cost information, panelists agreed.

The interest in cost analyses often becomes evident after an impact evaluation has been conducted without one.

“The very first question a policymaker will ask is, ‘That’s great, we’re glad to see that there’s impact, but what will it cost?'” said Elizabeth Brown, who leads the cost transparency initiative at the Center for Effective Global Action.

Where cost analyses are conducted, policymakers do use them. Two panelists offered separate examples from Kenya and Mexico where impact evaluations with cost analyses led government units to implement interventions that had been found to be effective, both in impact and in cost effectiveness.

Nonetheless, most impact evaluations don’t say anything about cost. Brown’s research for the World Bank showed that only 14 to 18 per cent of published impact evaluations had any kind of economic or cost-benefit analysis.

For some sectors, the picture is even worse, said Mark Sundberg, deputy vice president and chief economist of the Millennium Challenge Corporation. While cost-benefit analyses of large infrastructure projects like roads or power lines can be straightforward, cost analyses in other sectors are more difficult.

“The area I would say is most difficult is in policy and institutional reform,” he said.

One big reason for the lack of cost analyses is that they are often not explicitly included in the process by which evaluations are funded, said Donfouet Hermann Pythagore Pierre, associate research scientist at the African Population and Health Research Center.

Their rarity is also a self-perpetuating problem – cost analyses are of limited usefulness when there are few others to compare them to.

“The lack of international or national benchmarks … represent great barriers for promoting interest in carrying out cost benefit analysis,” said Thania Paola de la Garza Navarrete, head of CONEVAL’s evaluation unit.

Furthermore, professional incentives in academia discourage researchers from investigating costs. The journals which publish impact evaluations are much less interested in publishing cost analyses, Brown said. Accordingly, researchers focus on estimates of impact, not of cost.

One other issue that panelists raised was the quality of the cost analyses themselves. There needs to be transparency in methodology and consistency in its application, Sundberg said.

“If you have the impression that it’s who the modeler is who generates the results … you’ve lost the battle,” he said.

Watch the whole panel discussion here.