Co-producing evidence gap maps with IDB to strengthen an evidence-informed culture in Latin America and the Caribbean

Over the past year, 3ie and the Inter-American Development Bank (IDB), a founding partner of 3ie’s Global Evidence Commitment, co-produced two interactive evidence gap maps (EGMs): one on policing and citizen security, and one on transportation. Both are now publicly available, and their influence is already visible in how IDB teams design evaluations. This blog describes how that happened and what we learned along the way.

Starting from what IDB already had

IDB had invested substantially in knowledge management, building repositories of impact evaluations relevant to its work. The partnership aimed to use that body of work systematically, bringing together IDB’s sectoral knowledge and operational priorities with 3ie’s methodological expertise. EGMs became the vehicle for doing so: a structured way to turn an existing literature base into a map that decision-makers can navigate and rely on.

Co-producing the maps

The tools are the product of genuine collaboration. IDB’s teams brought deep domain expertise, and 3ie brought complementary methodological training and tools. The IDB team also focused on aligning these maps with IDB’s portfolio and how project effectiveness was evaluated to improve their use and impact.

For the Citizen Security EGM, 3ie and the IDB team worked through the methodology phase by phase: scoping, search, screening, data extraction and mapping, and critical appraisal. 3ie's methodological feedback at each stage was designed not just to refine the map, but to explain why each methodological choice matters. The EGM is now live on IDB’s evidence platform for security and justice, where it serves practitioners and decision-makers across the region.

Working with 3ie has strengthened the IDB’s capacity to systematically identify, screen, assess, and synthesize evidence in the citizen security sector. From a policy perspective, there is often pressure to identify and implement solutions quickly, given the urgency of the challenges facing the region. Our collaboration with 3ie, and the opportunity to benefit from its methodological expertise, strengthened our capacity to apply rigorous, transparent, and structured processes to generate relevant, replicable, and therefore more reliable information. This approach has helped ensure that our work is grounded in a robust understanding of the available knowledge and better positioned to effectively inform decision-making and policy dialogue with our clients, as well as our own knowledge agenda. The Evidence Gap Map on policing has provided our teams with an accessible and practical resource for navigating the existing body of evidence. By making research findings easier to identify and use, it has supported analytical, operational, and advisory work across the region. We look forward to continuing to update and strengthen this tool so that it continues to contribute to evidence-informed approaches in the sector.

Sergio Britto Lima, Citizen Security Division Consultant, IDB

Building capability alongside the maps

For the Transportation EGM, 3ie ran a multi-session training programme, including in-person sessions and an open webinar for economists, technical specialists, consultants and junior researchers. It covered the full arc of EGM methodology, including critical appraisals of relevant systematic reviews. 3ie and IDB teams co-produced this EGM, adhering to the highest methodological standards. The resulting map is publicly accessible on 3ie’s Development Evidence Portal, with a beta AI-assisted chatbot designed to make the map easier to navigate. As needs emerged, we also developed additional features in the online maps for translation and category aggregation.

Beyond the two maps, 3ie shared templates for EGM protocols, screening, data extraction and critical appraisal. IDB’s teams are now using these on their own. A co-produced user guide and technical note give other IDB divisions a roadmap for producing EGMs of their own.

Working with 3ie has been a great experience. Their commitment to maintaining high standards in their work is something we share at the IDB. The Transport Evidence Gap Map we co-produced has allowed us to learn about the current state of impact evaluation in our sector, including how the evidence has grown, where it is concentrated, and where gaps remain. We found these patterns so interesting that we decided to write a note about them, which we will publish soon. We are currently mapping the evidence to the indicators used to track our projects’ results. I believe the Transport EGM will play a critical role in making project design and implementation more evidence-based.

Leonel Borja, Transport Division Consultant, IDB

Seeing evidence shape evaluation design

The Transportation EGM is already shaping evaluation work across the IDB Group. It has helped the IDB’s Transport Division ask sharper questions, strengthen theories of change, inform strategy, and make more evidence-informed decisions across countries and contexts. Its influence is also extending to other IDB Group units, most recently through an ongoing road rehabilitation evaluation across Brazil, Ecuador, and Panama led by the Office of Evaluation and Oversight.

What has been most rewarding is seeing the work move beyond the maps themselves. From the outset, our shared goal was not simply to produce high-quality EGMs, but to strengthen the capacity within IDB to use systematic evidence in its own work. IDB colleagues really invested in the process, engaging deeply with the methods, asking difficult questions, and building the skills to apply them independently. Seeing the Transportation EGM now being used to strengthen theories of change, evaluation questions and data collection shows what this kind of partnership can achieve.

Shannon Shisler, Lead Evaluation Specialist, 3ie

Using EGMs to institutionalize evidence use

The most durable outcome of this partnership is how IDB’s teams now use evidence. The maps clearly show what is known and where the gaps are, and the templates and co-produced technical note let other divisions build on the same approach. IDB now approaches evidence synthesis as a powerful way to learn about the effectiveness of its interventions and programs rather than depending on new impact evaluations alone. This is reflected in how IDB is extending these frameworks to EGM work in other divisions, and its investment signals a commitment that goes well beyond these two maps.

Working together also refined practices that strengthen any evidence synthesis: looking beyond well-known journals to include grey literature and reduce publication bias, recognizing that study design alone is not a sufficient proxy for quality, and weighting findings by study quality rather than only counting positive and negative results.

Tanvi and Annie Vincent contributed to writing this blog.