Improving vaccination initiation and completion via vaccine indicator and reminder bands: a formative study in Nigeria

What we offer
At the core of our work, impact evaluations are rigorous studies that measure the effects of international development programmes. We focus on conducting impact evaluations on policy-relevant research questions where credible findings can drive decision-making.

3ie invented evidence gap maps, which provide a visual overview of existing and ongoing studies or reviews in a sector or sub-sector of international development. These maps help policymakers find the most relevant evidence for a given question, thereby improving decision-making.

Our synthesis products, including systematic reviews, integrate findings from multiple different studies which address a common research question. Because they look systematically across the evidence base to see what works and why, systematic reviews and other synthesis products are more reliable for decision-making than results from a single study used in an ad hoc way.

Often, policy decisions must be made quickly. To make sure that decision-makers get the evidence they need when they need it, 3ie has developed a set of helpdesk services and rapid evidence products. Our Rapid Response Briefs and Rapid Evidence Assessments draw from the latest high-quality research to answer policy questions in a fraction of the time of traditional evaluations or systematic reviews.

We work with governments and other organizations to build their capacities in evaluation and evidence-informed decision-making. In addition to providing trainings on a wide range of evaluation methods, we offer ongoing support services to ensure our partners have access to the most up-to-date evidence and evaluation tools.

3ie specialises in increasing access to, demand for and use of evidence by governments, parliaments, programme managers, civil society, programme participants and the media. We do this by emphasising the value of planning and engaging with stakeholders to ensure that evaluations and reviews are relevant and useful. We use robust and effective monitoring to measure evidence use so that we can convey evidence impact on programmes and policies with greater confidence.

Since 3ie was founded, transparent, reproducible and ethical evidence (TREE) have been core considerations in our work. We have developed and refined tools and best practices to ensure our studies apply technically rigorous methodologies, transparently share design and analysis decisions, yield computationally reproducible analysis, and incorporate foundational principles of research ethics into design, implementation, and dissemination. 3ie’s transparent, reproducible and ethical evidence (TREE) Policy articulates this commitment to TRE best practices.
Evidence hub
3ie’s Development Evidence Portal is the largest-of-its-kind repository of rigorous evidence on what works in international development. This portal includes evaluations and synthesis of studies conducted in low-and middle-income countries. It combines records from 3ie’s Impact Evaluation and Systematic Review repositories, as well as, evidence gap maps.

These provide a visual display of completed and ongoing systematic reviews and impact evaluations in a sector or sub-sector, structured around a framework of interventions and outcomes.

Evidence impact summaries briefly describe how 3ie-supported evidence has informed and influenced decision makers. Each summary highlights verified instances of evidence impact.

We provide funding for replications, conduct in-house replication research and publish guidance on replication methodology. We also provide funding to original authors of 3ie-funded for preparing their raw datasets.

As part of our mandate as a knowledge producer and translator for our main audiences, we publish a range of knowledge products. These include briefs, impact evaluation reports, systematic review reports and summaries, replication papers, evidence gap map reports, scoping reports and working papers.

3ie’s Registry for International Development Impact Evaluations (RIDIE) aims to enhance the transparency and quality of impact evaluation research before it begins.
Our work
- Working with governments
3ie’s evidence programmes support studies to fill critical knowledge gaps in a sector, sub-sector or in an area with limited rigorous evidence. We fund studies under a specific theme or which address a particular question or set of questions in programme areas where our donors want to expand global public knowledge of what works and what does not.

Improving adolescent sexual and reproductive health requires decision-makers to have evidence on what works or not, for whom and why, particularly in low- and middle-income countries (L&MIC).

Despite the availability of agricultural technologies, few smallholder farmers in developing economies adopt new inputs and practices. One of the factors preventing this is the lack of effective knowledge dissemination.

Agriculture is a major source of sustenance for rural populations in low- and middle income countries. But owing to weather, pests, diseases and price fluctuations, farmers face numerous risks, including crop and livestock losses.

The production and consumption of fish, a nutritious source of food for around one billion people, is rising globally. The bulk of aquaculture still originates from small-scale farming in developing countries, such as Bangladesh.

According to the United Nations estimates, 103 million youth worldwide lack basic literacy skills, and more than 60 per cent of them are women.

3ie is working with USAID’s Center of Excellence on Democracy, Human Rights, and Governance (DRG Center), and NORC at the University of Chicago to increase knowledge on the global advancement of democracy, human rights and governance.

South Asia, home to 600 million children, has the highest number of young people globally. While these youth contribute to the social and economic stability and prosperity of their families, communities and countries, they face several threats to their health, education and protection, including pressures to drop out of education, become child workers, marry and reproduce early.

3ie projects and programs are uncovering ways to promote gender equality and women’s empowerment. Our cross-cutting work seeks to inform what makes development interventions gender-sensitive and transformative in low- and middle-income countries, including challenging fragile contexts.

3ie, funded by the Bill & Melinda Gates Foundation, is supporting studies and other work on HIV combination prevention to maximize the useful knowledge and policy implications from these interventions.

To gain a better understanding of whether providing HIV self-tests to people would increase HIV testing rates, and who might benefit most from availability of tests, 3ie funded seven pilot interventions and their impact evaluations in Kenya, Uganda and Zambia.

The Saamuhika Shakti or Collective Impact initiative aims to improve the lives of waste pickers in Bengaluru, in India’s southern state of Karnataka, through a coordinated multi-sectoral approach.

We are supporting the generation of rigorous evidence in humanitarian contexts on interventions related to water, sanitation and hygiene, food security, multi-sectoral humanitarian programming and interventions targeting malnutrition.

In alignment with our mission, 3ie promotes rigorous, efficient, and ethical use of innovative data sources for impact evaluations, including in those conducted by 3ie, by 3ie research partners, and in the global development community more broadly.

A major challenge in the fight against vaccine-preventable deaths and diseases is the limited evidence available on innovative and successful community-based approaches for expanding immunization coverage in countries with low or stagnating vaccination rates.

Much of the evidence surrounding the integration of HIV services with maternal, neonatal, and child health services, as well as with sexual and reproductive health and family planning services, does not come from rigorous studies.

Widespread hunger, malnutrition, and water insecurity have devastating and long-lasting impacts on the health and well-being of millions of people around the globe.

With more than three billion people living in contexts that are highly vulnerable to climate change (IPCC Report 2022), mitigation and adaptation strategies are essential to minimize the long-term effects of climate warming. 3ie’s climate change research program focuses on promoting evidence-informed policies and programs to strengthen climate mitigation efforts.

3ie, the World Health Organization and the Partnership for Maternal Newborn & Child Health worked together to create a gap map to assess the evidence available on social, behavioural and community engagement interventions related to reproductive, maternal, newborn and child health programs in low and middle-income countries.

Everyone needs reliable access to a sufficient quantity of affordable, nutritious food. However, all too often, this need is not fulfilled. Despite best efforts, significant gaps persist between global nutrition targets and actual achievements.

Fragility has expensive, long-term consequences and trying to build peace in situations of protracted conflicts is becoming the norm. In 2016, 1.8 billion people – nearly a quarter of the world's population – were living in situations of fragility. Social cohesion is widely considered important in building sustainable peace in fragile contexts.

Safe sanitation is a key determinant of many public health outcomes and ending open defecation is necessary in order to achieve safe sanitation. To this end, the Indian government has led a massive sanitation program, Swachh Bharat Abhiyan - Gramin, to improve latrine access and use.

Launched in 2011, the National Rural Livelihoods Mission (NRLM) aims to link the rural poor in India to sustainable livelihood opportunities and financial services.

We set up our Replication Programme to address the need for a freely available global public good that helps improve the quality and reliability of impact evaluation evidence used for development decision-making. Replication is the most established method of research validation in science, yet it has not been fully embraced by the research community or development donors, leading to this gap.

3ie, in partnership with the Water Supply and Sanitation Collaborative Council (WSSCC), developed this program to build the evidence base in L&MICs and to support global efforts towards increasing equitable access to and use of WASH services.

We support impact evaluations to build the evidence base on the effectiveness of interventions that reduce the risks faced by the poor through participation in public works and employment programmes.

3ie, with support from the Bill and Melinda Gates Foundation, is undertaking a learning study to understand the barriers and facilitators in programs related to fecal sludge and septage management in the states of Odisha, Tamil Nadu and Telangana.

3ie, in partnership with the Millennium Challenge Corporation (MCC), is working to strengthen evidence use in MCC’s program designs and investment decisions. 3ie will support MCC to ensure their economic modelling, project design and evaluation approaches are at the cutting edge of development science while simultaneously building a repository of learning that benefits other policymakers and funders.

India has one of the lowest female labor force participation rates in South Asia. Among rural women, less than 30 per cent are engaged in productive work, paid or unpaid.

The natural resource governance sector is under-researched and programs in it are under-evaluated. 3ie supported seven impact evaluations to fill critical knowledge gaps on what works to improve governance in the extractives sector in low- and middle-income countries.

3ie champions the research transparency and reproducibility movement as a means of understanding and mitigating challenges to the credibility of social science research, while also working toward stronger integration of ethical principles into practice.

It is widely recognized that preventive measures are crucial in tackling the HIV epidemic in Sub-Saharan Africa. Three randomized controlled trials and numerous observational studies have shown that male circumcision reduces HIV acquisition by approximately 60 per cent for men, suggesting that efforts to increase male circumcision can play a significant role in HIV prevention.

.

3ie, in partnership with the Philippines National Economic and Development Authority (NEDA) and Australia’s Department of Foreign Affairs and Trade, is implementing a multi-year Philippines Evidence Program (also known as Policy Window Philippines).

Working in collaboration with the Office of the Prime Minister, the primary aim is to improve developmental outcomes through evidence-informed decision making in Uganda. 3ie is currently supporting evaluation of government programmes around youth livelihood, family planning, public service delivery and local governance, and universal primary education.

3ie and the government of Benin are working on a a multi-year regional initiative that aims to promote the institutionalization of evaluation in government systems across eight countries in West Africa, including: Benin, Burkina Faso, Côte d’Ivoire, Guinea-Bissau, Mali, Niger, Senegal and Togo.
Funding
- Open opportunities
View our current funding opportunities for evaluations, systematic reviews and internal replication studies.

Publication Type: Other evaluations

Country: Nigeria

Region: Sub-Saharan Africa (includes East and West Africa)

Sector: Health Services

3ie evidence programme: Innovations in Increasing Immunisation Evidence Programme
Author(s): Chisom Obi-Jeff, Ebubechi Nwaononiwu, Nahum Nuhu, Kelechi Irechukwu, Suleiman Mahmud, Tobi Bamiduro, Chike, Nwangwu, Jamila Ibiye Bello-Malabu, Noor Sabah Rakhshani, Chizoba Wonodi
Institutional affiliation(s): Direct Consulting and Logistics Limited, Health Strategy and Delivery Foundation, Precision Development Research and Advocacy, Independent researcher from Johns Hopkins International Vaccine Access Center
Grant-holding institution: Health Strategy and Delivery Foundation
Main implementing agency: Health Strategy and Delivery Foundation; and Direct Consulting and Logistics Limited
Sex disaggregation: No
Gender analysis: No
Equity focus: No

Context

According to the World Health Organisation, Nigeria accounts for approximately 28 per cent of global child mortality due to vaccine-preventable diseases. In 2016, the coverage for the third dose of pentavalent and full immunisation were 33 per cent and 23 per cent, respectively. While there has been a significant improvement in delivery of Nigeria’s routine immunisation programme, delayed vaccination and low overall coverage are still major causes for concern. This study explores the feasibility and acceptability of using anklets worn by children as a visual reminder for caregivers, to encourage timely initiation and completion of routine immunisation.

Intervention design

This study evaluated the feasibility and acceptability of the Vaccine Indicator Reminder (VIR) bands. Encased in a silicon mould, the band comprised a time-strip indicator and a one-time locking button. The time-strip indicator consisted of a white membrane and a dye blister; pressing the blister activated the band, which caused a red-coloured dye to be released along the membrane. The progression of the ink on the membrane was calibrated to reach the end-point in four to six weeks.

Each infant enrolled in the study received three VIR bands: yellow, purple and green. The yellow bands were provided to infants upon receiving their Bacillus Calmette-Guérin vaccination (either at birth or in the first two weeks of life). The band’s time-strip was calibrated to reach the end-point in six weeks, which corresponded to the due date for the first dose of the pentavalent vaccine. The infants who returned for the pentavalent vaccine at six weeks received purple bands, which were calibrated to indicate the time for the second dose of pentavalent vaccine. When children returned for the second dose, they received green bands, which were calibrated to remind caregivers to return in four weeks for the final dose of pentavalent. The intervention engaged health workers and traditional birth attendants (TBAs) to enrol children in the study. The TBAs also received monetary compensation for the delivery of services.

The study hypothesised that the VIR band will serve as a visual reminder for caregivers to vaccinate their children on time and complete the pentavalent vaccinations.

This hypothesis was based on three key assumptions:

Community will accept the VIR band;
Gradual change in the colour of the time-strip would serve as an active reminder for caregivers to vaccinate their children on time; and
Health workers and TBAs would be able to utilise the VIR bands to trigger conversations around immunisation in the community.

Evaluation design and methodology

This was a mixed-methods study conducted in three wards of Bunza Local Government Area in Kebbi State, Nigeria.

The study enrolled 503 infants who were followed for 18 – 22 weeks, till they completed their three doses of pentavalent vaccine.

Quantitative methods comprised a household survey which was conducted at baseline. Exit interviews were conducted during endline with caregivers of children who returned for the third dose of pentavalent vaccine.

Qualitative methods, including focus group discussions and in-depth interviews, were used at baseline and endline to explore the communities’ knowledge, attitudes and practices related to immunisation.

Primary evaluation questions

This study answers the following questions:

Will parents/caregivers and the community accept the VIR band?
Will parents/caregivers retain the bands on their infants’ ankles for the duration of the study?
Is it feasible for health providers to incorporate the use of the VIR band as part of routine immunisation services?
Will healthcare workers follow the protocol for the appropriate use of the VIR band?

Primary findings

The authors reported that although vaccination knowledge and intention to vaccinate were high at 91 and 100 per cent respectively, and there were no stated socio-cultural impediments to vaccine uptake, motivation to vaccinate was still low. At endline, only nine per cent of enrolled children had received their third dose of pentavalent vaccine by 18 weeks. The most commonly cited reasons for under-vaccination largely pertained to caregiver complacency.

Of 503 children enrolled in the study, 155 returned for the third dose of pentavalent vaccination. Exit interviews with the caregivers of these children found that 97 per cent of them perceived VIR bands to be an effective vaccination reminder.

In-depth interviews with the caregivers, health workers and community leaders revealed that the overall acceptability of the VIR intervention was high and it was perceived as an incentive by several caregivers. However, there were some general concerns and hesitancy about the band design; primarily, caregivers found the concept of ankle bands strange and would have preferred wrist bands. Additionally, the red-coloured dye in the time-strip indicator was also found to be concerning as it had led some caregivers to erroneously conclude that the band was sucking the infant’s blood.

A major setback to the VIR band intervention was the high failure rate of the time-strip indicator, which led to delays in vaccinating many enrolled children. In addition, the limited availability of VIR bands due to import restrictions may have reduced immunisation uptake among caregivers who perceived it as an incentive. This, in turn, limited the study team’s ability to accurately measure the effectiveness of VIR bands.

Implications

Given the study findings, the authors conclude that successful uptake of interventions such as the VIR band, is contingent upon well-planned sensitisation and mobilisation strategies. The research team found that using existing trusted community structures, especially community leaders and TBAs, helped them address knowledge gaps and encourage uptake of the intervention.

Despite this successful engagement, the authors acknowledge that the VIR band technology itself had limited success. The authors recommend further research to assess the effectiveness of the VIR band as a reminder and an incentive to caregivers.

Related publication

Vaccine indicator and reminder band to improve demand for vaccination in Northern Nigeria: A qualitative evaluation of implementation outcomes, ScienceDirect, May 2020