A mother with a six-month old girl receives a voice reminder in her local dialect to take her daughter for her next vaccination. She comes to the local health facility where the health worker takes off a small pendant attached with a black thread from the child’s neck, which has an embedded digital microchip. She places it on the tablet, and all the immunisation information related to the child appears on the screen. The health worker finds that second dose of pentavalent vaccine is due. She administers it to the child. She enters this information on the tablet and it automatically gets uploaded to the central server. Health officials from local, regional and state levels can see in real time whether the immunisation camp happened on the scheduled day and time and how many children were vaccinated. After the mother goes back home, she receives a thank you message for bringing her daughter for vaccination and is informed about its possible side effects.

All this is happening in a remote tribal area in Udaipur district, in the Indian state of Rajasthan. Some years ago, the sight of digital pendants, tablets with chip or barcode scanning facilities and sending automated voice calls to people living in predominantly rural and remote areas to promote immunisation would have been inconceivable. In recent years, use of innovative technology-driven health interventions are gaining momentum. However, several questions about their feasibility, acceptability, affordability, impact and scalability need to be addressed. 3ie, through its Innovations in Increasing Immunisation Evidence Programme, is addressing these questions to improve immunisation of children in several contexts around the world, from Nigeria and Uganda in Africa to Pakistan and India in Asia. More than two years into the implementation of this programme, we have learned some important lessons on developing and implementing community engagement interventions using different kinds of innovative technologies. These lessons are illustrated using examples from various 3ie-supported evaluations, which are summarised below.

A thorough contextual assessment is necessary for designing technology-driven interventions, which are likely to be acceptable and feasible for the intended beneficiaries.

In the evaluation in Udaipur mentioned earlier, researchers found that they follow a local custom that mothers make their children wear amulets strung on a black thread (colloquially known as ‘tabeez’) to protect them from the evil eye. This helped them leverage the locally prevalent traditions and come up with the concept of digitally encrypted and wearable vaccine records on a necklace made with black thread.

In the Indian state of Haryana, researchers planned to provide sugar and mustard oil to incentivise caregivers to take their children for immunisation. However, through discussions with the local programme managers, they found their plan was not feasible due to the logistical constraints with procuring and managing large quantities of mustard oil and sugar. Based on prior contextual analysis, they knew that most of the households own a mobile phone, which led them to the idea of offering mobile recharge vouchers as incentives instead.

In both these evaluations, a rich contextual assessment helped the teams design their interventions in a way that is likely to be acceptable and feasible to the community and programme managers.

Development and procurement of technologies is a time and resource intensive process that requires some flexibility in timelines.

The evaluations in Pakistan and Nigeria encountered unanticipated problems with the development of their technology: a low-cost, durable, wearable ankle band that contains a hardcopy of the vaccination record of a child. During the pilot stage, the teams found design and manufacturing defects in the bands that necessitated rounds of time-consuming iterations to the original design. In addition, the teams had to re-procure and test the final version of the bands. All these factors led to significant delays in the project implementation timelines.

Optimal use of technologies by health workers requires their intensive engagement in its development along with providing them thorough training and on-field support.

In both the Haryana and Rajasthan studies, use of tablet-based data collection apps by health workers are being tested. The teams found that the tablets may not be easy to operate for some of the health workers with limited literacy and technical skills. In addition, there was a concern that this could overburden them due to duplication in maintenance of health records on paper as well as on the tablet. To mitigate these concerns, the teams involved the health workers extensively in the development of apps and incorporated their feedback to build advance add-on features that would make record-keeping easier. They also trained the health workers several times throughout the project cycle to use the tablets effectively.

Additionally, both study teams were cognizant of the fact that not all technical glitches can be anticipated during the development and piloting phases. To address this, they created troubleshooting facilities for the health workers. The Haryana team created a 24-hour centralised helpline for the health workers to report issues with the tablet or the software. In the case of Rajasthan, the team deployed field monitors who were well versed with the tablet and pendant technologies to assist the health workers address their technical problems.

Careful assessment of the existing healthcare system to identify appropriate delivery mechanisms for technology-driven interventions is critical.

The Pakistan study had initially planned to involve two government cadres of community health workers, namely lady health workers and midwives to deliver the ankle bands to newborn infants. However, during a social mapping exercise, they found that the midwives were not active in the study area and they also realised that the cadre of lady health workers on their own would be insufficient to deliver the intervention. During this exercise they also identified another cadre of community health workers trained by Unicef as well as traditional birth attendants operating independently in the study area. To support the delivery of their intervention, the team decided to involve all the identified cadres of health workers. This exercise was important to understand whether the government’s human resources would have been sufficient to deliver the intervention and what sort of additional resources were required for effective delivery.

Proactive and thoughtful planning is required to adapt new digital technologies to the existing paper-based administrative processes in immunisation programmes.

The interventions in Uganda and India are testing changes in the paper-based administrative data collection methods and tracking of individual immunisation status by using m-health applications. Currently, paper-based records are collected and maintained by the frontline health workers for administrative purposes at various levels. Tablet-based data collection tools need to be adapted accordingly. The study team in Haryana has done this by ensuring that the data recorded in their app is consolidated and sorted into daily and monthly reports that have the same format as the paper-based immunisation registers. This way, health workers can copy data from the app to the register. The team also made provisions in their app for generating monthly reports in the same format as the ones generated through the government reporting system for use by health officials at higher levels. In addition, the team has created new reporting formats for monitoring performance of health workers, made possible by real time data collected through tablets.

These lessons show that technology-driven health interventions require a lot of careful thought, planning, assessment and resources. As the implementation of these interventions and their evaluations are ongoing, we are still in the process of learning about which of these interventions will actually work, where, how and at what cost to improve the immunisation coverage of children. We will continue discussing challenges and lesson learned in forthcoming blogs.

This blog was inspired by our field visits to some of these study sites and discussions during a 3ie workshop last year on challenges related to implementing technology-based interventions. A detailed summary of the discussions on this and other topics related to ongoing evaluation and implementation of immunisation interventions that took place during this event can be found here.