Closing the immunisation gap in Ethiopia: a formative evaluation of ‘The Fifth Child Project’Other evaluation
3ie evidence programme: Innovations in Increasing Immunisation Evidence Programme
Author(s): Shiferaw Dechasa Demissie, Heidi Larson, Tracey Chantler, Naoko Kozuki, Emilie Karafillakis, Justine Landegger, Lilian Kiapi, Petros Gebrekirstos, Siraj Mohammed, Samuel Wodajo, Anne Langston
Institutional affiliation(s): International Rescue Committee, London School of Hygiene & Tropical Medicine, Assosa Referral Hospital
Grant-holding institution: International Rescue Committee
Main implementing agency: International Rescue Committee
Sex disaggregation: No
Gender analysis: No
Equity focus: No
In 2013, Ethiopia’s national coverage of the third dose of the pentavalent vaccine was 37 per cent. The dropout rate between the first and third doses of this vaccine was 43 per cent.
Complex immunisation schedules, and traditional social and cultural norms create barriers to immunisation uptake. Additionally, limited community involvement and a lack of quality data have led to poor identification of populations that are most in need. To address these barriers, the International Rescue Committee is implementing the ‘Fifth Child Project’, where community-based health workers are collaborating with community leaders to register, counsel and track all pregnant women and infants in order to increase immunisation uptake.
The project employed a community engagement strategy involving community-based Health Extension Workers (HEWs) placed at health posts, as well as Health Development Army members (HDAs) who are also community members. The HEWs and the HDA members conducted home visits using a specially developed colour-coded calendar called Enat Mastawesha. The calendar promoted five key health-seeking behaviours for improving uptake of immunisation, and other health services during the pregnancy and postnatal period.
The intervention also included a user-friendly, data collection and defaulter-tracing tool, which was a simple paper-based document template with three built-in carbon copies used at the health post level for recording immunisation-related information of infants. It was used by the HEWs, HDAs and community leaders to identify and follow-up with the caregivers of defaulters.
This study hypothesised that providing training and tools, along with support for delivering immunisation services, will motivate HEWs and HDAs to conduct effective home visits. The resulting improvements in service provision would encourage caregivers to bring infants for timely immunisations.
This hypothesis was based on three key assumptions:
- Since HDAs are a part of the community, they are already trusted by other community members, and with adequate support, would be able to conduct quality home visits.
- Providing individualised data on immunisation defaulters to HEWs, HDAs and community leaders, will help them co-manage tracking immunisation defaulters and mobilising for outreach.
- Sustained delivery of immunisation services by the government (with support from the International Rescue Committee) will be a major determinant of successful project implementation.
Evaluation design and methodology
This study was conducted in the Assosa and Bambasi woredas or districts in the Benishangul-Gumuz region. It used a mixed-method approach to understand the integration of immunisation services at the health facility and community level. Performance of the defaulter-tracing system was assessed through routine analysis of monitoring data.
Qualitative and quantitative methods, including key informant interviews, focus group discussions and structured surveys, were used to explore the acceptability of the intervention among health workers and community leaders.
Primary evaluation questions
This study aims to:
- Understand whether the ‘Fifth Child Project’ has been integrated within the local health system.
- Understand how the intervention components were utilised by the community-based health workers in their interactions with caregivers.
- Assess the performance of the ‘Fifth Child Project’ by reviewing defaulter statistics and monitoring trends in the uptake of routine immunisations, specifically pentavalent vaccine.
The overall findings indicate that the intervention was well integrated into the local health system and enabled more systematic follow-up of unimmunised children. The project contributed to improved awareness about, and increased demand for immunisation. The calendar tool served as a communication aide for health workers and a catalyst for health-related discussions between family members. It supported more personalised interactions between health workers and caregivers, and played a significant role in ensuring timely immunisation.
The HEWs also found the defaulter tracing tool to be very useful. The tool had improved their access to immunisation data, and enabled them to count and identify defaulters more effectively.
As a result of the intervention, coverage of Pentavalent 3 increased from 73 to 81 per cent in Assosa woreda, and from 78 to 93 per cent in Bambasi woreda. Similarly, measles vaccine coverage increased from 77 to 81 per cent in Assosa woreda, and from 59 to 86 per cent in Bambasi woreda.
During the course of the intervention, nearly 54 per cent of all eligible infants defaulted at least once during the implementation period. Of these, approximately 84 per cent of defaulting infants were identified and subsequently immunised.
The implementation of the intervention did not differ in significant ways from the intended plan. A majority of HEWs and HDAs used the tools effectively, and could motivate mothers and caregivers to seek services such as antenatal care, immunisation and postpartum family planning.
However, certain assumptions, included in the original theory of change, were challenged. Evaluation findings and stakeholder feedback prompted implementers to slightly revise the calendar and defaulter-tracing tools, specifically in terms of what the images on the tools depicted (i.e. male involvement, infant feeding best practices), and improve further on their acceptability for a low literacy audience. The International Rescue Committee also suggests additional capacity-building of health staff to improve technical expertise in providing the continuum of care from pregnancy to institutional delivery, postnatal care, immunisation and family planning. This can be achieved through formal trainings, and on-job mentoring of technical and support staff across health facilities in targeted woredas.
Given the findings of feasibility, acceptability and potential benefits in improving both community engagement and quality of care, authors recommend a full impact evaluation of an integrated approach to support immunisation strengthening. The authors suggest that the key outcomes of interest for an impact evaluation should include immunisation coverage, coverage of other services such as antenatal care, skilled delivery, postpartum family planning, community members’ (including men) involvement, HEW workload, technical support requirements of health staff, scalability and cost-effectiveness.
The Fifth Child Project, Vaccine Confidence Project
Closing the global immunization gap, one child at a time, RESCUE AID, April 2017