Collaborative community checklists for immunisation: a feasibility and acceptability study in rural Myanmar

3ie evidence programme:  Innovations in Increasing Immunisation Evidence Porgramme
Author(s): Chris Morgan, Jessica Davis, Hnin Kalayar Kyaw, Aung Ko Ko, Htar Htar Lin, Aye Mya Chan
Institutional affiliation(s): Burnet Institute Myanmar, Department of Public Health, Ministry of Health and Sports, Myanmar
Grant-holding institution: Macfarlane Burnet Institute for Medical Research and Public Health
Main implementing agency: Burnet Institute Myanmar
Sex disaggregation: No
Gender analysis: No
Equity focus: No
Dataset: Available

Context

The World Health Organization has developed service quality checklists for use by health providers in surgery, safe motherhood and most recently immunisation. Operational research suggests that community scorecards can increase health service transparency, accountability, and help build collaborative relationships between health providers and communities. In Myanmar, health authorities are committed to increasing vaccination rates in all settings, especially hard-to-reach communities. They therefore agreed to test a new approach that combines both checklists and community feedback mechanisms, as a complement to other continuing work to improve immunisation service quality and access.

Intervention design 

The study assessed the feasibility and acceptability of a ‘Community Collaborative Checklist’ intervention which comprised of three key components:

1. The study team worked with immunisation providers (midwives) and health officials to adapt an immunisation session checklist of the World Health Organization to the local context. Providers were trained and supported to use this provider checklist during immunisation sessions.
2. The study team supported community members to identify barriers to immunisation uptake and design a checklist to assess quality of immunisation service. Caregivers were encouraged and supported to use this community checklist during immunisation sessions.
3. Community members were assigned to be volunteer checklist assistants and were trained to provide support to caregivers in using the community checklist at immunisation sessions. These assistants also collated feedback from completed checklists, and collaborated with community members and midwives to discuss any issues raised through the checklist.
4. In addition to these three key intervention components, the study team also conducted monthly health education sessions in each study village to build caregiver understanding of quality immunisation services and educate them on use of the community checklist.

The underlying theory of change for this intervention is that by using locally adapted immunisation session checklists, caregivers and midwives will work holistically towards improving the overall immunisation outcomes for children. In particular, the intervention would lead to an increase in caregivers’ knowledge on immunisation, increase the demand for quality services, and also improve communication between caregivers and midwives. Additionally, the use of provider checklists combined with feedback from the caregiver checklist will help midwives improve their service delivery. The improved quality of services and increased demand would lead to more children getting fully immunised, which in turn would reduce the prevalence of vaccine-preventable diseases.

Evaluation design and methodology 

This was a mixed-methods study conducted in three villages in rural Myanmar.

Qualitative methods were used to explore knowledge of and attitudes towards immunisation, including the elements of good quality services, barriers to and enablers of immunisation and acceptability of checklist use.

Quantitative data collection methods were used to assess supply-side determinants of quality and access to services, immunisation rates, usage of community checklists by caregivers and immunisation provider experiences of checklist use.

Primary evaluation questions 

The study addressed the following questions:

1. Was the intervention feasible and acceptable to immunisation providers and communities?
2. What was the perceived effect of the intervention on knowledge and attitudes amongst caregivers and providers?
3. What was the perceived effect of the intervention on immunisation service quality and uptake amongst caregivers and providers?
4. Did the intervention help providers and communities to identify barriers to immunisation and engage community members in addressing these barriers?

Primary findings  

The overall findings indicate that the intervention is feasible in rural and low-resource settings. In terms of participation, over 160 community members participated in the design of the community checklist and 84 per cent of surveyed caregivers used the checklist to assess immunisation services at least once. Caregivers reported that the community checklist was quick, easy to use and was not overly burdensome.

The development of the provider checklist also attracted substantial participation by midwives and health authorities. All midwives in the study area had used the provider checklist in planning and delivering immunisation services. They reported that using the checklist was relatively easy, and that there was increased recall regarding items to be prepared and actions to take before, during and after an immunisation session.

Caregivers, midwives and community members reported a number of benefits associated with the intervention, including perceived improvement in community’s knowledge regarding immunisation, quality of immunisation services and more active care-seeking for childhood vaccination.

Limitations of this study related to the localised scope and use of predominantly qualitative methods, which increased the risk of social acceptability bias influencing the responses of beneficiaries.

Implications

Overall, the intervention has proven to be feasible in rural Myanmar in its use of paper-based, scalable tools that are consistent with new national resource materials and other low-cost inputs that all work within existing structures. This includes checklist assistants (a role assigned to current health volunteers), village health committee’s review of the process and monthly health education sessions. It has also proven acceptable, with strong evidence that the checklist mechanism of reminder, review and accountability is a driver of change. It has proven acceptable to providers and caregivers, through provision of feedback that is anonymous, non-confrontational and indirect.

Given the findings of feasibility, acceptability and potential benefits in improving both community engagement and quality of care, authors recommend a full impact evaluation of a modified approach to support immunisation strengthening. The authors suggest that the key outcomes of interest for an impact evaluation should include coverage, completeness and timeliness of vaccination, changes in caregiver knowledge of immunisation and related services, satisfaction with service provision, and support for and commitment to immunisation.