Evaluation of a gamified mLearning solution for training micro insurance agents in Kenya

3ie evidence programme:  Agricultural Insurance Evidence Programme
Author(s): Andrew Mude, Nathan Jense, Duncan D. Khalai, Rupsha Banerjee, Diba D Galgallo
Institutional affiliation(s): International Livestock Research Institute
Grant-holding institution: International Livestock Research Institute
Main implementing agency: International Livestock Research Institute
Sex disaggregation: No
Gender analysis: No
Equity focus: Yes
Study type: Formative evaluation

Context

Extension and education is an integral part of adoption of different technologies. For index based livestock insurance (IBLI) it is no different, agents are hired by the insurance companies, trained, and tasked with the responsibility of carrying out extension, marketing activities and selling the product to prospective clients. Both of IBLI’s insurance partners in Kenya (Takaful Insurance of Africa an Apollo Pan Africa Insurance) have cumulatively sold over 14,000 contracts in the past 7 years in 6 ASAL counties of Kenya. Since 2015, IBLI has also been adopted by the Government of Kenya as the Kenya Livestock Insurance Program (KLIP), reaching over 15,000 households in 7 counties in the ASAL counties of Kenya. In both the voluntary (IBLI) and transfer (KLIP) cases, extension – creating awareness of the product so that (1) there is informed demand (2) the clients/beneficiaries have accurate information about the product – has been a key weakness of the marketing and sales strategies used. The current approach of relying almost entirely on intensive face-to-face meetings to train agents, who are then to provide extension and marketing services, is both expensive and unsustainable. Furthermore, it is not clear that it is successfully generate informed demand.

Intervention design

This study proposes two types of interventions: agent-targeted and client-targeted. Agent-targeted interventions consist of mLearning course training, an SMS refresher course and an IVR job-aid content, along with an initial face-to-face training. Client-targeted interventions consist of SMS courses, IVR-based FAQ platforms and radio campaigns.

Evaluation design and methodology

The initial evaluation design was a randomised control trial between mLearning, mLearning with gamification and traditional face-to-face training. However, the mobile application took longer to develop than expected. As a result, the application was repiloted with a small subset of agents from Isiolo County and a qualitative data collection was conducted. The objective of the SMS, IVR and phone surveys was to test for feasibility and understand decisions behind participating.

Primary evaluation questions

1. Can an mLearning solution be made to work in a remote and bandwidth challenged environment; does the level of user support required preclude scaling up in remote regions?
2. Is the mLearning course as effective as face-to-face training in developing an understanding of index based microinsurance and the IBLI Asset Protection Contract?
3. Is gamification effective in motivating sales agents to work through and do well in the course?
4. To what extent is sales agent knowledge correlated with contract sales and renewals?
5. Can automated, mobile-based Interactive Voice Response (IVR) systems be used effectively to gather data in ASAL areas?

Primary findings

1. In neither the first nor the second pilot did gamification seem to provide motivation for participation in the training course.
2. Of the insurance agents that were contacted, 66 per cent opted in for the course.
3. Of those that were invited to use the IVR platform, 80 per cent interacted with it.
4. Phone survey of agents and clients proved to be a very cost effective method for learning about product understanding among respondents.

Implications

1. Besides training of agents, direct interactions with the clients is also key to improve insurance take-up.
2. The larger agenda of client education also closely linked to the objective of improved household welfare.