Evaluating the impact of interventions to improve full immunisation rates in Haryana, India: Impact Evaluation Report 126 (September 2020)

Evaluation and Lessons Learned
Originally published
View original


Executive summary

Immunisation is a highly cost-effective way of improving child survival. However, full immunisation rates are low in India. In the state of Haryana, only 52.1 per cent of children aged between 12 to 24 months are fully immunised (IIPS n.d.), and rates decline rapidly after the initial vaccines in the immunisation schedule (e.g. Bacillus Calmette– Guérin, first dose of pentavalent) to the final vaccines (e.g. third dose of pentavalent and measles).

This programme built a common information structure leveraging m-Health software. This was used to evaluate the impact of several interventions aiming to increase demand for immunisation, namely: incentives, targeted reminders and the leverage of social networks in 140 primary health centres across seven districts of Haryana.

Working closely with government officials, the programme developed a new m-Health app for tablets and integrated this into the existing government programme for immunisation. Front-line health workers then used the app and recorded details of every child who was immunised at any immunisation camp in the sample primary health centres. There were three major interventions, cross-randomised at different levels:

  1. Small incentives in the form of mobile credit were provided to caregivers each time they brought their child to a vaccination camp, in half of the primary health centres. The incentives were delivered to the mobile phone number provided at the time of immunisation. Incentives varied in their levels, and in whether they increased over the course of the immunisation schedule, depending on the subcentre;

  2. Targeted text and voice call reminders were sent to caregivers to remind them it was time for their child to receive a specific shot. These were provided for 0 per cent, 33 per cent or 66 per cent of eligible children, depending on the sub-centre; and

  3. Key people or ‘seeds’ in the community social network were identified to spread information about immunisation. A rapid survey was conducted to obtain from villages the names of people generally good at transmitting information (‘gossip’), people who are generally trusted (‘trusted’), or people generally good at transmitting information and trusted (‘trusted gossip’). During the programme, the study sent monthly text messages and voice calls to one of these people, or to a randomly selected volunteer (randomisation was at the village level), to inform their fellow villagers of the importance of immunisation.

Monitoring data show that implementation of the programme was largely successful, especially considering the scale of operations (across 140 PHCs, 755 sub-centres and 2,360 villages, and with over 295,000 unique children registered in the database). Data entered into the tablets by front-line health workers were generally of high quality – there were almost no incidences of fake child records, and the child’s name, date of birth and vaccines entered were accurate over 80 per cent of the time.

Delivery rates of recharges were also high – they remained above 90 per cent throughout the programme. Delivery rates of text message reminders and seed messages were around 80 per cent for the duration of the programme, which is the industry standard.

The results of the programme show that incentives in the form of mobile credit had an overall positive impact on full immunisation rates when administered on a sloped schedule, where the amount of mobile phone credit was higher for the final two vaccines a child should receive in their first year. We see positive and significant effects in the social network experiment, when gossips are chosen as volunteers to relay information.
The effects of the reminder messages were relatively disappointing, with small effects found only in the sub-centres where 33 per cent of people received reminders.

We provide a range of cost-effectiveness calculations. The cost-effectiveness analysis shows that leveraging gossip seeds to spread information was a highly cost-effective intervention, with a ratio of USD5 per additional fully immunised child. For incentives, the low slope treatment arm was more cost-effective than the high slope treatment arm (USD61 per additional fully immunised child compared to USD93 per additional fully immunised child for the high slope). Considering that incentives are a transfer, and valuing them as a marginal cost of public funds (using the rate of 20%), the cost per additional fully immunised child was USD32 when low slope incentives were provided.

On balance, this programme demonstrated that demand-side interventions to promote immunisation are feasible at scale, and can be useful complements to developing infrastructure, particularly m-Health infrastructure, to improve access to immunisation.

In terms of pure fiscal costs, the most cost-effective intervention by far was the social communication intervention, which relied on community knowledge to identify good potential relays for immunisation messages. Our study demonstrated that this can be done easily and effectively through a rapid survey of a few households in the population.
Small incentives (with a sloped schedule) carry a larger extra cost, but when the cost of incentives is counted for as a transfer, they are also cost-effective.

The effect of targeted reminders – perhaps the most obvious intervention for a government intending to roll out an m-Health tablet programme – is limited by the fact that only children who have attended at least one session can be enrolled in the programme, so it could only affect the intensive margin. Here we find small and inconsistent effects.

Immediate next steps for exploring post-project policy influence include meeting with senior officials of the Haryana government, and to offer them support in considering how to integrate either a low-slope incentives scheme or a gossip scheme into its health programming and budget for the state.