Critical analysis of MICAH programme

Summary of published research¹

The efficacy of some single nutrition interventions has been frequently and thoroughly evaluated, e.g. iron supplementation, vitamin A supplementation and salt iodisation. However the effectiveness (efficacy in real world settings) of large-scale integrated health and nutrition programmes has not been thoroughly evaluated. A recent article critically analyses an NGO-led large scale, multi-country 10 year micronutrient and health (MICAH) programme with an adequacy evaluation (a documentation of time trends in the expected direction).

The MICAH programme was launched in 1995 by World Vision Canada and was implemented from 1996-2005 in five African countries, four of which are reported in the paper (Ethiopia, Ghana, Malawi and Tanzania). The programme reached 4 million direct beneficiaries and more than 6 million indirect beneficiaries. Context specific programme plans were developed for each MICAH country within a programme-wide framework of objectives and strategies based on baseline assessments of vitamin A, iron and iodine deficiencies. Multiple interventions, ranging from community-based supplement distribution to fortifying and diversifying foods to national-level advocacy for national policy changes, were conducted to address the deficiencies and target groups identified. Interventions were integrated into existing systems, structures and services, wherever possible, to increase potential for sustainability. Table 1 summarises the MICAH activities.

Source of table: Berti et al (2010). See footnote 1.

Programme areas within countries were poor and rural, where few or no other major development organisations were operational. A comparison of MICAH data with Demographic Health Surveys (DHS) data for national rural samples suggests that the selected areas were similar to or worse off than the rural average in each country at baseline. If the DHS data are extrapolated back to the baseline year, assuming a linear trend, then 1997 MICAH indicators in Ethiopia were worse than DHS for Vitamin A capsule coverage in children < 5 years of age, and better for measles coverage and latrine access.

Methods

Cross-sectional surveys were conducted in each programme area at baseline (1996/7), the end of phase one (2000) and the end of phase 11 (2004). The surveys were conducted in the same month of the year in each country. Two-stage cluster sampling was employed, in which clusters were randomly selected using probability proportional to population size. Key indicators were assessed through structured interviews with a standardised questionnaire and collection of biochemical, clinical and anthropometric data. The sample sizes per country were between 900 and 4801 randomly selected households per survey. Data were collected by trained enumerators and clinical staff. The data were subjected to post-hoc methods of quality determination and if of suitable quality, included in the adequacy evaluation. The magnitude of the change for each key indicator from baseline to follow-up final survey was compared with that observed in published controlled trials and reports of other large-scale programmes. If improvement was of comparable magnitude to the high end observed in controlled trials, the impact was considered high. If the improvements were of a range common in other programmes, the impact was considered moderate. If the change was smaller than other programmes, but greater than zero, the impact was considered low. Testing of differences from baseline to follow up was done by chi-square for categorical variables and t-tests for continuous variables.

Results

Most collected data were of moderate or high quality and therefore included in the adequacy evaluation.There were moderate to large improvements in vitamin A status in Ethiopian school-age children, in children less than 5 years of age in Tanzania and Ghana and amongst mothers in Ghana. Iodine status improved in Malawi and Tanzania. Anaemia rates and malaria prevalence decreased in women, pregnant women and pre-school children in Ghana, Malawi, and Tanzania, but anaemia increased in Ethiopian women. Large increases were reported for rates of exclusive breastfeeding (EBF) and immunisation. Child growth improved to the maximum that would be predicted with the given interventions.

Discussion

Most of the collected data were considered of good quality. The exceptions were anthropometric data in Ethiopia, anthropometric data for children under six months in Ghana and Malawi, breastmilk retinol analysis and urinary iodine in Tanzania and EBF data in all countries. An evaluation of the change in indicators over time (adequacy evaluation) revealed many positive results of the MICAH programme (see Table 2). MICAH programme staff reported five aspects of the programme that they believe uniquely contributed to the positive results:

Results-based management identified programme activities that were not producing positive outcomes and allowed for mid-stream corrections. Examples include:

• The decision in Malawi (following the 2000 evaluation) to reduce the geographic spread and intensify the intervention in a smaller area to ensure all participants received the interventions.
• Provision of regular, intensive technical support in the form of regular email correspondence, monitoring and support visits by World Vision technical staff and expert consultants, and annual training workshops.
• Implementation of a broad-based integrated package of interventions rather than relying on a single 'magic bullet' intervention.
• Community participation in programme design, implementation, monitoring and evaluation, which meant some interventions could be tailored to suit community preferences. This was especially relevant to animal husbandry, where existing practices differed from community to community.
• Regular supervision of staff in communities.

There were fewer positive results in Ethiopia, which may be because the intervention efforts were diluted over a larger number of beneficiaries compared to other countries. However, there was a positive impact on vitamin A status in school-age children - both an uncommon target and success.

In an attempt to estimate the impact of MICAH independent of other local and global influences, comparisons were made with DHS data for the national, rural samples. Table 2 shows the change in those indicators for which there were both MICAH and DHS data at baseline or follow-up and end-line. The change is measured as the difference of the differences (the difference between the MICAH difference between end-line and baseline and the DHS difference between end-line and baseline). For most indicators in all four countries, MICAH areas outperformed rural areas of the country as a whole. The comparison groups are not perfectly suited as control groups - different years, baseline conditions and other differences not related to MICAH. However, the MICAH performance compared with the DHS (along with the general concordance between coverage and outcome indicators in MICAH samples) suggests the improvements amongst programme beneficiaries are greater than the general trends, and greater than would have occurred MICAH was not implemented.

Although the evaluation has documented the trends over time, it still falls short of a full adequacy evaluation - the causal pathway should be relatively short and simple, the expected impact must be large and confounding must be unlikely.

Conclusions

Numerous nutrition and health impacts were observed in the intervention areas, often of a magnitude equal to or larger than observed in controlled interventions or trials. These results show the value of integrated long-term interventions.

¹An adequacy evaluation of a 10-year,four-country nutrition and health programme. Peter R Berti, Alison Mildon, Kendra Siekmans, Barbara Main and Carolyn MacDonald. Int. J. Epidemiol. Advance Access published March 3, 2010