Management of acute malnutrition programme review and evaluation

Summary of evaluation¹

Young girl recovering from severe malnutrition, OTP centre in Kaedi, Mauritania

By Yvonne Grellety, Hélène Schwartz and David Rizzi

Yvonne Grellety is an independent advisor on international health and nutrition to humanitarian agencies working in the developing world. She has extensive experience in the emergency nutrition sector over 30 years and more, working with MSF and ICRC, ACF and UNICEF, particularly in challenging contexts.

Helene Schwartz is currently working with UNICEF WCARO as the IMAM focal point. She has previously worked in nutrition programmes including CMAM with a number of agencies in many countries, including ACF in Sudan, Burundi and Nepal, UNHCR in Chad and UNICEF in West and Central Africa.

David Rizzi is currently working for UNICEF North Korea on the CMAM programme and providing technical assistance to the Ministry of Public Health. He has worked and consulted for NGOs and UNICEF in many countries including Angola, Burundi, DRC, Mali, Mauritania and Chad.

In 2009, twenty countries in West and Central Africa were implementing programmes to address acute malnutrition. These programmes include the protocol, training modules, monitoring and evaluation and support for implementation at in- and outpatient facilities and at community level. The aim in most of the countries was primarily to increase the coverage of the programme.

UNICEF WCARO (West and Central Africa Regional Office) considered it important to have an independent evaluation of the progress and quality of implementation and coverage of the programmes and to identify their strengths and weaknesses, in order to provide sound technical advice to country offices. The planned evaluation comprised the review of the existing programmes, involving field visits of 10 to 15 days each in nine countries that were willing to participate: Benin, Burkina Faso, Côte d’Ivoire, Democratic Republic of Congo (DRC), Liberia, Mali, Mauritania, Sierra Leone and Togo.

The key tasks were:

• to analyse the national programme’s documents (protocols, training modules and monitoring evaluation tools) and provide recommendations to ensure that they were inline with the international standards, current best practice and any new scientific evidence
• to assess the implementation at field level and provide feedback on front-line activity and problems
• to review critically the effects of, and problems with, scaling up the programme and identify the main bottlenecks, and
• to make recommendations at national and regional level in order to ensure the quality control of the programme within the region.

After a ten days preparation mission in Dakar, a first evaluation was made in Mauritania and Benin, in order to test and standardise the methods. The personnel then divided into two teams. A total of 35 interviews and/or observations were conducted with programme managers of the Ministries of Health and international/ national non-governmental organisations (NGOs), 34 evaluations of in-patient facilities (IPF), 50 of outpatient therapeutic programmes centres (OTP) and 10 of supplementary feeding centres (SFC).

For each country, the team provided a detailed report describing the protocols, tools and the strategy used, with recommendations. Available individual data and monthly reports of databases were collected, analysed and their results integrated in the annexes of the final country report.

Main results

The national protocols were updated between 2005 and 2009. All included outpatient care, but still very few countries had adopted the 2006 WHO Growth Standards. Two countries used the opportunity of the evaluation visit to update their protocol (Benin and Côte d’Ivoire). However, the protocols of Burkina Faso, Mali, Sierra Leone, Mauritania and DRC required revision. The protocol of Togo was found to be accurate, clear and well-formatted for practical use. The protocol of Liberia was also found to be consistent with the current generic protocol.

A number of technical issues were evident in many countries, including a complete lack of standardisation. These included the admission and discharge criteria, the correct preparation of the therapeutic milks - especially problems with the scoops, the size of the packaging and preparation of small quantities - the treatment of malaria and some complications, the excessive use of some drugs such as metronidazole, anti-vomiting drugs, paracetamol, zinc tablets, the use of chloramphenicol as the first line antibiotic and management of infection where there was antibiotic resistance. Other issues needed stronger emphasis such as the correct application of the appetite test, the use of a severe acute malnutrition (SAM) number, and clearer understanding/definition of the current terms used.

Many general issues remained unsolved, such as the integration of management of acute malnutrition with other national protocols such as community Integrated Management of Child Illness (IMCI), HIV and TB programmes. The frailty of health systems with consequent danger of overburdening the system and degrading existing services, lack of human resources, the rapid turnover of staff and the absence of pre-service training made the process of implementation very weak and fragile. There was almost total absence of regular and adequate supervision and evaluation visits. Even if the community-based approach for outpatient care of acute malnutrition facilitated rapid scaling up, this resulted in many of those implementing the programme being isolated and often overburdened. The importance of supervision, coordination and support at the district level was repeatedly emphasised.

In terms of monitoring and evaluation, the lack of standardised definitions of terms, of printed tools and standard formatted reporting forms, complicated the analysis and the comparison between countries.

No national Ministries of Health visited had managed to have a sustainable system at community level. Community mobilisation was rare and depended entirely on the initiative and motivation of either a district officer or the supervisor of the health centre - this weakened the programme enormously.

Orders for supplies and deliveries were, for all the countries, the biggest challenge. There is a factory in DRC, Lubumbashi, but otherwise all products need to be imported. Some OTPs ran out of Ready To Use Therapeutic Food (RUTF) and had to close down due to the shortage of supplies. This was particularly a problem in emergency situations, e.g. in South DRC, programmes were opened without proper district supervision with emergency funds and then failed due to lack of supplies. This failure of logistics had a detrimental effect on the confidence of the population, and the reputation, sustainability and viability of all the programmes. The most affected were the Supplementary Feeding Programmes (SFPs) which were plagued by repeated and/or extended stock shortages in almost all the programmes and countries evaluated.

A summary of the strengths and weaknesses are given in Table 1.

Lessons learned

The lack of pre-service training weakened the scale up and exacerbated the effect of the high rate of staff turnover. Supervision and in-service training then has to be continuous and considered as key activities to alleviate this problem, provided that trained supervisors and evaluators can be identified in the absence of pre-service training. Supervision of the individual facilities and coordination within the district health management team is an absolute key to ensuring acceptable programme quality.

The lack of a nutrition budget line within the Ministries of Health’s strategy weakened the sustainability of the programme and makes the programme completely dependent on UNICEF, NGOs and donors.

These programmes cannot be dependent on emergency or short-term financing. The programmes must be implemented in terms of medium and/or long-term development. Governments and international partners, including donors, the UN staff and NGOs, face many challenges with management of acute malnutrition programmes including regular, timely delivery of supplies, regular collection and reporting of programme data and integration into an understaffed general health system faced with issues of training, recruitment, payment and support of health workers and high staff turnover.

Some countries, such as Benin, and some NGOs adopted a weekly outreach strategy to the non-catchment areas to screen and treat SAM cases. This strategy had a great effect on the defaulter rate.

Questions

Is it correct to scale up rapidly at the expense of the quality of the programme? Presumably there is a balance to be struck and a minimum standard of care and functionality of the programme defined. The very high defaulter rates in most programmes indicate that the beneficiaries are not satisfied with the quality and operation of the programmes. On the other hand, it is important to scale up as rapidly as possible to improve access to services. This will require investment in training, supervision, organisation and logistics.

Key questions remain such as can the quality and scale of the programmes improve if the health system as a whole remains weak? How should the programme be expanded and how can the danger of overburdening front line staff to the detriment of all other services provided from health centres be avoided or ameliorated?

Recommendations

The generic protocol should be reviewed to emphasize more clearly the different types of management activities, so that relevant information is readily available and all terms used clearly defined and standardised. This will allow a greater coherence at the regional level. The management of complications should be revisited with the increasing resistance of pathogens to antibiotics within the region.

Several technical details need to be addressed such as packaging of therapeutic products and the excessive use of certain medicines (administration of multiple drugs likely to be more toxic in the malnourished, is common). Training on the management of complications should be conducted on a regular basis due to excessive turnover of the staff until pre-service training has been in place for the majority of graduates to be familiar with SAM, moderate acute malnutrition (MAM) and their management.

Better integration of the management of acute malnutrition is needed in health centres and hospitals. Proper nutritional screening methods and tools should be incorporated into the integrated management of child illness (IMCI) protocols and implemented.

For programmes implemented at district level, the training and motivation of the nutrition focal point and district team is the key to a successful programme. At this level, logistics training, in-service training and ensuring evaluation and supervision skills should be included in programme budgets and plans to ensure outreach, monitoring and quality control.

A strong coordination between health and nutrition is necessary to achieve a harmonious integration of programmes (management of SAM, community-based management of malaria, diarrhoea and pneumonia, IMCI, promotion of essential family practices) at all levels. SFP programmes with large numbers of children should not rely on existing health staff and facilities.

The continuity of a programme requires steady, long-term funding to be effective. Many do not understand that a SAM programme will prevent death but not reduce the incidence of SAM. Likewise, a MAM programme should reduce the incidence of SAM, but not the incidence of MAM. Also donors need to be aware of the challenge to sustain programme activities after the initial emergency or conditions of vulnerability fade from attention.

The increase of scale of most integrated management of acute malnutrition (IMAM) national programmes should be slowed in order to recognize and resolve the current issues in implementation.

Conclusions

Without a balance between basic training, supervision, in-service training, logistical coordination and agreements on the time-frame of the programme, the increase of scale of programming will likely be compromised and be burdensome for the existing health service. If the budgets are uncertain and programmes cease or the capacity is over-stretched and provides a poor service, the programmes may fall into disrepute. If there are large scale programme failures, donors and governments are likely to withdraw support. The programmes evaluated have many very positive aspects and in well resourced and focused hands, give impressive results. Treatment of SAM averts a large number of child deaths. Nevertheless, the integrated management of SAM programmes must be strengthened to capitalize upon these results. The positive aspects should encourage medical schools and nursing schools, the Ministries of Health and Education to prioritise and ensure that these programmes are well conceived and sustainable. It is their responsibility to ensure the success of these programmes.

For more information, contact: Helene Schwartz, email: hschwartz@unicef.org

¹Management of Acute Malnutrition Programme Review and Evaluation. Fieldwork from 18th January - 30th April, 2010. Report 2010. Yvonne Grellety, Hélène Schwartz and David Rizzi. The Field Exchange summary was prepared by Hélène Schwartz and reviewed by Yvonne Grellety.