Lessons learned from evaluations of the impact of WFP programmes on moderate acute malnutrition in the Sahel

View this article as a pdf

Lisez cet article en français ici

By Jonas Heirman, Mica Jenkins and Jennifer Rosenzweig

Jonas Heirman leads impact evaluation activities for the World Food Programme (WFP). Jonas has previously worked in research and evaluation capacities for the UK Department for International Development (DFID), the Institute of Development Studies (IDS) and European Centre for Development Policy Management (ECDPM).

Mica Jenkins is the Research and Evidence Officer for the Nutrition Division at WFP headquarters. She holds an MSc in Health and Human Development with a focus on nutrition and sustainable food systems. Prior to joining WFP, Mica implemented rural development and research projects in Mozambique and in the Dominican Republic.

Jennifer Rosenzweig is a senior development professional with 20 years’ experience working on nutrition and health issues for United Nations agencies and non-governmental organisations (NGOs). She has worked in the WFP Nutrition Division since 2013.

Context

Recent estimates show that more than one quarter of wasted children under five years old live in Africa (UNICEF, 2019). Among these 18.2 million children, 14 million suffer from moderate wasting. While there is evidence of the effectiveness of interventions to treat moderate acute malnutrition (MAM) (Lazzerini et al, 2013; Gera et al, 2017; Fabiansen et al, 2017), there remains insufficient understanding of the relationship between MAM treatment and prevention programmes. In response to this knowledge gap, World Food Programme (WFP) commissioned a four-country evaluation series in the Sahel to assess the relationship between treatment and prevention in emergency and post-emergency contexts (WFP, 2018).

The findings from this evaluation are relevant to the ongoing global effort to improve the continuum of care for acute malnutrition. In particular, the findings related to prevention programmes, through blanket supplementary feeding alone or in combination with food assistance for assets and/or targeted supplementary programmes, support a better understanding of specific mechanisms to reduce the prevalence and incidence of acute malnutrition.

Summary of the four evaluations

Although considerable evidence exists on the effectiveness of MAM interventions in controlled environments, more evidence is needed on the effectiveness of these interventions in the contexts in which WFP operates. To address these evidence needs, the MAM impact evaluation series was commissioned by WFP’s Office of Evaluation and managed by International Initiative for Impact Evaluation (3ie).

High rates of poverty and insecurity in the Sahel pose particularly significant challenges for nutrition interventions. Chad, Mali, Niger and Sudan were selected by WFP and ongoing programmes were paired with researchers. This approach created limitations for the feasibility of certain impact evaluation methods and no study could use a randomised controlled trial. Despite this, each study provided useful evidence for WFP’s nutrition programming. The following is a summary of each evaluation.

Targeted and blanket supplementary feeding programmes in Chad (Saboya et al, 2018)

The evaluation in Chad focused on the impact of WFP’s targeted supplementary feeding programme (TSFP) and blanket supplementary feeding programme (BSFP) in a region with high levels of MAM. BSFPs are a preventative intervention, whereas TSFPs are a MAM treatment intervention. The primary evaluation question was:

What is the impact of interventions for preventing moderate acute malnutrition on the incidence and prevalence of moderate acute malnutrition in children under two years with varying levels of access to moderate acute malnutrition treatment?

The evaluation used propensity-score matching to compare beneficiaries of the BSFP with non-recipients. Distance to nearest health centres was used to estimate the additional effect of access to the TSFP.

The evaluation found raw correlation between BSFP and MAM of 5 percentage point reduction (from 17% to 12%). Testing for significance, the evaluation found a 4.7 to 8 percentage point (comparing OLS, PROBIT, IPW and IPWRA models) reduction in propensity to become MAM in households benefitting from the BSFP, which is significant at the 5% level when accounting for false discoveries. It also found that the incidence of MAM is 3.6 percentage points lower (significant only in PROBIT model) when there is good access to the TSFP. Lack of reliable census data and maps meant that the identification of areas with good or poor access to TSFP had to be done directly in the field.

Food assistance for food-insecure populations during conflict in Mali (Gelli et al, 2018)

Mali faces significant development and security challenges. The impact evaluation examined the effects of general food distribution and school feeding on populations in conflict-affected areas. The primary evaluation questions were:

• What are the impacts of conflict and food assistance on child malnutrition and other development outcomes?
• What are the effects of food assistance on conflict-affected populations?
• What coping strategies were most effective in mitigating the effects of conflict?
• How did humanitarian aid influence the effectiveness of coping strategies at household and community level?

The evaluation relied on a longitudinal study of a population in the Mopti region. The population was subdivided according to: i) extent of humanitarian assistance received since conflict; and ii) extent to which areas were affected by conflict. Data was collected through two household surveys (2012 and 2017). WFP food-assistance activities in northern Mali during the period evaluated included BSFP, TSFP, general food distribution (GFD) and school feeding. The effect of food assistance on populations was analysed by combining propensity-score matching and difference-in-difference estimations.

The evaluation found no significant effects of food assistance on the prevalence of MAM in the population surveyed. However, all forms of assistance were found to increase the consumption of vitamin A and monthly food and non-food expenditures.

Treatment and prevention of moderate acute malnutrition in Niger (Brück et al, 2018)

In Niger WFP’s Protracted Relief and Recovery Operation (PRRO) employed a wide range of interventions, including targeted food assistance (TFA), blanket supplementary feeding (BSF), food assistance for assets (FFA), and targeted supplementary feeding (TSF). The impact evaluation assessed to what extent FFA in combination with other interventions is more effective at addressing MAM than prevention or treatment alone. The primary evaluation question was:

What are the impacts on nutrition outcomes of various combinations of programme components in WFP’s protracted relief and recovery operation?

The evaluation relied on quantitative and qualitative methods of analysis. Quantitative analyses relied on panel survey data collected in 2014 and 2016. To analyse impact, the evaluation used difference-in-difference analysis to compare three groups: i) people receiving FFA at baseline but no assistance by end-line; ii) people receiving FFA at baseline who stopped receiving FFA and instead received TFA, BSF and/or TSF at end-line; and iii) people receiving FFA at both baseline and end-line and receiving another intervention at end-line.

The evaluation found that children who received both FFA and treatment and/or prevention assistance at the end-line of the evaluation (group three) were 20% less likely to suffer MAM than children receiving no assistance at end-line (group one). Additionally, children in group three were 15.5% less likely to suffer from MAM than those in group two. These findings suggest the FFA is potentially a valuable addition to MAM interventions. However, data availability and the non-random assignment of individuals made the evaluators cautious not to overstate the validity of findings.

MAM treatment and prevention programmes in Sudan (Guevarra, 2017)

In Sudan WFP delivers both MAM treatment and prevention interventions. In 2015 WFP expanded the use of food-based prevention of MAM (FBMAM) in areas already receiving TSFP interventions. WFP has also explored the relative effectiveness of integrating food-based interventions with social and behaviour change communications (SBCC) and water, sanitation and hygiene (WASH) interventions. The primary evaluation question was:

What is the impact of different MAM treatment and prevention interventions on the incidence and prevalence of moderate and severe acute malnutrition in children under 5 years and pregnant and lactating women?

The evaluation relied on a stepped-wedge, cluster-controlled trial design that followed the expansion of FBMAM interventions in areas receiving TSFP. However, the assignment of clusters to individual expansion rounds was not random and expansion also faced implementation delays. In addition, the evaluation nested a two-arm, parallel cluster-controlled design to assess the prevalence of malnutrition in areas with treatment compared to those with both treatment and prevention.

The evaluation was unable to detect any significant difference in the prevalence of global acute malnutrition (GAM), MAM or severe acute malnutrition (SAM) in populations receiving prevention interventions. The importance of ensuring quality and delivery of services was highlighted as an area for future WFP attention. However, the evaluation did find up to a 12% reduction in the number of at-risk individuals after the fourth round of FBMAM interventions. The evaluators offer three interpretations for these findings: i) there is a time-lag between at-risk reduction and prevalence rates; ii) prevalence rates remained high in areas with prevention interventions because cases shifted from SAM back to MAM; or iii) prevention coverage was not high enough to reduce population-level prevalence rates. Also, no significant effects were found for SBCC interventions due to implementation challenges and low participation.

Lessons and recommendations from the MAM series (WFP, 2018)

The MAM series was particularly useful for WFP because it provided significant new evidence related to operational challenges and the impact of treatment and prevention interventions in the Sahel. A synthesis of the four evaluations identified the following lessons:

1. Greater attention to the timing, sequencing and roll-out schedule of a package of interventions is likely to result in enhanced effectiveness.
2. Closer partnerships and coordination can support more effective and efficient delivery.
3. Barriers to achieving better coverage and access include, but are not limited to, infrastructure deficits. Awareness of the availability of quality services is also important.
4. The quality, availability and lack of monitoring and evaluation (M&E) data are problematic.

These lessons informed the following recommendations for WFP:

1. WFP should invest in the strategic deepening of its relationships with partners and stakeholders, providing capacity-strengthening where relevant;
2. WFP interventions should pay greater attention to communication with target groups, effectiveness of case-finding, and community sensitisation, all of which may improve targeting, efficiencies and effectiveness of MAM treatment and prevention programmes;
3. MAM programme components (e.g., packages) should be better tailored to context and may need to be integrated into longer-term, multi-sector solutions;
4. Greater effort should be made to share data among agencies working towards Sustainable Development Goal 2 and mainstream compatibility of tools and systems;
5. Greater support and attention are needed to improve the collection and use of monitoring and cost data.

These recommendations point to the importance of considering MAM treatment and prevention interventions along the continuum of care. The layering of treatment, prevention and other interventions such as FFA were found to have significant benefits in terms of effectiveness, depending on the combination and context. In addition, all studies highlighted the importance of improving the collection of comparable cost data to enable more strategic decision-making. After the synthesis in 2018 many of these lessons and recommendations were immediately taken onboard by WFP.

Acting on recommendations

The MAM series provided WFP with evidence and recommendations to support improvements in the planning and delivery of malnutrition treatment and prevention interventions. WFP Nutrition accepted all five recommendations made by the synthesis report and immediately began incorporating them into partnership, policies, guidance and support. The following are examples of how MAM series recommendations influenced WFP’s nutrition programming.

Recommendation 1: Deepen strategic relationships and support capacity-strengthening of partners

At the corporate level, the WFP Nutrition Policy (2017-2021) made partnerships a central priority and recognised the need for nutrition actors to work jointly to identify and improve upon delivery platforms to ensure that all vulnerable groups have access to healthy diets. WFP’s partners in nutrition are diverse and include national governments, other UN agencies, non-government organisations, academic institutions and the private sector, all of which bring important strengths to WFP’s nutrition programmes.

In 2018 WFP piloted its external digital learning platform (nut^x), which is designed to share the organisation’s experiences and expertise in nutrition with external partners, including governments, UN agencies, national and international NGOs, academic institutions and private sector actors¹. This platform offers a broad range of e-learning courses and activities aligned with WFP’s nutrition priorities, with the aim of promoting capacity-strengthening of partners and fostering the achievement of SDG2. nut^x will be officially launched in quarter four of 2019.

Recommendation 2: Enhance communication with target groups and community sensitisation

The MAM series identified weaknesses in WFP’s communications, particularly in the implementation of SBCC. To address these and other weaknesses in SBCC across contexts, WFP is investing in SBCC in multiple ways. Since finalising and launching its comprehensive, field-oriented SBCC Guidance Manual for Nutrition programmes in 2018², WFP Nutrition has focused on country-level actions to improve the quality, reach and impact of its SBCC programmes, including training for over 350 staff and partners in five regions. As a result of the evaluation recommendation to review SBCC interventions, actions and delivery platforms in Sudan, WFP expanded its efforts to include a nutrition-oriented TV show, radio programme and the sending of bulk SMS messages to the target population. These innovative media-based approaches build on the traditional approaches of community mobilisation, nutrition or health days, care groups and targeted SBCC training to strengthen staff and partner capacity.

In Chad WFP is piloting a peer-to-peer model in remote areas through which ‘role mothers’ share good practices on hygiene and nutrition with a group of 12-15 women over 12 days. The model includes the use of ‘nutricards’ used by role mothers to teach other mothers how to prepare nutritious meals using local foods, as well as interactive games with their children to stimulate cognitive development. The approach builds community ownership and aims to improve the coverage and efficiency of the MAM treatment programme.

Recommendation 3: Tailor responses to context

WFP’s new country strategic plan (CSP) approach has encouraged every country office to ensure its programmes are tailored to country context and needs. In West Africa the formulation of the CSP process was used as an opportunity to embed nutrition-sensitive thinking into country portfolios. WFP’s Sahel resilience scale-up initiative embedded nutrition-sensitive thinking and actions into the selection of all activities. Findings from Niger on the benefits of layering FFA with other interventions also encouraged the integration of activities in the Sahel.

Nutrition analysis is being aided by WFP’s Fill the Nutrient Gap³ assessment, which reveals context-specific barriers to the consumption of healthy and nutritious foods and supports multi-sector decision-making on the prevention of malnutrition. The assessment focuses on gaps in nutrient intake to inform country-level policies and actions that can be taken to improve nutrition among the population, with a focus on the most vulnerable.

Recommendation 4: Improve inter-agency data collection methods and enhance data sharing

WFP is making significant efforts to improve the quality of nutrition data collected for monitoring and analysis. SCOPE CODA is WFP’s cloud-based innovation, with the potential to transform data management in malnutrition treatment programmes. The application gives a digital identity to patients and tracks healthcare services, replacing paper-and-pen records, ration and referral cards and reports in healthcare centres with a personalised smartcard linked to an electronic database. It reduces human error, streamlines data and improves the effectiveness of treatment of acute malnutrition. SCOPE CODA strengthens partner coordination by providing a unified platform and linking to existing systems for all stakeholders in community-based management of acute malnutrition (CMAM), including governments, UNICEF, international organisations and implementing partners, to access and share programme information. It is being developed as a single data system across treatment programmes for SAM (managed by UNICEF) and MAM (managed by WFP) and thus will result in better partner coordination, integrated data management and reduced loss at referral.

Recommendation 5: Improve use of monitoring and cost data

Although an effort was made to collect and analyse cost data in this evaluation series, the findings were inconclusive and point to a need for more rigorous cost studies. WFP is improving the quality and usefulness of nutrition intervention monitoring and cost data. To improve cost analysis, WFP recently launched a new costing tool that supports countries to collect comparable data on interventions. The tool uses both monitoring and financial data from within WFP to generate estimates for MAM treatment programming, using a globally standardised methodology. This ensures consistency in both data sourcing and calculations across countries to enable global comparison. Furthermore, the tool analyses additional information, such as cost drivers, to support programmers in the field as they work to improve cost-effectiveness over time.

The tool will be used to estimate the cost to WFP to treat a child for MAM. With partner collaboration it can also be used to determine the full cost to treat a child for MAM. The tool is internal as it is built around WFP data systems in order to minimise burden as well as encourage consistency between countries. However, the methodology and results of the tool will be available to partners.

Conclusion

The findings from the MAM impact evaluation series are supporting a process of learning and improvement in WFP. The synthesis of these evaluations provided practical recommendations that are currently guiding WFP’s efforts to improve acute malnutrition treatment and prevention interventions. These improvements also provide new opportunities to test theories and generate new evidence to answer questions raised by the MAM evaluation series.

For more information please contact  Jonas Heirman.

Endnotes

¹nutx is accessible at: https://cdn.wfp.org/nutrition/nutx/

²Social and Behaviour Change Communication: Guidance Manual for WFP Nutrition. Available at: https://docs.wfp.org/api/documents/WFP-0000102103/download/

³ttps://www1.wfp.org/publications/2017-fill-nutrient-gap?_ga=2.217522065.62125258.1558712434-190838471.1546502188

References

Brück T, Ferguson NTN, Ouédraogo J and Ziegelhöfer Z, 2018. Impacts of the World Food Programme’s interventions to treat malnutrition in Niger. 3ie Impact Evaluation Report 80. New Delhi: International Initiative for Impact Evaluation (3ie). Available at: doi: https://doi.org/10.23846/tw6IE80

Fabiansen C, Yaméogo CW, Iuel-Brockdorf A-S, Cichon B, Rytter MJH, Kurpad A et al, 2017. Effectiveness of food supplements in increasing fat-free tissue accretion in children with moderate acute malnutrition: A randomised 2 × 2 × 3 factorial trial in Burkina Faso. PLoS Med 14(9): e1002387. https://doi.org/10.1371/journal.pmed.1002387

Gelli A, Tranchant J-P, Bliznashka L, Diallo AS, Sacko M, Siegel E, Aurino E and Masset E, 2018. The impact of food assistance on food insecure populations during conflict in Mali. 3ie Grantee Final Report. New Delhi: International Initiative for Impact Evaluation (3ie).

Gera T, Pena-Rosas JP, Boy-Mena E, Sachdev HS (2017) Lipid based nutrient supplements (LNS) for treatment of children (6 months to 59 months) with moderate acute malnutrition (MAM): A systematic review. PLoS ONE 12(9): e0182096. https://doi.org/10.1371/journal.pone.0182096

Guevarra E. Mandalazi S. Belagamire K. Albrekstan K. Sadler K. Abdelsalam K, Urrea G and Alawad S, 2018. Impact evaluation of the World Food Programme’s moderate acute malnutrition treatment and prevention programmes in Sudan. 3ie Impact Evaluation Report 79. New Delhi: International Initiative for Impact Evaluation (3ie). Available at: doi.org/10.23846/tw6IE79

Lazzerini M, Rubert L, Pani P, 2013. Specially formulated foods for treating children with moderate acute malnutrition in low- and middle-income countries. Cochrane Database of Systematic Reviews 2013, Issue 6. Art. No.: CD009584. DOI: 10.1002/14651858.CD009584.pub2.

Saboya M et al, 2018. Impact evaluation of WFP’s programs targeting moderate acute malnutrition in humanitarian situations in Chad. Impact Evaluation Report 78. 3ie: https://assets.publishing.service.gov.uk/media/5b73fb1440f0b6139e05a458/ie78-mam-chad.pdf

UNICEF, WHO and World Bank (2019) Levels and trends in child malnutrition: key findings of the 2019 Edition of the Joint Child Malnutrition Estimates. Geneva: World Health Organization; 2019 Licence: CC BY-NC-SA 3.0 IGO.

WFP, 2018. Synthesis report on four evaluations of the impact of WFP programmes on nutrition in humanitarian contexts in the Sahel. https://docs.wfp.org/api/documents/80ec7f0c-bd3b-4689-ad2e-c5254e98066b/download/?_ga=2.171021408.1395824924.1552663230-595083097.152941453