Multi-sectoral Nutrition Programming – Exploring Impact

Multi-sector nutrition programmes (MSNPs) have gained increasing prominence over the last two decades in order to address the many direct and underlying determinants of malnutrition. The Scaling Up Nutrition (SUN) Movement has played a critical role in mobilising governments and partners to drive forward such initiatives. However, given the challenges of setting up multi-sector monitoring and evaluation (M&E) systems, and due to the design of many multi-sector programmes, the ability to determine the impact of such programmes in achieving their ultimate objective of stunting reduction is often limited. Growing evidence of national and sub-national success stories, including in the most recent Maternal and Child Undernutrition Progress series, reinforces the crucial importance of MSNPs to address the underlying determinants of undernutrition. However there remains little exploration of the efficacy, effectiveness and impact of such approaches compared to single sector interventions and limited information on appropriate designs of M&E systems for MSNPs. This report, commissioned by Irish Aid, synthesises the available evidence on the impact of MSNPs and documents the type and quality of M&E systems established to measure impact. It aims to answer the question: Do MSNPs collect data that allows for the generation of evidence on impact and, if so, what is the level of reported impact?

Specifically, the following questions are explored:

1. Were evaluation designs sufficiently rigorous to allow for impact to be evaluated?

2. As precursors to impact, what is the level of coverage1 and convergence2 of interventions within MSNPs? Are these indicators assessed and, if so, how are they assessed?

3. What impact, if any, do MSNP evaluations demonstrate on primary nutrition outcomes (stunting, wasting, underweight and anaemia?)

4. Are MSNP evaluations able to demonstrate an impact on secondary nutrition outcomes such as infant and young child feeding (IYCF) indicators, Water, sanitation and hygiene (WASH), food security, household dietary diversity score (HDDS), minimum acceptable diet (MAD) and income/ expenditure?

5. Do MSNP evaluations consider the scale-up of interventions?  

A systematic search was conducted to identify relevant evaluation reports, programme implementation reports, programme description documents, programme proposals, research reports, peer-reviewed publications and systematic reviews/ meta-analyses. Forty-five evaluation reports were identified that met the inclusion criteria (30 of which have impact data for analysis). To enrich the answers to the questions posed, these were accompanied by 11 country assessments from the 2018 Joint assessment by the multi-stakeholder SUN platform (JAA) (from Bangladesh, Burundi, Cambodia, Ethiopia, Guatemala, Kenya, Malawi, Nepal, Niger, Philippines and Senegal), eight country case studies developed by the Emergency Nutrition Network (ENN) (from Ethiopia, Senegal, Kenya, Niger, Zimbabwe, Nepal, Bangladesh and Philippines) and a broader synthesis by Maximising the Quality of Scaling Up Nutrition Plus (MQSUN+) which included an additional six SUN country case studies (from El Salvador, Madagascar, Benin, Lao PDR, Pakistan and Guatemala).

Key findings:

• Of the evaluations included in this analysis, we found that of the reviewed MSNPs, the predominant sectors engaged were agricultural (n=25/30) and health sectors (n=19/30). Nearly two- thirds had some form of food, cash or asset transferred to beneficiaries either at the household or community level. The average length of most projects evaluated was five years with a range of three to 10 years.
• The majority of evaluations reviewed were of the pre- and post-test design with no comparison group. These were specifically designed to look at improvements over time on primary and secondary outcomes rather than draw conclusions on impact versus a control.
• Even the most rigorously designed evaluations noted the difficulty in attributing any or all impacts on the outcome indicators to the intervention alone.
• Of the reviews that included a control group, and therefore able to measure impact most robustly, the majority (7/11 (64%)) showed a positive impact on one or more primary nutrition outcomes (child stunting, wasting, underweight or anaemia). 
• Secondary nutrition outcomes were more widely measured in evaluations with 72% (18/25) showing a positive improvement in household dietary diversity and 65% (13/25) indicating a positive improvement in IYCF indicators.
• Two of the highest rigour evaluations did not include coverage estimates for their intervention indicating that there can be a mismatch between the strength/ quality of the design and the extent to which it provides essential information especially relevant for MSNPs.
• While all of the 30 evaluations reviewed were of programmes engaging multiple sectors, with multiple interventions targeted at a common group (for example, the first 1000 days of life), interventions were often not delivered jointly or in a coordinated way across sectors.
• Most programmes had yet to go to national scale, despite many having national rollout ambitions. Only eight of 30 evaluations provided information on scale-up.

Recommendations:

Programme design and scale-up: Careful attention is needed when designing multi-sector programmes so that interventions or services from multiple sectors are able to reach the same target households or individuals in a coordinated manner. This convergence of sector programming is important for providing a comprehensive package of services that has the potential to impact the multiple interconnected drivers of malnutrition in a given context. MSNPs should be embedded in government structures and services and placed under government ownership to ensure programme coverage, convergence and scale-up at national level. Indicators to assess programme coverage should be integrated into national information systems within a broad range of sectors (health, agriculture, education) and a more objective way of comparing levels of programme convergence and reviewing lessons learnt in this regard should be explored. Process evaluations that look more qualitatively at project implementation with a particular focus on the integration of sectors should also be encouraged by the donors funding interventions.

Measuring impact: Guidance on effective and standardised MSNP evaluations is needed, as well as greater availability of funding for quality, largescale evaluations to take place. A minimum level of rigour should be set, ideally allowing for at least the assessment of change in outcomes between time points interpreted against the backdrop of secular trends. Ensuring the inclusion and importance of secondary nutrition outcomes (such as HDDS, food insecurity scores, indicators of IYCF and WASH practices, standardised measures of women’s empowerment and indicators of household finances) rather than largely focusing on stunting impact is both important and more realistic for many programmes. Donors can play a key role in ensuring these standards are set and met and that programmes have sufficient funding allocated to enable rigorous evaluations and the measurement of impact. Innovative solutions allowing for the regular monitoring of undernutrition need to be developed to reduce reliance on standard impact-evaluation and periodic surveys. The use of the growth monitoring programme as a means of tracking improvements in nutrition should be explored. This could be achieved without rolling out national measurement of length or height through sentinel site surveillance or targeting children at certain ages.

A number of interesting articles on multisectoral programming in FEX 65 can be accessed here and are reflected on by Natalie Sessions in a recent blog here.