Letter challenging conclusions and analysis of Lancet Undernutrition Series, by Fiona Watson and Carmel Dolan

Challenging conclusions and analysis of the Lancet Undernutrition Series

Dear Editor,

We welcome the opportunity provided by the Lancet to highlight the "desperately neglected" subject of nutrition through its series on maternal and child undernutrition. We appreciate the substantial scope of the series as well as the scientific rigour applied in producing it. We find the analysis in many instances to be both thoughtful and perceptive. However, we take issue with some of the major conclusions of the series and the fact that certain areas of analysis have been over-simplified and poorly developed.

There are four key conclusions in the series which we would like to briefly comment upon:

• There is a lack of evidence about effective programming.
• There are a small group of ‘single’ interventions that are effective in reducing undernutrition.
• There are a small number of countries who have successfully reduced undernutrition.
• There is a lack of leadership and ‘fragmentation’ in the international nutrition system.

Lack of evidence base

The authors cite a limited evidence base for programme effectiveness but, at the same time, throw out a great deal of data because of failure to meet criteria for admissible evidence. The vast bulk of the acceptable evidence is for single interventions, which address the immediate causes of malnutrition. Single interventions are easier to evaluate and demonstrate impact on malnutrition levels than multi-sectoral interventions. Randomised control methodologies may be applied to the former, but are very difficult, in practical terms, to apply to the latter.

We feel that there has been a lack of caution from authors in highlighting which interventions are ‘proven’ to be effective. The impression given is that unless there is evidence of effectiveness using strict scientific criteria, an intervention should not be implemented. For example, it is stated in paper 4, page 1, "Nutrition resources should not be used to support actions that have not been proven to have a direct effect on undernutrition...." This would effectively mean that all emergency food aid programming (with the exception of foods used in therapeutic care programmes) should cease as there is no firm evidence base for its effectiveness (Duffield et al 2004 ).

What we would like to have seen addressed is the question why there haven’t been more evaluation studies which meet the scientific criteria (randomised control trials) set by the Lancet? The obstacles to such evaluation need to be understood, in particular the need to develop more rigorous and operationally feasible methods to evaluate the impact of multifaceted approaches. Scientific evaluation of impact needs to be planned at the outset of programmes. Donors need to be aware and willing to provide the resources needed to fund rigorous scientific evaluation. Currently, donor funding mechanisms for interventions tend to favour agencies with a ‘track record’ for delivering outputs rather than outcomes (for example the delivery of food rather than the impact of the food on undernutrition). The focus on outputs rather than outcomes at donor level reflects administrative pressures and the lack of nutritional expertise within donor agencies.

We would have liked the Lancet to highlight the lack of epidemiological expertise, within donor and implementing agencies, to develop and implement appropriate methodologies for impact evaluation.

Another key factor which allows this ‘status quo’ to prevail is the lack of institutional accountability for programme outcomes. There is no single agency with a mandate or responsibility to ensure that nutrition intervention impact is assessed and, therefore, no overview within the system as to which programmes or design features deliver the desired impact. Consequently it is left to entrepreneurial individuals or agencies to amass such evidence and ultimately push for better designed interventions. Such an ad hoc situation is unsupportable. While the Lancet series highlights this deficiency (paper 5), there is no institutional analysis of how to remedy this situation. An important opportunity for advocacy has therefore been missed.

Small group of effective interventions

The Lancet identifies 14 interventions of which three (breastfeeding counselling, vitamin A supplementation and zinc fortification) have the greatest proven benefits in terms of reducing stunting and micronutrient deficiencies. Most of these 14 interventions are ‘single’ interventions (eight on micronutrient supplementation or fortification, three on promotion of good infant and young child feeding through counselling and behaviour change, one on treatment of severe malnutrition, one on intervention to reduce tobacco consumption and one on handwashing and hygiene intervention).

The reviewers in paper 3 report that they have "analysed large-scale nutrition programmes, to derive estimates of population effect, achievable coverage levels and sustainability" but it is unclear what conclusions have been drawn about the success or otherwise of multi-sectoral nutrition programmes. Only one large-scale nutrition programme met the criteria for prospective evaluation (Progressa in Mexico) and the authors conclude that "given the paucity of effectiveness data, strengthening of monitoring and rigorous assessment of large-scale nutrition programmes are imperative".

Furthermore, certain interventions were excluded from the analysis (education, untargeted economic strategies or those for poverty alleviation, unconditional cash transfers and microcredit programmes etc.) Overall, the implication is that the short route of single interventions is the way to reduce undernutrition. There is no discussion about the added benefit of linking interventions together (for example growth monitoring alone cannot impact on nutritional status but linked with vaccination, vitamin A, iron/folate supplementation, breastfeeding counselling, de-worming, may well have an impact). Neither is there discussion about linking short-term with longer-term strategies, though implicit throughout the series.

We were disappointed by the failure of analysis. We would at least like to have seen an analysis of funding decisions in relation to nutrition programming. For example, has funding been biased in favour of short term, single interventions, such as micronutrient supplementation, and not on longterm investment in governments and longer-term multi-sectoral programmes?

Small group of countries where undernutrition has been reduced

A number of countries are reported to have successfully reduced undernutrition. These are Costa Rica, Cuba, Sri Lanka, Thailand, Mexico and, more recently, China. However, the analysis as to how these improvements occurred is limited and is largely descriptive. Crucially, the link between the 14 nutrition interventions identified in paper 3 and the country level improvements is not made explicit. For example, did improvements in nutritional status in China arise mainly due to micronutrient supplementation and improved rates of breastfeeding or are more complex factors or longer term factors at play and if so, what are these?

Furthermore, there is no analysis at a country level of the ‘how’ in relation to nutrition programming rather than the ‘what’. We would have liked the ‘how’ question to have been addressed in this series. How have some countries been able to reduce undernutrition while still remaining poor, while in other countries the opposite seems to have occurred? What social, political and funding mechanisms were needed and how much capacity development was necessary to achieve these reductions?

Lack of leadership and fragmentation

The lack of responsibility for, and leadership in, international nutrition is recognised and the ‘fragmentation’ in the sector identified. But there is no analysis of why or recommendations for what can be practically done.

What is lacking in this section is an historical analysis of past initiatives to improve coordination and coherence in the sector and why these have petered out over time. For example, there is no analysis of why post-International Conference on Nutrition initiatives evaporated. What de-railed the process?

Analysis of the nutrition ‘architecture’ for responding to emergencies is especially weak. Generalisations about the politicisation of food aid are poorly supported by the evidence presented, while the issues highlighted are a regurgitation of the recent literature and add little insight into ways forward. The authors could have made far better use of historical analysis and positive examples to indicate how to make progress where this is needed.

In conclusion, we were disappointed by the superficial analysis of the Lancet series in a number of important areas especially in light of the efforts put in to the quantitative analysis. We feel that there has been a lack of answers to critical questions relating to the lack of an evidence base, the ‘how’ of nutrition programming, funding decisions and accountability.

What most concerns us is what a senior government figure from a developing country would make of the conclusions. Will s/he conclude that national nutrition policy should be re-written to focus only on micronutrient supplementation/fortification and promotion of good infant and young child feeding practices? What other policy and strategy insights has the Lancet series given us to support improvements in nutrition in what is acknowledged to be this "desperately neglected" area which claims millions of young lives every year?

Regards
Carmel Dolan and Fiona Watson,
NutritionWorks