Review of Published Evidence for Impact of Emergency Feeding Programmes

Summary of unpublished paper¹

The ENN have recently completed a review with funding from CIDA of the published evidence for the impact and cost-effectiveness of 6 key humanitarian interventions commonly implemented in emergencies (general rations, supplementary feeding, therapeutic feeding, measles vaccination, vitamin A supplementation and bednet distributions). The overall aims of the review were to identify gaps in the literature and develop methodologies and institutional mechanisms for filling these gaps.

The review has focused on a narrow definition of impact - it is measured as a change in population nutrition prevalence or mortality rates due to an intervention.

A hierarchy of study types is generally recognised in clinical medicine with the randomised control trial considered to be the 'gold standard' method providing the highest level of evidence. Observational studies and case-series data provide the weakest level of evidence. Given the practical difficulties of conducting an RCT in an emergency situation this report uses a framework (adapted from Habicht et al, 1999²) with which to assess the strength of the different studies reviewed. All types of economic evaluation information were included in the review.

Standard methods to collate and appraise the literature for a systematic review were employed. The method involved

1. a search of the 5 most relevant databases,
2. a secondary reference search,
3. a hand search of the main journals, and
4. expert advice on the literature.

The quality of each study was assessed using standard critical appraisal techniques.

The most important finding of the review is that very few studies assessing the impact of any of the interventions in an emergency context have been published. There is virtually no publicly available information on the cost-effectiveness of different nutrition-related interventions commonly implemented in emergencies.

The number of published impact and economic evaluation studies undertaken in emergencies located by the search are shown below.

A limited number of studies assessed the impact of GFD and SFP in emergencies, however the majority of these were observational and do not provide very plausible evidence of impact. The evidence base for TFPs is somewhat stronger.
The lack of published impact and cost effectiveness information - particularly in relation to emergency feeding and food security support programmes is of enormous concern. There are key areas of uncertainty regarding both the utility of certain types of intervention, e.g. SFP or GFD versus cash transfer, and over issues of design within programme types, e.g. community versus administrative targeting in general ration programmes. There are also rapidly emerging new types of programming at the interface of HIV and nutrition for which impact and cost information is urgently needed. This lack of impact and cost-effectiveness information militates against cross-sectoral comparison of interventions in relation to nutrition and mortality impact.

There are a number of understandable reasons for the dearth of published information, e.g. the ethical difficulties of undertaking research in emergencies and the fact that there are far fewer epidemiologists involved in emergency feeding than in more medically oriented interventions like measles vaccination. However, one overarching key factor is the absence of an agency with responsibility for taking an overview of the effectiveness of different types of intervention and intervention design. This lack of corporate accountability has allowed the institutional status quo to prevail. Thus agencies which have built up expertise and mandates around certain types of intervention (or intervention design) will continue to practice these in emergencies without serious examination or challenge.

This review argues that one way to address the gap in information on impact and cost-effectiveness is to make greater use of the so called 'grey literature' (unpublished information held mainly by implementing agencies which may be in a variety of forms, e.g. project reports, annual audits, monitoring forms, etc). Greater standardisation of agency reporting will enhance capacity to use this type of information. However, it is also probable that much of this grey literature could be used retrospectively to answer a number of questions. The review discusses how to increase access to, and use of, the grey literature. In conjunction with this, specialised impact studies could also be commissioned to address key questions. The review examines how these studies may be carried out for each of the 6 interventions by identifying the most ethically feasible and methodologically robust approach.

The review also explores the gap in information on costs of interventions and methodologies for obtaining such information. It is recognised that this is not a straightforward discipline and that methodologies need to be developed, and reporting standardised.

Given the multiplicity of stakeholders and (vested interests) in this sector the review argues the case for creation of an independent body/institutional mechanism with responsibility for increasing information on impact and costeffectiveness in this sector. Without establishing such a body it is likely that little will change. This body would take responsibility for identifying key gaps in knowledge regarding impact and cost-effectiveness. It will develop and co-ordinate mechanisms for making greater use of the grey literature and promoting impact studies. The agency would also have an advocacy role where emerging evidence indicates a need for change in implementation practice.

In the event that there is insufficient support for establishing such a body a more piecemeal and potentially realisable alternative may be for donors (individually or as a group) to take more responsibility and fund/help establish research/implementing agency partnerships which aim to address specific questions in particular programme areas where impact and costeffectiveness information are urgently needed.

For more information contact; J. Shoham at jshoham@easynet.co.uk

¹Duffield.A et al (2004): Review of the published literature for the impact and cost-effectiveness of six nutrition related emergency interventions. Paper prepared by the ENN.

²Habicht, JP, Victora, CG and Vaughan, JP (1999) Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact. International Journal of Epidemiology 28: 10-18.

³The economic evaluation of bednets was a cost-effectiveness study which included a measure of impact so this study could be classified in either column.