Lessons learned in West Darfur: challenges in assessment methodologies

By Aranka Anema

Aranka Anema has a background in medical anthropology. She is currently working with MSF-Holland as Medical Editor and has been involved with the Nutrition Guidelines and nutrition evaluation workshops.

Over the last twenty years, humanitarian organisations have accumulated a wealth of technical expertise in nutrition to guide emergency interventions. Causation models have been fine-tuned to predict the scope and severity of nutritional crises. Early warning systems and nutritional surveillance tools impart information about market fluctuations, food availability and insecurity. Anthropometric indices (e.g. Z-scores, Weight for Height) provide data about individual and population nutritional status, indicate the severity and prevalence of malnutrition, and detect vulnerable groups. Furthermore, mapping systems estimate changes in food access, means of livelihood and coping mechanisms.

Malawi, a food and seeds distribution in Zomba District June 2002

In spite of these developments however, humanitarian aid organisations are often still unable to assess accurately the extent and severity of looming and/or actual nutritional crises, or determine appropriate and optimal intervention strategies. MSF recently conducted an evaluation workshop to address challenges it faced during a nutritional intervention in West Darfur (2001).¹ Questions remained about how and why MSF's nutritional assessments failed to predict accurately West Darfur's nutritional situation, and consequently failed to steer effective and appropriate programme responses.

Unpredictability in the Context of West Darfur

West Darfur has a population of 1.6 million people, 90% of whom live in isolated rural areas. The region is affected by sporadic tribal conflict and poor rainfall, resulting in sudden market fluctuations, livelihood changes and displacement. The national health system is non-functional (e.g. primary health care is limited and inaccessible; drug supply and vaccination coverage are low; medical staff numbers are inadequate). The combination of geographic isolation, erratic precipitation, existing socio-political instability and an ineffective health system render inhabitants particularly vulnerable to medical and nutritional problems. Seasonal food shortages (or 'hunger gaps') in the region combined with drought create a time bomb in the waiting.

Whilst West Darfur chronically teeter-totters between sustainable food security and food crisis, inhabitants employ diverse coping mechanisms to survive. They engage in alternative systems of trade, adopt different means of livelihood and modify their staple diet (e.g. from sorghum and meat to wild berries, nuts, seeds, etc.). It has been argued that premature humanitarian intervention can interfere with local coping capacity, and create a culture of dependency. The alternative to this, however, is to delay humanitarian aid, and risk causing increased mortality and morbidity. Humanitarian organisations are thus faced with the operational dilemma of determining when exactly they should intervene.

Assessment Tools Gone Awry

Central to determining an appropriate time for intervention is the accurate prediction of a population's nutritional needs. Market and livelihood assessment indicators provide key information about whether and when a 'hunger gap' is developing into food crisis. However, as MSF experienced, these nutrition information systems cannot capture the complexity and unpredictability of nutritional situations.

In December 2000, the FAO and SC-UK reported alarming food shortages in Western Sudan. Market and livelihood indicators pointed towards an imminent nutritional crisis in North Darfur:²

• The harvest in 2000 was reported to be worse than in previous years, due to insufficient rainfall, pests and insecurity.
• Market prices for sorghum and staple grains (e.g. millet) were at a record high (prices had doubled and in some instances tripled, compared to previous years).
• The price of livestock (sheep and goat) had plummeted.
• Daily wage rates were decreasing.
• Residents were selling assets, pursuing alternative income generating activities, migrating for labour and changing their sustenance habits (i.e. in meal type and frequency).³

MSF-H conducted its own assessment in West Darfur, confirming SC-UK's findings. Market and livelihood indicators suggested that rates of severe malnutrition would increase dramatically within the coming few months, and that humanitarian intervention was imperative to avoid wide scale loss of life and destruction of livelihoods.

MSF-H and SC-UK decided to take a proactive approach to maintaining food security and local coping capacity. MSF-H deliberated about whether to initiate a general food distribution (GFD) to cover the entire population and a Blanket Food Programme (BFP) targeted to vulnerable groups, or to adopt an alternative approach. In the end it was decided that a GFD would be too complicated to implement effectively due to poor logistical and human resource capacity (i.e. inadequate staff, food pipeline and institutional networks), lack of political support from the Sudanese government, and difficult access due to the rugged geographical terrain. MSF-H opted for a measles vaccination campaign, coupled with MUAC screening. These initiatives were supported by emergency preparedness measures (e.g. the delivery of foodstuffs and materials) before the rainy season. A nutritional surveillance was implemented to monitor trends in nutritional status, internal displacement and coping capacity, and produce nutritional data to lobby for a GFD.⁴

Indecision set in, however, when assessment findings from SC-UK and MSF-H showed discrepancies in malnutrition rates. SC-UK's Weight-for-Height assessment in North Darfur revealed a global malnutrition of 23.4% among the resident population and 26.1% among the displaced. MSF-H's MUAC assessment in West Darfur, on the other hand, yielded global malnutrition rates ranging between 4-12%. The inconsistency in malnutrition rates between the organisations prompted MSF-H to repeat its survey in West Darfur using Weight-for-Height measurements. The second assessment confirmed preliminary MUAC results and the fact that West Darfur's nutritional situation was not as critical as in North Darfur. Confusion stemmed from the fact that the nutritional situation in West Darfur was expected to be the same as, or even worse than, that of North Darfur. The discrepancy between MSF-H and SCUK's assessment findings was eventually explained by a combination of factors: variations in methodology and cut-off points (i.e. MUAC versus Weight-for-Height) and geographic differences in nutritional status (i.e. between North and West Darfur). Erring on the side of caution, however, MSF-H continued its operation.

The general indecision about whether a food crisis was even imminent was confounded by problematic access to West Darfur. The onset of the rainy season, lack of roads and geographic dispersal of the population caused logistical constraints and dilemmas about appropriate intervention strategies.⁵ MSF-H vaccinated 18 000 children, and by August 2001 determined the situation was sufficiently stable to close the programme.

Lessons Learned

MSF-H's experience in West Darfur brings to light several ideological and operational issues. In an attempt to clarify decision-making procedures for future situations, working groups at the MSF-H evaluation addressed several action points:

1. MSF-H agreed that a proactive or preventative approach to nutritional interventions is necessary in all contexts, regardless of the potential for unnecessary expenditures on financial and human resources. Once significant morbidity is reported and high mortality is imminent a reactive approach is considered ineffective, since it implies the late provision of assistance. A proactive approach addresses the causes of mortality and prevents potential for rises in morbidity and mortality.
2. MSF-H's proactive approach should be triggered by forecasts from other sources (e.g. early warning information from reliable organisations such as WFP, FAO, SC-UK in Darfur 2001). However, since early warning information can be misleading through inter-organisational inconsistencies in assessment methodology, differences in assessment protocol need to be systematically communicated. Methodologies should be clearly defined in advance, and cut-off points should be transparent and adhered to during the assessment process.
3. Contrary to MSF-H's predictions, West Darfur's population proved to be resilient and innovative in the face of food shortage. Although socioeconomic and anthropometric indicators provide invaluable facts about food access, availability and insecurity, they cannot identify the totality of factors that influence nutritional situations. The dependency of all organisations on quantifiable assessment data risks leaving out other crucial information. In order to act in a timely and appropriate manner, MSF-H must gather more context-specific information about coping strategies, specifically among populations experiencing chronic food insecurity. By understanding traditional coping strategies with respect to food shortages (e.g. seasonal migration patterns, trade routes, etc.) humanitarian organisations can potentially distinguish between 'normal' famines and 'famines that kill.'⁶
4. MSF-H agreed that future decisions to intervene in West Darfur would be determined by the same methodology as the 2001 experience; namely broad-based assessment findings (including more relevant socio-cultural information) and analyses of available infrastructure and resources. Meanwhile, the decision about exactly when to intervene was refined. MSF-H has labelled the grey area between early warning information and the actual onset of a nutritional problem as the 'window of opportunity' for proactive intervention. Proactive interventions will be initiated immediately after the early warning stage, and triggered by the prevalence of the majority of the following indicators: global malnutrition rate over 20%, increase in displacement and morbidity, wide-scale slaughter of cattle, and degradation of geo-political security.

While a lack of response to early warning information can be disastrous, an inappropriate response to assessment indicators can take away from local coping capacity and waste humanitarian aid resources. The 2001 West Darfur experience enabled MSF-H to evaluate past strengths, weaknesses, opportunities and trends that can assist future decision-making and policy. MSF-H looks forward to further refining its proactive approach by gathering context-specific information about coping strategies, promoting inter-organisational communication about assessment methodology and defining its 'window of opportunity' for intervention.

Comments on the above article from Anna Taylor, Nutrition Advisor, Save the Children UK.

Save The Children UK welcomes MSF's endeavour to evaluate assessment finding in Western Darfur last year. However, I would like to comment on some of the findings as well as make a number of additional points.

Firstly, Darfur is a huge place. North Darfur alone is the size of France. As far as we know, MSF was operating in one rural council in West Darfur. It may be dangerous to make generalisations about the whole of West Darfur based on the experiences of one area alone. Save the Children UK conducted a Household Economy Assessment jointly with WFP, HAC and GoS in late March 2001 in West Darfur. The data collection and analysis was not as comprehensive as the data collected in North Darfur as baseline assessments had not been conducted. However, 14,534MT was estimated to be needed in the form of free food. Save the Children intervened in May by distributing 3,000MT of food aid. WFP committed 5,000MT but delivered only 900MT and the government in West Darfur distributed over 1000MT. In July 2001 rates of malnutrition were found to be 15% and 21% (including 4% severe malnutrition) in the two food economy zones studied. These data were gathered using two-stage cluster surveys. The latter, at least, is a rate concomitant with a food and nutritional crisis.

MSF's interpretation of events appears to be that a crisis was predicted but never occurred for a variety of reasons, including the depth of coping strategies of the affected community. However, we are not convinced that MSF's data were of adequate quality to draw these conclusions and even if they were data drawn from one rural council should not be used to generalise about the entire province. Save the Children's analysis of the situation in West Darfur was that it became serious but that general food distributions probably mitigated the worst effects of the crisis.

Finally, I would like to comment on the lessons learned and add a few key lessons from our own experience. Certainly, Darfur presents an extremely difficult operating environment. Save the Children's work in Darfur has shown the importance of an indepth understanding of the context of the emergency intervention in order to be effective and timely. This can only be achieved through consultation and communication with all key stakeholders and through careful assessments of the economic situation as well as nutritional circumstances. For SC UK a proactive approach means intervening before malnutrition rates reach high levels in order to prevent nutritional deterioration and protect livelihoods.

¹Nutritional Interventions in Open Situations: Evaluating the Experience es in Darfur, North-Sudan. MSF-H Workshop: Amsterdam, January 31, 2002

²Save the Children (UK). Darfur Food Information Bulletin. Feb. 19, 2001 Volume 54

^3,4MSF-H Assessment Report:West Darfur State, Sudan. March 16-23, 2001

⁵Anema, A. 2002 'Lessons Learned in West Darfur: Challenges in assessment methodologies.' Field Exchange. Issue 16.

⁶De Waal, Alex 1988 'Famine Early Warning Systems and the Use of Socio-economic Data'. Disasters. 2(1):81-91