Impact evaluation of BSFP during a nutrition emergency in Kenya
By Cyrus Shahpar and Leisel Talley
Cyrus Shahpar is a medical epidemiologist with the Emergency Response and Recovery Branch at the US Centres for Disease Control and Prevention. His research interests include evidence-based use of new technologies in complex humanitarian emergencies.
Leisel Talley is an epidemiologist with the Emergency Response and Recovery Branch at the US Centres for Disease Control and Prevention. Her research interests include moderate acute malnutrition and prevention of acute malnutrition in humanitarian emergencies.
The authors would like to thank our many partners in Kenya who conducted the BSFP programme. This includes ALDEF, IRC, Islamic Relief, Merlin, Oxfam, Save the Children, UNICEF andWorld Vision. Special thanks go to our organising partner, WFP Kenya, and especially Yvonne Forsen and Joyce Owigar. Lastly, this project could not have been done without dedicated support from the WFP offices in Lodwar and Wajir, and the many Kenyan staff who conducted the evaluation.
The findings of this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention
Location: Kenya
What we know: BSFPs have become a standard approach to food security crisis to prevent a decline in nutritional status in vulnerable groups, usually children. There is a lack of evidence of impact on nutritional outcomes in different contexts. BSFPs are resource intensive and have many implementation challenges.
What this adds: A prospective evaluation of a BSFP (five 30 day distributions over eight months) in two counties of northern Kenya was conducted using a longitudinal cohort of non-malnourished children (6-36 months). As a group, nutritional status improved but a subset of children became malnourished. In Turkana, most of those who became malnourished recovered, however considerably less did so in Wajir. Reported access to treatment of severely malnourished cases was very low. Children who had a lower WHZ or MUAC or lower dietary diversity upon enrollment were more likely to develop acute malnutrition. BSFP 30-day rations were consumed in less than 14 days for more than half of beneficiaries. Recommendations include consistent use of a single admission criterion such as height, timely distribution and strengthened referral systems/follow up for malnourished cases.
From 2010-2011, Kenya experienced a severe food security crisis affecting an estimated 3.75 million people1. Nutritional assessments conducted between April-June 2011 in northern Kenya, found alarming estimates of the prevalence of acute malnutrition among children less than five years of age of 24.4-37.4%. These data indicated a critical situation and significant deterioration from previous nutritional assessments conducted in 20102.
The World Food Programme (WFP) and partners responded by implementing a blanket supplementary feeding programme (BSFP) in six counties (Marsabit, Isiolo, Mandera, Wajir, Turkana, and Samburu) in northern Kenya. The initial target population of children 6-36 months of age or less than 95 cm in stature and pregnant and lactating women (PLW) was provided with monthly rations of corn soya blend and oil, systematic interventions (vitamin A supplementation, deworming and immunisation) and health education.
Methods
A prospective evaluation of the BSFP in northern Kenya was conducted using a longitudinal cohort of non-malnourished children between 6-36 months of age enrolled in the BSFP in two of the six counties (Turkana and Wajir). Thirty evaluation sites were selected in each county, based on the anticipated number of children who would attend the first supplementary food distribution at each distribution site. The cohort was identified and enrolled during the first distribution; there was no subsequent enrollment. The cohort was followed for four subsequent distributions; distribution was intended to occur at 30 day intervals at each distribution site. The duration of a distribution cycle (the number of days between the distribution at the first distribution site and last distribution site) and the mean interval from prior distribution (the average number of days between the current distribution and prior distribution across all sites) were calculated. Reasons for delays in distribution were observed.
At each distribution site during each distribution, a household questionnaire was administered to caretakers of the children enrolled in the cohort. The questionnaire collected data on potential risk factors for malnutrition, including household demographics, recent morbidity and treatment, water and sanitation, utilisation and consumption of the supplementary ration (sharing, selling etc.), access to general food distribution or other programmes, household food security, admission into therapeutic feeding programmes, and feeding practices. We also asked about dietary diversity as calculated by summing the 7 food groups consumed on the prior day (1 point per food group: cereals, pulses, dairy, meat, eggs, vitamin A rich fruit and vegetables, other fruit and vegetables). Anthropometric measurements (weight-for-height Z score [WHZ] and mid upper arm circumference (MUAC)) were also collected for enrolled children using standard techniques3.
Primary outcomes were the overall change in mean WHZ, and the incidence of acute malnutrition (WHZ <-2 standard deviations, a MUAC <125 mm or the presence of bilateral pitting oedema) at any distribution following enrollment. The difference in mean WHZ between distributions was analysed using paired t-tests for comparing group means. Bivariate comparisons of mean WHZ were used to evaluate individual risk factors for changes in nutritional status, and a hypothesis-driven logistic regression modeling was used to identify baseline risk factors for incident malnutrition. Methods are described in detail in the full report4.
Results
A total of 3,856 children were screened during the first BSFP distribution at 59 sites (29 in Turkana5 and 30 in Wajir) of whom 2,779 were eligible for enrollment in the evaluation (1,386 in Turkana and 1,393 in Wajir). Children who had implausible increases or decreases in height or weight or who were obviously not the enrolled child were excluded. This resulted in available data on 1,209 (87%) and 1,266 (91%) of children in Turkana and Wajir, respectively. Of these, 757 (63%) attended all five distributions in Turkana, and 1012 (84%) attended all five distributions in Wajir. These children were included in the analysis below.
Five distributions of thirty days of ration occurred over eight months with significant variation in the time between each distribution. Reasons for the large differences in the interval between distributions included delays in transport and lack of prepositioned commodities. In Turkana, the duration of distribution for each distribution cycle ranged from 33 to 78 days with mean interval from the immediate prior distribution ranging from 27 to 62 days. In Wajir, the duration of distribution for each distribution cycle ranged from 10 to 49 days with mean interval from the immediate prior distribution ranging from 29 to 78 days. In both counties, the period between distribution cycles two, three and four was approximately two months each.
During the BSFP, overall mean WHZ among children who attended all five distributions improved in both counties (Figure 1). In Turkana, mean WHZ increased from the first to the fourth distribution from -0.92 to -0.69, before plateauing at the final (fifth) distribution. In Wajir, WHZ rose from -0.89 at the first distribution to a high of -0.52 at the fifth distribution. In Turkana, the change from the first to all subsequent distributions was significant. In Wajir, there was a significant change from the first to all subsequent distributions; however the mean WHZ decreased significantly from the first to third distribution.
In Turkana and Wajir, 15% and 22% of the children, respectively, who came to all five distributions became acutely malnourished by our criteria during the evaluation period. For these children, at any distribution following identification of malnutrition, all severely malnourished children in Turkana returned to normal, while 98% of moderately malnourished children improved to a normal status at the subsequent distribution. In Wajir County, 54% of moderately malnourished children and 63% of severely malnourished children improved to a normal status by the subsequent distribution. Only 4% and 6% in Turkana and Wajir, respectively, reported currently receiving treatment for acute malnutrition at the subsequent distribution.
In both counties, missing one or more distributions, and baseline characteristics of having an illness in the two prior weeks, receipt of other food and non-food aid, household size, number of children in the household, having a parent caregiver, having a measles vaccination, vitamin A status, and ration sharing were not significantly associated with the mean change in WHZ from the first distribution (D1) to the fifth distribution (D5). In Wajir, baseline breastfeeding was significantly associated with larger increases in WHZ from D1 to D5. In both Turkana and Wajir, children who developed acute malnutrition at any distribution had a significantly lower dietary diversity than those who had normal nutritional status throughout the programme.
Attendance at a prior distribution and delays in distributions did not have an association with the development of malnutrition. Logistic regression modeling of baseline risk factors and subsequent development of malnutrition by any measure showed that low baseline WHZ (-2 to -1.5 compared to the WHO reference population) was the significant risk factor for developing malnutrition during the programme period.
Discussion and limitations
Our primary goal was to evaluate the impact of the BSFP as an integrated package (ration, immunisation, systematic treatment, and education) on preventing deterioration in nutritional status among children 6-36 months of age. This entailed specifically using WHZ, MUAC and presence of bilateral oedema for outcome determination. The results suggest that as a group, children enrolled in this BSFP experienced an overall improvement, not a decline in nutritional status by WHZ. This was statistically significant from the first to the last distribution in both Turkana and Wajir counties. These interesting results provide a detailed look at the cohort of children over an eight month period, but as expected, it was impossible to fully attribute these findings to the BSFP.
Clearly, the situation in these counties was dynamic and overall household indicators improved during BSFP. A key finding was that BSFP rations were consumed in less than 30 days; over half of households reported that the ration lasted less than 14 days in both Turkana and Wajir counties, less than half the planned distribution cycle of 30 days. Although this is not a new finding in ration-based programmes, it is important to consider in the analysis and interpretation of the data.
Despite the overall improvement in WHZ score, a subset of children attending all distributions (15% in Turkana, 21% in Wajir) developed malnutrition at least once. In both Turkana and Wajir, children who had a lower WHZ or MUAC, or who had a lower dietary diversity upon enrollment, were more likely to develop acute malnutrition. In general, some malnourished children will spontaneously recover to a normal nutritional status and this happened in both counties. In both Turkana and Wajir, very few malnourished children reported ongoing treatment or treatment in the prior month.
Significant risk factors for developing acute malnutrition while enrolled in the BSFP varied by county, highlighting the complexity of implementing standardised BSFP across large geographic areas and diverse populations. Overall, the inconsistency between significant risk factors by location may be due to spurious associations expected when examining a large number of variables in a statistical model. Clearly, children with low baseline WHZ score were at significant risk for subsequent malnutrition.
The analysis presented in this report takes a conservative approach. If an individual had one implausible value in an anthropometric measure or index at any visit, they were excluded from analysis. Despite the presence of implausible anthropometric measures, children with implausible data were not different to children with valid measures with regard to child and household characteristics.
The implementation of BSFP was challenging, and this evaluation assessed the programme as it was actually implemented. Our data highlighted issues with the application of admission criteria into the programme, as 20% and 25% of children enrolled in the programme and subsequently included in the cohort should have been deemed ineligible for the BSFP programme by height or age (i.e., children above 95 cm in height and/or older than 36 months) in Turkana and Wajir, respectively. The BSFP was designed as five monthly distributions, but challenges such as insecurity, poor coordination, inconsistent food pipeline, and transport difficulties led to frequent delays in distribution.
There are several limitations to this evaluation. In general, the impact of a BSFP is methodologically difficult to assess since individuals are not routinely followed over time in these programmes. Furthermore, as it is a blanket programme, there is no control group for comparison. Additionally, the BSFP in northern Kenya was an integrated programme with a ration and a package of interventions, making it difficult to identify any singular effect. There are also many potential confounders at the child, household, village and county levels, which we did not control for that could influence the nutritional status of the cohort over time. The most prominent of these is the overall improvement in food security and pasture and animal conditions, as a result of adequate rainfall during the evaluation period.
Several questions arose during analysis. A key issue is the development of acute malnutrition and more specifically, what is an acceptable level of malnutrition in a BSFP? We do not have consensus or guidance on this question. The absence of established benchmarks for both of our outcome indicators (change in WHZ score and incident malnutrition) make evaluation of BSFP effectiveness difficult to determine. Also, given the complexity of BSFP and the implementation of the strategy, what is the best evaluation methodology for BSFP?
There are several recommendations for improving BSFPs based upon both the data from the evaluation and field observations made during the distributions. First, a single admission criterion such as height (rather than age) should be used by all programmes, and this should be consistently applied across all distribution sites. Second, to obtain the greatest impact of the programme, the provision of the ration must be timely and with minimal delays. Third, referrals of malnourished children to treatment programmes for acute malnutrition need to be strengthened and guidance on referral and follow-up of cases should be provided to the partners providing services for treatment. Finally, BSFPs have become standard programming in nutrition emergencies; ongoing evaluation of these programmes should be considered given the tremendous resources required.
For more information, contact: Cyrus Shahpar, email: cshahpar@cdc.gov
1United Nations Office for the Coordination of Humanitarian Affairs. (2012). Mid-year review of the 2012 Kenya Emergency Humanitarian Response Plan. Full article available at: http://www.unocha.org/cap/appeals/mid-yearreview-2012-kenya-emergency-humanitarian-response-plan
2Nutrition Working Group. (2011) Kenya Nutrition Survey Summary. Full article available at: http://ochaonline.un.org/kenya/SectoralWorkingGroups/NutritionWG/tabid/4444/language/en-US/Default.aspx
3United Nations, editor. How to Weigh and Measure Children, Assessing the Nutritional Status of Young Children in Household Surveys. New York: United Nations; 1986
4valuation of a Blanket Supplementary Feeding Programme in Two Counties in Kenya, August 2011 – March 2012 http://www.unicef.org/nutritioncluster/files/BSFP-Report-Cleared-19March2013.pdf.
5One selected site in Turkana was not visited because it was inaccessible to the evaluation team