IYCF assessment with small-sample surveys - A proposal for a simplified and structured approach

By Ernest Guevarra (VALID International), Katja Siling (VALID International), Faraja Chiwile (UNICEF Sierra Leone), Mueni Mutunga (UNICEF Sierra Leone UNICEF Sudan), Joseph Senesie (UNICEF Sierra Leone), Walton Beckley (UNICEF SierraLeone), Hamidine Hassane (UNICEF Niger), Massaoud Williams (Niger National Institute of Statistics), Kazim Lamine (Niger National Institute of Statistics), Mara Nyawo (UNICEF Sudan), Farah Mohamed Ibrahim (UNICEF Sudan), Masresha Tessema Anegago (EHNRI), Tesfaye Hailu Bekele (EHNRI), Gezahegn Shimelis Taddesse (Concern), Terefe Getachew G'Tsadik (Concern), Adane Tefera Beyene (Concern), Pankaj Kumar (Concern), Grant J Aaron (GAIN), and Mark Myatt (Brixton Health)

This article describes an approach to assessing infant and young child feeding (IYCF) practices using small-sample surveys which was developed jointly by VALID International; CONCErN Worldwide; Save the Children; UNICEF in Sierra Leone, Niger, and Sudan; the Sierra Leone Ministry of Health and Sanitation; the Niger National Institute of Statistics; the Ethiopian Health and Nutrition research Institute (EHNrI); the Sudanese Federal Ministry of Health, the Global Alliance for Improved Nutrition (GAIN), and Brixton Health. It is, in large part, a development of earlier work undertaken by the International Food Policy research Institute (IFPrI) and the Food and Nutrition Technical Assistance (FANTA) project.

The purpose of this document is to describe ongoing work and to propose a structured approach for IYCF indicators suited for use in small-sample surveys (i.e. sample sizes similar to or smaller than the n = 210 used in EPI vaccine coverage survey).

We have attempted to address problems that we have experienced using the set of IYCF indicators that have been proposed by the WHO. We do not believe that we have all the answers. We will have got some things wrong. Our intention is to let the emergency nutrition community know what we have been doing in the hope that our mistakes can be corrected and our work improved.

The statements in this publication are the views of the author(s) and do not necessarily reflect the policies or the views of UNICEF.

Location: Global

What we know: There is increasing demand to assess IYCF practice in communities to inform programming design and monitor impact. Standard WHO IYCF indicators exist that require sample sizes not achievable in many programme areas.

What this article adds: An approach has been used in Sierra Leone, Niger and Sudan to assess IYCF practices using smallsample surveys. It produces a principle composite IYCF indicator to classify IYCF practice as ‘good’ or ‘not good’ amongst 0-23 month old children. A set of diagnostic indicators is also calculated. The sample sizes used are 210 children or less.

The precision achieved was similar to that achieved by a typical EPI vaccine coverage survey. Field teams have found the approach user friendly, efficient and cost effective. There are limitations; for example, the data collection method is a simplified version of WHO methodology that has not been validated. The authors welcome feedback to develop this work to address an important gap area.

The problem

A set of IYCF indicators has been proposed by the WHO for the population level assessment of IYCF practices^1,2. The proposed indicators are intended for use with large-sample surveys (e.g. MICS, DHS) and are not suited to monitoring and evaluating sub-national (i.e. regional, district, and subdistrict) programs using small-sample surveys¹.

In our experience, there are problems with operationalising the indicators proposed by the WHO:

1. The indicator set lacks a clear structure or hierarchy. An overall indicator is not present and there is no clear procedure or guidance for interpreting the set of indicators as a whole.
2. Some of the indicators have very complicated definitions. Box 1, for example, shows a particularly complicated example (Indicator 7: Minimum acceptable diet). The WHO documentation notes:
   ....the calculation......appears cumbersome. However, most users will be processing data using computer software, which simplifies the calculation process².
   We are, however, unaware of software that performs the required calculation in a simple and standardised manner. This functionality is not present in commonly used survey software such as ENA for SMART. It is not present, without additional programming by the user, in any statistical package.
3. Many of the proposed indicators are unsuited to use with small samples. Indicator 4 (Introduction of solid, semi-solid or soft foods), for example, uses data collected for children aged between 6 and 8 months only. If, for example, we were estimating this indicator using data arising from a SMART survey with a sample size of n = 544 (i.e. the largest sample size mentioned in the SMART manual) then this indicator will be estimated using a sample size of about:
   
   n ≈ ((8 – 6 +1)/(59 – 6+1)) x 544 ≈ 30
   
   The most commonly used SMART sample design means that the effective sample size (i.e. after accounting for survey design effects) will likely be smaller than this. A sample size of n ≈ 30 is too small a sample size to provide an estimate with useful precision. Accurate and reliable classifications may be possible using sequential sampling techniques (e.g. LQAS) but no guidance is given regarding suitable class thresholds.
   
   This sample size problem means that many of the indicators proposed by the WHO are not suited for use with small-sample surveys or indicator mapping methods. The key document in which these indicators are defined cautions:
   
   … the sample sizes used in monitoring and evaluation of smaller scale programs may be quite small, some of the recommended indicators may be too imprecise to be of use in assessment or in monitoring change for these programs. This is particularly likely for indicators with narrow age ranges in the numerator and the denominator¹.
   
   This is the case with six (from fifteen) of the indicators proposed by the WHO.

The problem that we have tried to address in the work reported here is to create a simple, highly structured, and usefully comprehensive IYCF indicator set that can be used with small sample sizes.

The single indicator approach

The approach we have used is to produce a single indicator:

Percentage of children aged between 0 and 23 months receiving good infant and young child feeding

with “good infant and young child feeding” defined as exclusive breastfeeding in children aged under six months and as age-appropriate feeding practices (defined in terms of continued breastfeeding, dietary diversity, and meal frequency) in older children.

Age-appropriate feeding practice in older children is measured using an infant and child feeding index (ICFI) derived from the index devised by Mary Arimond, Marie Ruel, and Purmina Menon of the International Food Policy Research Institute (IFPRI) and subsequently developed by IFPRI and the Food and Nutrition Technical Assistance (FANTA) project as a Knowledge-Practices-Coverage KPC2000+ indicator^3,4,5:

* It is reasonable to use the more inclusive 12 - 24 month age-group here

The total ICFI score is a measure of appropriate child feeding practices:

ICFI = Breastfeeding + Dietary Diversity + Meal Frequency

using age-specific weightings (see Table 1) for each term. All children aged between six and twenty-three (or between six and twenty-four) months receive a score between zero and six. Children receiving a score of six are classified as receiving good infant and young child feeding. The scores given in Table 1 are presented as suggested values and should be subject to further review.

If the survey sample does not include children aged between zero and six months, as might be the case in a nutritional anthropometry survey (e.g. SMART) without a “top-up” sample of children aged below six months, then the ICFI score may still be used. The sample size used for the ICFI in a SMART survey with a sample size of n = 544 will be about:

n ≈ ((23 – 6 +1)/(59 – 6+1)) x 544 ≈ 180

Which is large enough to detect small changes in mean ICFI scores between survey rounds as well as to estimate proportions with useful precision (i.e. with 95% confidence intervals of better than about ± 10%). This enables (e.g.) periodic SMART surveys to be of use in assessment or in monitoring change for IYCF programs. This is something that the indicators proposed by the WHO cannot do.

The principal IYCF indicator is calculated from the counts of children found in the cells of a two-by-two table:

% GOOD = (Number classified as good / total number of records) x 100

This is the principal indicator for monitoring and evaluating IYCF programs. It is a simple estimate of the proportion of children in the population who are receiving good infant and young child feeding.

A set of diagnostic indicators are also calculated. These indicators show the contribution of breastfeeding, dietary diversity, and meal frequency to the calculated value of the primary indicator. This approach can be seen as a hierarchical indicator approach. Structuring the indicators in this way allows for effective analysis and presentation of data. For example:

Figure 1 shows the results from a survey with a sample size of n = 192 from m = 16 clusters undertaken in Sierra Leone. In this example, the poor level of the primary indicator is mainly due to poor feeding practices in older children in terms of meal frequency and (to a lesser extent) dietary diversity. It would be sensible for the IYCF program being monitored by this survey to focus attention on improving complementary feeding practices. The thresholds used in Figure 1 for the diagnostic indicators are suggested values and should be subject to further review.

Figure 2 shows the results from a small S3M indicator mapping survey (n = 1,392 from m = 29 clusters with local estimates calculated using data about n = 144 children taken from three neighbouring clusters each contributing data about 48 children) undertaken in Ethiopia. In this example, the main problems of IYCF are poor age-appropriate dietary diversity and low levels of exclusive breastfeeding particularly in communities in the south-west of the survey area. The thresholds used in Figure 2 for the diagnostic indicators are suggested values and should be subject to further review.

The structured indicator approach aids decision making by focussing attention on the overall program aim of improving IYCF practices.

Sample size and precision

Table 3 summarise the sample sizes used and the precision achieved for estimates of the proposed principal and diagnostic indicators in fifteen RAM type surveys using a sample size of n = 192 from m = 16 clusters undertaken in nine districts of Sierra Leone between June 2012 and September 2013. Table 3 also presents expected results from a SMART survey with a sample size of n = 544 children aged 6 – 59 months assuming a design effect of 1.5 and a uniform age-distribution.

1 Results from 15 surveys with a sample size of n = 192 from m = 16 clusters
2 Assuming n = 544 children aged 6 – 59 months with a design effect of 1.5 using expected
levels from 15 RAM type surveys
3 Half-width of 95% confidence interval (observed mean from fifteen RAM surveys,
expected precision for SMART survey)
4 Assumes a sample size of n = 544 children aged 6 – 59 months and a uniform age
distribution
5 Approximately one-quarter of the sample will be aged 0 – 5 months
6 No children aged 0 – 5 months in the SMART sample
7 Approximately three-quarters of the sample will be aged 6 – 23 months

The precision achieved for all indicators is similar to that achieved by other surveys of key child survival indicators such as EPI vaccine coverage surveys. Better precision may be obtained, if required, by increasing the overall sample size or by collecting the sample using more smaller clusters.

Experiences with the new IYCF indicator

We have now used the simplified and structured IYCF indicators in the form described above in the DRC, Ghana, Ethiopia, Niger, Sierra Leone, and Sudan. Our experiences with this approach to assessing IYCF practices have been:

The data are easy to collect, enter, and analyse. Box 2 shows a typical data collection form. Data may be entered and analysed in a spreadsheet by program staff 6. The calculation of indicators using spreadsheets has, however, proved to be error-prone. Dedicated software has been developed and is now available. The software is free, open-source, customisable, and can work with data in a wide variety of formats (e.g. plain text, dBase, SAS, SPSS, STATA, and EpiInfo/EpiData). Figure 3 shows a screenshot of the software being used with data from a standardised small-sample monitoring and evaluation survey of coverage (IYCF counselling, CMAM screening, vitamin A supplementation, anti-helminthic drug distribution, and growth monitoring), global acute malnutrition (GAM), IYCF, and WASH indicators from a district in Sierra Leone.

The indicators are integrated and multidimensional (see Figure 1). This makes results easy to report, present, and use.

The results shown here are from a small-area RAM type survey from Sierra Leone6

The indicators are easily interpretable by program staff and program managers. Table 1 and Table 2 show the complete calculation. These are readily understandable, and have a clear face-validity compared (e.g.) to the WHO proposed indicator presented in Box 1.

We have used the indicators in RAM and S3M type surveys. The indicators work well with the small sample sizes (i.e. typically between n = 96 and n = 192) used in these types of survey. The indicators may also be used with SMART type surveys. The use of the indicators in RAM and S3M type surveys is, however, more cost-effective. Experiences in Ethiopia, Niger, and Sierra Leone show that RAM and S3M surveys cost between about 20% and 25% of the cost of collecting similar data over a similar area using SMART type surveys but this may be context dependent. Ongoing work using RAM in an urban setting in Ethiopia suggests that costs may be as high as 45% of those associated with SMART surveys. The use of quick and cheap survey methods allows IYCF to be monitored over small areas and on a frequent basis without excessive expenditure on survey activities. The precision achieved by these surveys is similar to that achieved by a typical EPI vaccine coverage survey (see Table 3). The levels of precision achieved were considered useful by UNICEF, ministries of health, and NGOs for these surveys to be used to inform program design and to monitor program outcomes. Better precision may be obtained, if required, by increasing the overall sample size or by collecting the sample using more small clusters.

This is one of fifteen indicators proposed by the WHO

This questionnaire has been localised for use in Sierra Leone (i.e. local names and recipes have been used).

An age question is needed. This may be added to this questionnaire or be part of a larger
questionnaire of which this questionnaire is a component. Age should be recorded in months.

The exclusive breastfeeding (EBF) diagnostic indicator makes use of all collected data (i.e. not just the response to question F1) and is calculated as:

if F1 is TRUE and F2 is FALSE and F3 = 0 and
all {F4A, F4B, F4C, F4D, F4E, F4F, F4G, F4H} are FALSE
then EBF = TRUE
else EBF = FALSE
if AGE > 5 months then EBF = NOT-APPLICABLE

Question F4A is use to calculate the EBF indicator but is not treated as a food-group in ICFI and dietary diversity diagnostic indicators.

The results shown here are from an S3M type survey from EthiopiaLocal estimates are from n = 144 (n = 48 from each of three neighbouring clusters)

The results shown here are from a small-area RAM type survey from Sierra Leone⁶

The indicators can be complemented by the collection of other indicators relevant to IYCF and child survival such as food security, safe drinking water, good sanitation, coverage of (e.g.) IYCF counselling services, vaccine coverage, and wealth/poverty in the same survey.

The indicator presented here offers clear advantages over those proposed by the WHO.

Conclusion

Improving IYCF practices is an important programme goal. The indicators proposed by the WHO are of limited value in planning, monitoring, and evaluating IYCF programs. A new approach is needed. This article has presented a useful alternative to the indicators proposed by the WHO. More work, such as improving the ICFI scorings algorithm presented in Table 1, is required to finesse the proposed indicators.

For more information, contact: Mark Myatt, mark@brixtonhealth.com

¹WHO, IFPRI, UC Davis, FANTA, USAID, UNICEF, Indicators for assessing infant and young
child feeding practices : Part 1 - Definitions, Geneva, World Health Organization, 2008

²WHO, IFPRI, UC Davis, FANTA, USAID, UNICEF, Indicators for assessing infant and young
child feeding practices : Part 2 - Measurement, Geneva, World Health Organization, 2010

³Arimond M, Ruel MT, Generating Indicators of Appropriate Feeding of Children 6 through 23
months from the KPC 2000+, Washington DC, FANTA/AED, 2003

⁴Arimond M, Ruel MT, Progress in Developing an Infant and Child Feeding Index : An Example
Using the Ethiopia Demographic and Health Survey 2000. Food Consumption and Nutrition
Division Discussion Paper #143, Washington DC, IFPRI, 2002

⁵Ruel, MT, Menon P, Creating a Child Feeding Index Using the Demographic and Health
Surveys: An Example from Latin America. Food and Nutrition Discussion Paper #130,
Washington DC, IFPRI, 2002

⁶Ellie M, Margai M, Kamara, EK, Suale JJ, Kamara J, Kandeh KY, Kamara I, Rogers IA, Kamara
TA, Mansaray S, Bangura A, Sesay MAU, Kamara EI, Mansray A, Mutunga M, Senesie J,
Beckley W, Myatt M, Tools and indicators for monitoring and evaluation of IYCF & CMAM
programs, Sierra Leone Ministry of Health and Sanitation, CAWeC, CARE, UNICEF,
Freetown, Sierra Leone, 2013