The Challenge of Applying CSAS in DRC

By Jason Stobbs, AAH-USA

Jason Stobbs is a long-serving member of Action Against Hunger - USA, volunteering over the past four years in diverse contexts such as Afghanistan, Cambodia, DRC and Chad. A Cambridge Master's graduate in Anthropology and specialist in food security and nutrition issues, he has specifically worked on the development of survey methods in situations of food scarcity.

The contributions of Action Against Hunger - USA and Mark Myatt to this article are gratefully acknowledged.

Some happy faces met along the way

Programme coverage has become an increasingly important indicator for measuring the success of humanitarian interventions. This is especially true in the context of nutritional feeding programmes, where lack of information regarding coverage may seriously dampen the impact of therapeutic and supplementary feeding centres. How coverage is measured is particularly relevant. Practical solutions to the methodological problems of estimating coverage using the 'traditional' two-stage cluster approach (currently considered standard by SPHERE) have been offered by the more recently developed centric systematic area sampling methodology (CSAS)¹. Whilst applying this method in the field, Action Against Hunger-USA (AAH-USA) encountered a number of issues in the sampling and in on-the-ground assessment techniques. This article seeks not to expound on the coverage results of the survey, but rather to act as a case study of the concrete application of the methodology in the field. This article details both the problems and successes encountered when employing the CSAS methodology, and offers some suggestions for further fine-tuning.

Figure 1 Quadrat grid positioned on programme target zone

Figure 1 shows the positioning of the grid on the map of Uvira territory. Note that this 'computerised' image is only an approximate rendering of the physical map used during this stage of the survey. Minor cartographic variations between the two maps have been noted.

Notice the topography. A single axis of communication traverses the territory north south, via quadrats 2, 4, 8, 12, 17, 21, 25 and 29. The main town of Uvira is located in quadrat 25. Finally, it is important to know that Uvira is a 'shelved' territory, composed of highlands (5, 6, 9, 10, 14, 15, 19, 23, 26, 27) of middlelands (1, 7, 11, 16, 20, 24, 28) and of plains (2, 8, 12, 13, 17, 18, 21, 22, 25, 29).

You will also notice that where the intervention area corresponds to less than 50% of the area of a particular block, they have been coloured red and have been excluded from the survey. As a result, the territory has been divided into 29 blocks of 133 km_ each.

Methodology

A mother with her malnourished child identified during the survey

In January 2005, AAH-USA conducted a nutritional coverage survey using CSAS in the Uvira territory located in eastern Democratic Republic of Congo (DRC) in order to assess the level of coverage of the ten nutritional centres located there. The CSAS was particularly chosen as the twostage cluster methodology had previously failed in this territory to give a representative analysis of the malnourished population, where survey results did not concur with the nutritional infrastructure reports.

As per a traditional nutritional coverage survey, the CSAS methodology targets children from 6 to 59 months of age and records weight, height, age, gender, mid-upper arm circumference (MUAC), presence/absence of bilateral oedema and enrolment/ nonenrolment in a nutritional programme. The method can be summarised as a series of steps as follows²:

1. First, a correct and accurate map of the target zone for the programme is necessary (in this case, the entire Uvira territory).
2. A grid is positioned on this map. This grid must be composed of squares of equal size (quadrats), with the number of quadrats varying according to the size and shape of the programme target zone. The objective is to divide the programme target zone into quadrats of equal size. Each quadrat must be spatially homogeneous, that is, the geographical space covered by the quadrat must be equally accessible to the population within. In addition, the grid is positioned inclusively, that is to say that geographical areas are included if more than 50% of their surface area is within the quadrat.
3. When the grid is positioned, the geo graphic centre of each quadrant is determined and an exhaustive list of the villages present is compiled.
4. Each village is then classified in order of priority: the 1st is the village closest to the centre of its quadrat, the 2nd is the closest to the 1st, and so on.
5. The survey is conducted on all of the quadrats, and a team surveys each square for one entire day (from morning to evening).
6. An active case finding method is employed in CSAS. Community health workers and/or local mothers' groups are the best sources of case information. When in the village or villages designated by the methodology, the team asks one or several representatives of the population to take them to all children aged between 6 and 59 months who are:

• malnourished and/or ill. Because there are close links between malnutrition and illness, it is often difficult to tell at a glance whether a child is only ill or whether s/he is also malnourished. In order to obtain an exhaustive sample, every case is measured, and children who are only ill are separated from those who are malnourished during the post-survey analysis.
• enrolled in a nutritional programme, even if they no longer show clinical signs of malnutrition. After the second month spent in a Supplemental Feeding Centre, a malnourished child no longer falls with in the admission criteria, but must still be included in the coverage survey.

An aid worker measuring MUAC during the survey

Anthropometric measurements of children identified as malnourished by the community are then taken. For each case measured, the team asks whether the child is participating in a nutritional programme. When a child claims to be a participant in a programme, the child's name and all relevant information are taken in order to crosscheck whether the child is actually enrolled. If a child who is allegedly malnourished is not at home, the team proceeds to the next house, but returns later. This procedure is continued until there are no more cases known by the community.

When an entire village has been visited, the team goes to the next village on the list, and the next, repeating this process until the end of the day. The number of children measured in a single day is considered sufficient to constitute a representative sample.

7. When all of the quadrats have been surveyed, the information is compiled and analysed. The calculation of the formula for the coverage rate is the same as with a two-stage cluster method, but the results are much more representative:

AAH-USA coverage survey

The AAH-USA nutrition survey took place between January 21st and January 28th 2005, and covered the entire territory of Uvira (including the highlands and middle-lands). Prior to this, two weeks were spent training the five survey teams, which included using the cartographical data to define the quadrats and assigning the itinerary to the teams. The teams slept in the field during the survey period and did not return to base camp at night.

Each team comprised three members (a team leader and two measurers). Each was assigned a number of quadrats, depending on the distance to be travelled and the ease of access. For example, a team assigned to the plains along the axis of communication was assigned five quadrats, while a team assigned to the most remote corners of the highlands was only assigned two. The team leaders were required to act autonomously and gauge, according to their own judgment and the information they obtained locally, the level of threat ahead (the Uvira territory is not a stable area) and the itinerary to employ (whilst still proceeding to villages according to the centric pattern and making way towards the quadrat to be surveyed the next day). Needless to say, such responsibilities required that the teams be thoroughly trained, and relied upon a high degree of initiative and responsibility from the team leaders.

Discussion

'Spatial homogeneity' assumption

The sampling methodology in a systematic centric area is frequently utilised in ecological studies to determine spatial distribution of animal or vegetable species over a vast surface, as well as in human geography to evaluate the radius of distribution of certain points of sale. Its main advantages are its ease of utilisation, its capacity to derive a representative cross-section for a vast area, the simplicity of the analysis of the data, and the spatial dimension of the analysis which gives it an added value. However, it is built on the hypothesis that each quadrat is spatially homogenous. During our field survey, this assumption clashed with the reality of the area's topography. Uvira is a 'shelved' territory, varying from highlands to middle-lands to plains, as one moves from west to east. Despite experimenting with various quadrat sizes and grid positionings during step 2, no fully satisfactory positioning was found in order to ensure spatial homogeneity. This in turn led to troubling results in the survey, as illustrated by the case of quadrat n°25 (see box). The survey information revealed a striking difference between the middle-lands and the plains in terms of population mobility. However, in following the CSAS methodology, the level of coverage for quadrat n°25 was calculated to be 0%, a clearly misleading figure given the number of beneficiaries in the centres coming from other villages located in the plains.

Although this problem can clearly be attributed to a lack of spatial homogeneity within the quadrat, a viable solution is hard to come by. Indeed, with the lake Tanganyika and the middle- lands barely 4km apart from one another, and the town of Uvira nestled in between, it is difficult to imagine how to reposition the grid in a more suitable way. This difficulty in ensuring spatial homogeneity within the quadrats is symptomatic of the entire area of Uvira, where the presence of a sole axis of communication and of numerous hills and rivers poses certain difficulties in the positioning of the grid. It might be possible to use smaller quadrats, which would have the added benefit of allowing post-survey stratification, However, given that each quadrat takes a day to survey, the ensuing increase in personnel and/or time needed for completing the survey could make the cost prohibitive. Alternatively, the original quadrat size could be kept but a random sample of villages within each quadrat could be used. Another possiblity would be to locate quadrats purposively so as to ensure that plains and mountains were included.

Although this is a case of the method 'failing', it is not a gross failure as it provided useful information. One of the aims of a coverage survey is to identify where coverage is likely to be weak. In this survey, quadrat n°25 gave us good insight into a coverage problem linked to the cultural resistance of the hill folk to bring their children to a centre in the plains, regardless of proximity. This demonstrates how, as in all surveys, this method must be accompanied by a contextual interpretation.

Sampling sizes and active case research

The active case-finding method has the advantage of providing a nearly-exhaustive cross-section of malnutrition cases in the villages surveyed. In addition, it allows for coverage of more villages than in a classic survey. For example, in the AAH-USA survey, 112 villages were visited instead of an average of 30 covered in a classic survey. This guarantees greater accuracy and greater representativeness in the sampling.

In areas of low-density population, where the villages become more distant from one another, the size of the cross-section is very small (e,g. quadrat n°15, where the villages are so far apart from one another that only two villages could be surveyed). Although this is essentially not a problem, insofar as the results reflect the geographic and nutritional realities of the area, a small sampling renders comparison between the areas difficult, if not impossible. For instance, how does one compare the quadrat n°15 with quadrat n°14, where 11 villages were surveyed and 3 cases of malnutrition were confirmed? The exercise is simply impossible, due to the discrepancy in sample size. To diminish the size and to increase the number of quadrats seems, once again, to be the best way to refine the results and avoid this pitfall.

The quality of the cross-section is built entirely on the capacity of health networks or other community representatives to detect malnutrition cases. The voluntary exclusion of malnutrition cases, whether for reasons of discrimination or for taboos surrounding malnutrition, is therefore a reality that must be taken into account during the sampling. This being the case, it is regrettable that no pre-testing was undertaken during this survey. Such a pre-testing would have ensured that the definition of the terms used to define malnutrition cases in the communities was homogeneous and allowed us to compare the number of cases registered with the help of the community with the number of cases registered at the time of an exhaustive screening of the community. However, it appears that the casefinding employed was reasonably sensitive since, prior to their measurement and confirmation of their anthropometric status, many children were put forward who turned out to be 'false-positives' (i.e. children identified as malnourished by the population but not falling within malnutrition criteria). Indeed the final sample size was much smaller than anticipated - out of 858 children measured, 511 were excluded from the final analysis as false positives.

Conclusions

In retrospect, there are a number of pros and cons to using the CSAS methodology in this context. On the positive side, the input and handling of data is considerably more rapid than in a 'classical' two-stage cluster survey. Furthermore, since the cross-section is quite a lot bigger than would have been in a traditional survey, 262 malnutrition cases were detected - considerably more than the 30 cases detected in a previous cluster survey carried out in June 2004. The spatial dimension of the analysis also provided useful information for the orientation of AAH-USA's programmes. For instance, quadrat n°11 turned out to have over 20 cases of malnutrition without any coverage support. This led to the opening of a nutritional infrastructure and the improvement of screening and consciousness-raising activities. Thus, in this particular instance, the coverage survey doubled as a case-finding and referral activity for the programme.

Measuring length during the survey

However, the cost, time and work intensity is considerably greater than that of a classical two-stage cluster method. In the traditional two-stage cluster methodology, a cluster is finished once 30 children have been sampled, whereas in the case of the CSAS methodology, a quadrat takes an entire day to complete, with as many children measured as possible. This demands great effort and planning, especially if villages within a given quadrat are not close to each other. Furthermore, the need to keep quadrats spatially homogeneous entails, in this case, a reduction in size and an increase in the number of quadrats, which in turn increases the cost and length of the survey. Similarly, the disparity in quadrat sample sizes makes for difficult comparisons, unless the size of the quadrats is reduced. Finally, the reliance on village health workers or key informants, although a clear positive in terms of programme integration and sustainability, does in turn create a need for pre-testing in every village, increasing, once again, the time and cost of the activity.

In conclusion, this method has a clear usefulness in the framework of an evaluation of nutritional coverage. However, it requires a high degree of preparation and a thorough knowledge of the area.

For further information, contact Jason Stobbs, email: Jasonstobbs@yahoo.com and Marie-Sophie Simon, AAH-USA, email: mss@aah-usa.org

View the postscript that this article relates to

¹Developed by Mark Myatt, researcher at the Institute of Ophthalmology of London and a consultant for the WHO.

²For a description of the CSAS method, see Field Exchange issue 21, New Method for Estimating Programme Coverage, p11 and 'A Field Trial of a Survey Method for Estimating the Coverage of Selective Feeding Programmes', Mark Mayatt, Teshome Feleke, Kate Sadlet, Steve Collins, Bulletin of the World Health Organization, January 2005, n 83 (1).