New Method for Estimating Programme Coverage

By Mark Myatt

Mark Myatt is a consultant epidemiologist and senior research fellow at the Division of Epidemiology, Institute of Opthalmology, University College London. His areas of expertise include infectious disease, nutrition, and survey design.

This article gives an overview of the coverage estimation method developed for the Community Therapeutic Care (CTC) Research Programme in Malawi¹.

Coverage is becoming an important indicator for monitoring and evaluating humanitarian interventions. Coverage indicators for selective feeding programmes were included in the SPHERE project's humanitarian guidelines for the first time in 2003². Current approaches to estimating therapeutic feeding programme coverage rely on the use of nutritional anthropometry surveys that commonly employ a two-stage cluster sampling strategy. Such surveys will be familiar to many humanitarian practitioners as 'thirty-by-thirty' surveys. Coverage is calculated either directly using survey data or indirectly using survey data, programme enrolment data, and population estimates. Both methods assume that coverage is similar throughout the entire survey area and both can provide only a single wide-area coverage estimate.

The two-stage cluster-sampling approach uses population proportional sampling (PPS) in the first stage to select cluster locations and proximity sampling in the second stage to select households and children.

The PPS approach is unsatisfactory because:

• The bulk of data are collected from the most populous communities. This may leave areas of low population-density (i.e. those areas consisting of communities likely to be distant from health facilities, feeding centres, and distribution points) unrepresented in the sample. This may cause surveys to evaluate coverage as being adequate even when coverage is poor or non-existent in areas out side of urban centres.
• There is no guarantee of an even spatial sampling. This is true even when the population of the survey area is evenly distributed. Again, PPS will usually leave some areas unrepresented in the sample.
• It relies on population estimates which may be inaccurate in emergency contexts, particularly if population displacement, migration, or high mortality has occurred in the target population.

The proximity sampling approach is unsatisfactory because:

• The proximity method is unlikely to return a representative sample at the cluster level. It is not possible to estimate coverage reliably for a cluster without taking a representative sample from the cluster location.
• Even if a representative sample were taken at the cluster level, the withincluster sample size is too small to esti mate coverage at the cluster level with reasonable precision. For example:

The sample size available to estimate coverage at each cluster location is likely to be only one or two cases.

These problems are not important if the assumption that coverage is similar throughout the entire survey area is true. Such as assumption is, however, unlikely to be true of many centre-based programmes or during the start-up phase of a programme, where coverage is likely to be better in the areas closest to feeding centres. If coverage is uneven, the ability to identify areas with poor coverage is an essential first step towards improving program coverage and, hence, program impact. Current methods do not provide this.

The method outlined here allows an estimation of coverage in the usual way, but also allows identification of areas with poor coverage within a programme area.

Step 1 : Find a Map

The first step in the new coverage estimation method is to find a map of the program area. Try to find a map showing the location of town and villages. A map of 1:50,000 scale is ideal.

Step 2 : Draw a grid

The next step is to draw a grid over the map. The size of each square should be small enough for it to be reasonable to assume that coverage will be similar throughout the square. A square of 10km by 10km will probably be small enough in most circumstances.

Step 3 : Select the squares to sample

Select the squares with about 50% or more of their area inside the program area.

Step 4 : Select the communities to sample

Select the community closest to the centre of each square. If prevalence is low then you might need to select more than one community from each square. Select the communities closest to the centre of each square. Select the communities to be sampled and the order in which they should be sampled in advance of visiting the square.

Step 5 : Case-finding

When you visit a community find cases using an active case-finding method. It is usually sufficient to ask community health workers, traditional birth attendants, traditional healers or other key informants to take you to see "children who are sick, thin, or have swollen legs or feet" and then ask mothers of confirmed cases to help you find more cases. You could also use door-to-door screening. It is important that the case-finding method that you use finds all, or nearly all, cases in the sampled communities. Each case is confirmed by applying the program's entry criteria (e.g. < 70% of the median weight-for-height from the NCHS reference population and /or bilateral pitting oedema). When you find a confirmed case you should then find out whether that child is in the programme. Remember to follow-up on children reported to be in a therapeutic feeding centre or at a distribution point on the day of the survey.

Step 6 : Record the data

You need only record the number of cases found and the number of those that are in the program for each sampled square.

Step 7 : Calculate coverage

Coverage is calculated as the number of cases that you found that are in the program divided by the number of cases that you found. You should calculate this separately for each square as well as for all of the squares together. You could use a weighted method to calculate the coverage for all of the squares combined but this is not essential.

Step 8 : Plot the data

Coverage data is plotted as a mesh map and as a histogram. The length of the sides of the filled squares on the mesh map reflects the level of coverage found in each square (calculated in step 7). The small open squares indicate quadrats (squares of approximately equal area) with zero coverage. You might find it helpful to mark the location of feeding centres, distribution points, health posts, and roads on the map. This will help you interpret the results of the coverage survey.

For further information on this method, contact Mark Myatt, email: mark@brixtonhealth.com

¹Coverage method presented at CTC meeting in Dublin, 8-10th October, 2003. See this issue for meeting summary.

²http://www.sphereproject.org/handbook/index.htm

³A number of formulae are used for direct coverage estimation. The simplest and most intuitive variant is shown here.