Considerations regarding coverage standards for selective feeding programmes
By Ernest Guevarra, Saul Guerrero and Mark Myatt
Ernest Guevarra leads Valid International's coverage assessment team. He has formal training as a physician and a public health practitioner and invaluable informal training as a community worker from the communities with whom he has worked. Most recently, he has worked in Sierra Leone, Niger, Sudan, Ethiopia, and Ghana.
Saul Guerrero is Head of Technical Development at ACF-UK and a founder of the Coverage Monitoring Network. Prior to joining ACF, he worked for Valid International in the research & development of the CTC model. He has supported SAM treatment programmes in over 20 countries.
Mark Myatt is a consultant epidemiologist. His areas of expertise include surveillance of communicable diseases, epidemiology of communicable diseases, nutritional epidemiology, spatial epidemiology, and survey design. He is currently based in the UK.
Introduction
The SPHERE set of standards for the coverage of therapeutic feeding programmes are:
Rural settings > 50%
Urban settings > 70%
Camp settings > 90%
We believe that these standards are simplistic:
No methods for estimating coverage are specified. This is important because there are several methods for estimating coverage. Some of these lack precision, tend to produce biased results, or can yield impossible coverage estimates (i.e. coverage above 100%).
No estimator is specified. It is not clear whether the estimator should or should not include cases in treatment who meet neither programme admission nor discharge criteria (i.e. recovering cases). In some programmes this can strongly influence the coverage estimate. A worst case of this potential source of confusion was observed in a community based management of acute malnutrition (CMAM) programme in Bangladesh. In this programme, coverage calculated using only active severe acute malnutrition (SAM) cases (point coverage) was 0% and coverage calculated using active and recovering SAM cases (period coverage) was almost 90%.
The split into rural, urban, and camp contexts may be too coarse to be meaningful. For example, refugee camps can be very different places from internally displaced persons (IDP) camps in terms of security and access to services. Urban settings are seldom homogenous. Important categories of settlement such as informal peri-urban communities appear to have been overlooked. Peripatetic1 lifestyles / food economies such as transhumant (seasonal) pastoralism have been overlooked.
The context-specific standards appear to have been established without recourse to evidence. Experiences with Community Therapeutic Care (CTC) and CMAM programmes over more than a decade suggest satisfactory levels of coverage are more difficult to achieve in urban settings than the SPHERE standards suggest. A Coverage Monitoring Network review of coverage assessments from 104 CMAM programmes undertaken between April 2003 and March 2013 found that 40% of rural programmes met or exceeded the 50% coverage standard but that no urban or camp programmes met the appropriate coverage standard. For urban contexts, the standards appear to be ambitious. For rural contexts, the standards appear to be unambitious. This is also an argument for improving the way we do urban programming.
No consideration is given to space. It is unclear whether the standard applies to an overall average or is to be achieved everywhere in the programme area.
No consideration is given to time. No guidance is given as to when coverage should be assessed. This assumes that coverage is "switched on" rather than achieved through considerable care and effort. It reflects the "build it and they will come" ethos that has been associated with many recent CMAM coverage failures.
SPHERE simplifies coverage to a single figure. Over a decade of experience with CTC and CMAM programmes shows that coverage is a complicated issue and that considerably more than a single coverage estimate is required to inform and reform programmes to increase coverage, effectiveness, and met need.
In this article, we concentrate on issues of space and time and argue that the SPHERE standards need further definition in order to take these factors into account.
Space
SPHERE standards are unclear as to whether the standard applies to an overall average or is to be achieved everywhere in the programme area. Figure 1 illustrates the problem.
The overall average coverage achieved is 50% but coverage is spatially uneven. It is very high in half of programme site catchment areas and very low in the other half. Nowhere is coverage close to the 50% average. Concentrating on an overall average can lead to poor programme management decisions. In the programme illustrated in Figure 1 we might be tempted to maintain the status quo (i.e. because we appear to have met the coverage standard) rather than focus attention on applying the good practice seen in the successful programme sites to the failing programme sites.
Figure 1 is an extreme example specifically created to illustrate a point but coverage in failing programmes is often very patchy. Figure 2, for example, shows coverage found by a Centric Systematic Area Sampling (CSAS) survey in a CMAM programme in Niger. The average overall coverage in this programme was estimated to be about 18%. Coverage estimates for the separate grid squares ranged from between zero and 80%. Coverage was patchy. A programme in which the overall average coverage is 18% but is not patchy and a programme in which the overall average coverage is 18% and is patchy are both failing programmes, but will probably require very different changes in order to improve coverage. Effective monitoring and evaluation of CMAM programmes therefore requires that overall average coverage results be accompanied by an indication of the patchiness of coverage.
Concentrating on an overall coverage estimate can lead to us making poor programme decisions that allow a situation of poor equity of treatment to evolve or be maintained. Rights-based standards such as SPHERE should not allow this to happen. This means that we should be applying the standard in the sense of it being met everywhere rather than being met as an overall average. One practical implication of this approach is the need for coverage assessment methods that can reveal spatial variation in coverage. Appropriate methods (e.g. SQUEAC, SLEAC, CSAS, and S3M)2 are available. These methods provide mapping of coverage as well as information on coverage bottlenecks needed to inform programme reforms.
Time
The SPHERE standards have no temporal component. There is no specification of how long it should take for the standard to be achieved. They read as if coverage is something that can be ‘switched on’ when, in reality, coverage is something that takes time and effort to achieve.
Figure 3 shows a simple model of how coverage changes over time. If no very poor programme design decisions have been made and proper attention has been paid to community sensitisation and mobilisation, then coverage will increase rapidly until the standard is met or exceeded. The key question is:
How long do we allow before the coverage standard should be met or exceeded?
This is not a simple question. The answer will vary by context. A simple example of contrasting contexts is emergency vs. development settings. In an emergency setting we would want a very short attack phase, measured in days or weeks, and resources will usually be available to achieve this. In a development setting we often find ourselves working in poorly functioning health systems operating with severely constrained resources. In such settings we accept, or are forced to accept, a longer attack phase measured in months or years. The question is also complicated by spatial issues such as the spatial distribution of the population, health facilities, and the prevalence and incidence of acute malnutrition. The utilitarian principal of providing the greatest good for the greatest number will usually apply. This means that we will make an effort to triage communities into those that most require the intervention (high need), those that least require the intervention (low need), and those in between (moderate need). We would then allow different durations of attack phase for each group. For example:
| Need category | Acceptable duration of the attack phase | |
| High | Short | 3 - 6 months |
| Moderate | Moderate | 1 - 2 years |
| Low | Long | 2 - 5 years |
An issue arising from this rational approach to programming (i.e. the most effort for the most cases) is that current mainstream tools for assessing the prevalence of acute malnutrition (e.g. SMART) are capable of presenting only wide area averages with estimators that are incapable of providing usefully precise estimates of SAM prevalence with sample sizes that can be collected at reasonable cost. We urgently need prevalence assessment methods that can reveal spatial variation in the prevalence of moderate acute malnutrition (MAM) and SAM. Methods to do this are currently under development by UNICEF, VALID International, GAIN, and Brixton Health.
The audit cycle: A framework for monitoring and evaluating coverage
In development settings, when we may achieve standards after a long effort, we need a framework that allows us to monitor whether we are on track for meeting standards and what, if any, programme changes are needed. For this we propose an audit cycle (Figure 4).
The audit cycle aims to provide continual and incremental improvements to practice. This means that the standard should be increased once the previous standard has been met. The aim of audit is to approach best practice over a number of audit cycles. Once best practice has been achieved (e.g. in CMAM programmes in rural settings this means coverage levels of 80% or higher), the audit process continues in order to confirm that best practice is being sustained.
Standards in the audit cycle are interim targets. This means that it is legitimate to set an early standard that is below the SPHERE standard because it is a milestone on the path to meeting the SPHERE standard. The SPHERE standard should also be seen as just a milestone on the path to best practice.
Box 1 illustrates the use of the audit cycle as a framework for coverage monitoring.
Box 1: Coverage, coverage limits, and audit
The pattern of coverage presented in Figure 3 may be interpreted as coverage increasing until it meets a limit that is imposed by the barriers and bottlenecks that act to limit coverage (Figure 5).
The process of audit using a coverage assessment technique such as SQUEAC aims to discover and address barriers and bottlenecks in order to improve coverage. The process is ongoing (either periodic, continuous, or a mixture of both) so that progress can be monitored and new barriers and bottlenecks (e.g. due to seasonality or unintended consequences of reform) identified and addressed. Coverage under audit will usually follow a bumpy trajectory (Figure 6).
It is important that audit continues after high coverage is achieved in order to confirm that good practice is being sustained and to identify and address new barriers and bottlenecks.
Conclusion
Standards such as those proposed by SPHERE are not without value. There is room for improvement. There is need for well-defined and nuanced standards. In this article we have proposed:
- Coverage standards be applied everywhere rather than as an overall average that might not apply anywhere.
- Coverage standards include context specific time elements which may be influenced by spatial factors.
- Coverage standards should be part of a monitoring and evaluation framework designed to provide continuous and incremental improvements to programming. We propose the audit cycle for this.
These proposed changes to the current SPHERE coverage standards require assessment tools that can map need and coverage. They also require organisations that have the skills and the will to apply these tools and move the coverage monitoring agenda forward. We already have tools (i.e. SQUEAC, SLEAC, CSAS, S3M surveys and the audit cycle) and organisations (e.g. The Coverage Monitoring Network) to facilitate some of these proposals. Further work is required on developing tools that can map need.
For more information, contact: Saul Guerrero, email:Â s.guerrero@actionagainsthunger.org.uk
1Those who travel from place to place, especially based in places for relatively short periods of time, e.g. nomadic pastoralism, nomadic hunter-gathering, itinerant craftspeople /traders/workers, show people, gypsies, tinkers, travellers, squatters
2SQUEAC: Semi-Quantitative Evaluation of Access and Coverage; SLEAC: Simplified LQAS Evaluation of Access and Coverage; S3M: Simple Spatial Survey Method. See overview at http://www.brixtonhealth.com/updateRAM02.pdf
Imported from FEX website