Why coverage is important: efficacy, effectiveness, coverage, and the impact of CMAM Interventions

By Mark Myatt and Saul Guerrero

Mark Myatt is a consultant epidemiologist. His areas of expertise include surveillance of communicable diseases, epidemiology of communicable diseases, nutritional epidemiology, spatial epidemiology, and survey design. He is currently based in the UK.

Saul Guerrero is the Head of Technical Development at Action Against Hunger (ACFUK). Prior to joining ACF, he worked for Valid International Ltd. in the research, development and roll-out of CTC/CMAM. He has worked in over 20 countries in Africa and Asia.

Introduction

Community-based Management of Acute Malnutrition (CMAM) has reached a crucial point in its evolution. What began as a pilot study just over a decade ago, is now a cornerstone of nutrition policy in over sixty countries. In 2011, for example, CMAM interventions in these countries treated almost two million severely wasted children. As the scale-up of CMAM services continues, it must provide the level of quality that proved so decisive in CTC / CMAM displacing the previous centresbased inpatient treatment paradigm. How should the quality of CMAM services be defined? The importance of coverage has been highlighted but the rationale behind the importance attributed to coverage is seldom explained. This article describes the importance of coverage and the reasons why it should be used to assess the quality of CMAM services.

Efficacy

The efficacy of the CMAM treatment protocol can be defined as how well the CMAM treatment protocol works in ideal and controlled settings. Efficacy is measured by the cure rate:

This is usually estimated in a clinical trial or by observing the cure rate in the set of least severe cases admitted to a CMAM programme and following the CMAM treatment protocol precisely.

The cure rate of the CMAM treatment protocol is close to 100% in uncomplicated incident cases.

Examples of uncomplicated incident cases are:

• Children with MUAC between 110 mm and 114 mm and without medical complications.
• Children with mild nutritional oedema and without medical complications.

The cure rate associated with the CMAM treatment protocol has changed little since it was first proposed. For example, the per-protocol cure rate observed for uncomplicated cases in an early CTC programme in Ethiopia was approximately 94%¹.

Effectiveness

There is little room for large improvements in the efficacy of the CMAM treatment protocol. We cannot significantly change the efficacy of the CMAM treatment protocol but we can change the effectiveness of the CMAM treatment protocol. The effectiveness of the CMAM treatment protocol can be defined as the cure rate observed in an entire beneficiary cohort under programme conditions.

Effectiveness depends, to a large extent, on: Severity of disease: Early treatment seeking and timely case-finding and recruitment of cases will result in a beneficiary cohort in which the majority of cases are uncomplicated incident cases. The cure rate of the CMAM treatment protocol in such a cohort is close to 100%. Late treatment seeking and weak casefinding and recruitment will result in a cohort of more severe and more complicated cases. The cure rate in such a cohort may be much lower than 100%.

Compliance: Programmes in which the beneficiary and the provider adhere strictly to the CMAM treatment protocol have a better cure rate than programmes in which adherence to the CMAM treatment protocol treatment is compromised. Poor compliance can be a problem with the beneficiary (e.g. selling RUTF or sharing RUTF within the household) or a problem with the provider (e.g. RUTF and drug stock-outs) and both have a negative impact on effectiveness.

Defaulting: A defaulter is a beneficiary who was admitted to a programme but who left the programme without being formally discharged. Defaulting early in the treatment episode is the ultimate in poor compliance.

An effective programme must, therefore, have:

Thorough case-finding and early treatment seeking: This ensures that the beneficiary cohort consists mainly of uncomplicated incident cases that can be cured quickly and cheaply using the CMAM treatment protocol.

A high level of compliance by both the beneficiary and the provider: This ensures that the beneficiary receives a treatment of proven efficacy.

Good retention from admission to cure (i.e. little or no defaulting): This also ensures that the beneficiary receives a treatment of proven efficacy.

Impact and coverage

Meeting need (also known as impact) requires both high effectiveness and high coverage:

Impact = Effectiveness x Coverage

In the Tanahashi model of coverage, impact (SERVICE ACHIEVEMENT) is:

SERVICEACHIEVMENT = CONTACTCOVERAGE X EFFECTIVEENESS

Coverage can be expressed as:

Coverage depends directly on:

Thorough case-finding and early treatment seeking: A case that is not admitted into the programme is a non-covered case. Late admissions are coverage failures because they will have been non-covered cases for a considerable period of time before admission.

Good retention from admission to cure: This is the absence of defaulting. Defaulters are children that have been admitted to the programme but leave the programme without being formally discharged, without being transferred to another service, or without having died. Defaulters are, therefore, children that should be in the programme but are not in the programme. This means that high defaulting rates are associated with low programme coverage.

Effectiveness at end of January 2013 was estimated to be 92%.

Coverage at end of January was estimated (using SQUEAC) to be 63%.

Impact can be estimated as:

Impact = Effectiveness x Coverage = 92% x 63% = 58%

Data courtesy of ACF (Pakistan)

Coverage also depends indirectly on:

Thorough case-finding and early treatment seeking: This ensures that the majority of admissions are uncomplicated incident cases, which leads to good outcomes (Figure 1). Late admission is associated with the need for inpatient care, longer treatment, defaulting, and poor treatment outcomes (e.g. nonresponse after long stays in programme or death). These can lead to poor opinions of the programme circulating in the host population, which may lead to more late presentations and admissions and a cycle of negative feedback may develop (Figure 2).

A high level of compliance by both the beneficiary and the provider: This ensures that the beneficiary receives a treatment of proven efficacy leading to good outcomes and good opinions of the programme (Figure 1).

Good retention from admission to cure (i.e., little or no defaulting): This also ensures that the beneficiary receives a treatment of proven efficacy leading to good outcomes and good opinions of the programme (Figure 1).

Coverage and effectiveness depend on the same things and are linked to each other:

Good coverage supports good effectiveness. Good effectiveness supports good coverage. Maximizing coverage maximises effectiveness and met need.

The implications of:

Impact = Effectiveness x Coverage

are illustrated in Figure 3 and Figure 4. Programmes with low coverage fail to meet need (i.e. have limited impact). Programmes that seek to deliver a high impact can only do so by achieving high levels of coverage.

The key measure of programme quality is impact:

Impact = Effectiveness x Coverage

This means that monitoring and evaluation (M&E) activities in CMAM programmes should concentrate on measuring both effectiveness and coverage. Effectiveness can be measured using a simple intention to treat analysis of programme exits (Figure 5). Over the past decade a number of low-resource methods capable of evaluating programme coverage, identifying barriers to service access and uptake, and identifying appropriate actions for improving access and programme coverage have been developed and tested. The Coverage Monitoring Network (CMN) has been established to assist non-governmental organisations (NGOs), United Nations (UN) agencies, and governments use these methods to help maximise the impact of CMAM programmes.

For more information, contact: Saul Guerrero, email: s.guerrero@actionagainsthunger.org.uk

¹The data for this result are taken from Table 3 (page 14) of: Collins S, Community-based therapeutic care. A new paradigm for selective feeding in nutritional crises, HPN, London, Volume 48, November 2004 which shows 440 cases discharged as cured with five deaths in cases admitted without complications or prior hospitalisation. Half of the 49 cases reported as non-recovered after four months in OTP were assumed to be uncomplicated cases. Transfers to hospital or stabilisation centre (93 cases) were classified as complicated cases. This is a per-protocol analysis and excludes defaulters (57 cases). It should be noted that this programme admitted children with MUAC < 110 mm.