Participatory, decentralised monitoring to improve quality of CMAM services in Sudan

Location: Sudan

What we know: Community-based management of acute malnutrition (CMAM) programming requires a continuous cycle of monitoring, feedback and action to improve quality.

What this article adds: In Sudan UNICEF and the Ministry of Health (MoH) carry out nation-wide participatory, decentralised monitoring to track outpatient therapeutic programme (OTP) progress and improve coverage and quality. This involves fives steps: defining state annual and quarterly targets; systematic data review and identification of low-performing states; engagement of implementing partners to identify low-performing CMAM sites and bottlenecks; development of joint action plans; and monthly review of progress. A case study is provided of Red Sea State, which showed slow progress on achieving targets in 2017 (by July 2017 only 6,837 children with severe acute malnutrition were admitted, 39% of the state annual target). OTPs were classified according to agreed criteria (‘high’ 8-12 children/month; ‘average’ 4-8 children/ month; low <4 children per month), resulting in identification of 27 low-performing sites. Identified bottlenecks included inadequate staff; no staff incentives; low level of education/understanding of treatment protocol; no case-finding; low community acceptance; and scattered population. A resulting action plan included quarterly monitoring/mentoring by MoH/UNICEF at the 27 sites; monthly local monitoring meetings; screening by mothers and mother support groups of SAM children; and community mobilisation through community leaders. By the end of December 2017, 14,280 SAM children were admitted in Red Sea state, showing considerable improvement. 

Background

High levels of malnutrition were reported in the 2013 S3M (simple spatial surveying method) survey. This indicated the need for the scale-up of the community-based management of acute malnutrition (CMAM) programme, guided by a national scale-up plan (2015-2017) (Tewoldeberhan et al, 2017). Currently there are 1,110 outpatient therapeutic programmes (OTPs) and 125 stabilisation centres (SCs) in Sudan, which have treated nearly a quarter of million children annually since 2016.

Designing and implementing interventions in Sudan requires consideration of multiple, diverse and sometimes complex contexts, including: conflict-affected areas with limited access; population displacement; influxes of refugees (mainly from neighboring South Sudan); camp and host community settings; returning populations; scattered settlements; nomadic settlements; peri-urban settings; and indigenous populations in mountainous areas. Such diverse and challenging contexts provide challenges for CMAM programming, affecting access, coverage and quality of the programme. There is a need in this context for multi-layer monitoring to track progress of CMAM programmes, identify bottlenecks and provide timely, corrective actions.

Participatory, decentralised monitoring methodology

UNICEF collaborates with the Ministry of Health (MoH) and non-governmental organisations (NGOs) at state and federal levels to carry out decentralised monitoring to track progress towards improved coverage and quality of the CMAM programme. The approach uses an inclusive, bottom-up approach, with strong participation of the state-level MoH and all partners working in the state, irrespective of whether they are UNICEF programme partners or not. Based on analysis of routine data, best practices for well performing facilities are noted and in poor performing facilities bottlenecks are identified and action plans developed with concrete activities to be implemented immediately for rapid improvement. The approach focuses on quick wins to get more children into the programme while systematically putting in place measures to address more resource-intensive bottlenecks.

This approach has yielded good results in the past two years, rapidly increasing admissions in previously poor performing CMAM sites and better positioning states to reach national admission targets. The approach has been used in seven states, including East Darfur, Central Darfur, South Darfur, Red Sea, South Kordofan, West Kordofan and North Kordofan, all of which were identified as low-performing and far off their annual targets. After implementing the approach, rapid catch-up was recorded.

Participatory, decentralised monitoring consist of five key steps: defining annual and quarterly targets for each state; tracking progress through systematic data review and identification of low-performing states; engagement of implementing partners in the low-performing states on a deep dive analysis to identify low-performing CMAM sites and bottlenecks causing low performance; development of joint action plans targeting the bottlenecks; and monthly review of progress after implementation of action plans (Figure 1). Each of these steps is described below.

 Figure 1: Five steps in participatory, decentralised monitoring of CMAM 

Step 1: Setting of annual state target

Since 2016 UNICEF has committed to treating 250,000 children suffering SAM in Sudan annually, representing half the total burden. Each state has an agreed target that contributes to achieving the national target, based on the prevalence of malnutrition in that state. Annual state targets are further broken down into four quarters, based on the seasonal admission trends over the past three years. Some states have further broken their quarterly targets into monthly targets for admission. Such annual and quarterly targets allow the tracking of results and facilitate planning, including allocation of financial resources and supplies.

Step 2: Tracking progress and identifying low-performing states

Each month, each state collates and submits admission and performance data to the Federal Ministry of Health (FMoH), through which UNICEF receives summary data per state. On a quarterly basis, UNICEF and FMoH hold a joint review and sort states into ‘no risk’, ‘low risk’ and ‘high risk’, based on whether they have achieved their set quarterly admissions target and performance indicators for quality. FMoH also shares a list of low-performing localities with UNICEF. States that are deemed high risk are prioritised for participatory, decentralised monitoring. Feedback is shared with all states and implementing partners through UNICEF Field Offices.

Step 3: Participatory analysis of performance by centre and identification of bottlenecks

High-risk states first undergo a deep-dive analysis of performance and bottlenecks. UNICEF engages with each state MoH to plan for the participatory analysis, including agreement on timelines and mobilisation of participants, often including local-level nutritionists from both MoH and NGOs and state-level coordinators and managers. This is a rapid exercise, lasting not longer than one and a half days. 

A decentralised analysis is performed in collaboration with the FMoH (which maintains the CMAM database) on trends of admission, cure, defaulter, death and non-respondent rates. This initial analysis shows the landscape of programme performance at state level, indicating which localities are reporting the highest or lowest admissions, which partners are performing relatively better and which months have the least admissions. Sphere standard indicators are also reviewed to give more context to the analysis.

Following this, a range of partners are engaged to develop context-specific benchmarks from which OTPs can be classified as having high or very high admissions, average admissions, or low or very low admissions, with links to state annual targets. This is a deviation from the previous simple approach of dividing the total annual target by total number of facilities, under which the vast majority of OTPs are likely to be classified as low-performing. Engagement with partners is a key part of this process; partners do not want their projects to be classified as low or average-performing, so discussions are often prolonged until multiple partners can reach agreement. The result is a minimum standard of performance for OTPs within the state.

Given that many OTPs function at minimum-acceptable standards, those that exceed these will compensate for facilities that fail to improve for genuine reasons. Due to different state characteristics – such as state targets, demography and number of OTPs – the expected number of admissions per OTP varies from state to state. OTPs classified as low-performing in very high-burden states may fall among the best-performing in low-burden states. Factors such as population movement and other demographic dynamics also impact on admissions. The benefit of this process is the uncovering of OTPs with low admissions (often those in remote areas) that would otherwise have been masked by high admissions in better-performing OTPs in those states showing good overall CMAM performance.

Following the admission analysis and classification of centres, a qualitative analysis is carried out to understand facilitators of high performance in facilities that perform very well and bottlenecks in facilities that do not. Local-level programme officers are invited to participate in this process, given their knowledge of the centres and valuable insights into the CMAM programme.

Step 4: Development of joint action plans

For each low-performing OTP, a plan of action is needed to improve performance and accelerate progress. These action plans are based on bottlenecks and facilitators identified and normally consist of activities that can be rapidly implemented within resource budgets and local capacities. Long-term actions and actions that require additional resources, such as staffing or scale-up of services, are addressed in a progressive manner alongside these more short-term, rapid solutions, which provide initial motivation and quick results. The absence of such easier actions has often been the main cause of poor performance in the past.

Implementation often starts the week after the exercise to maintain momentum among implementing partners and state MoH staff. Where additional resources become necessary, UNICEF provides direct technical and financial support. Action plans are calibrated at local level to the extent possible so that as many centres as possible can benefit, whether by regular monthly feedback meetings among OTP staff, on the job training and mentoring, locality-wide mid-upper arm circumference (MUAC) screening, provision of drugs, redeployment of staff, or provision of other services (such as expanded programme on immunization (EPI), integrated management of childhood illness (IMCI) or reproductive health (RH)). Localities with a higher number of low-performing OTPs are prioritised over those with just one or two as it is more practical to address five low-performing OTPs in one locality than address five scattered in different localities.

Understaffing is a common longer-term bottleneck identified as a factor for low performance in OTPs that is not a simple problem to solve. In the past UNICEF has discussed the issue with state and federal MoH authorities and, while there have been instances where the FMoH national programme has deployed surge support to low-performing localities (such as in refugee response situations), or has temporarily redeployed staff from other localities or federal level, in many cases the issue remains. Poor access to remote populations is another common barrier identified. Where this is temporary, such as due to moving of communities to farmland areas, it is addressed through satellite and mobile clinics run by NGOs and sometimes with state MoHs (funded by UNICEF or FMoH when combined with EPI).

Step 5: Follow up

After initiating implementation of the action plan, low-performing centres and localities are closely monitored monthly to track progress. This is important to ensure that actions are taken in a timely and effective way.

Case study of Red Sea State

Red Sea State is in the east of Sudan, bordering the Red Sea, Eritrea and Egypt. The capital, Port Sudan, is a commercial city and the only sea port in Sudan. The MICS 2014 reported global acute malnutrition (GAM) and SAM as 14% and 2.3% respectively. The planning of SAM services in the state, as in all other states across Sudan, is based on the 2013 S3M report, which reported 20% GAM and 7.5% SAM for Red Sea. The National Baseline Household Survey 2009 also reported more than half of the state population as poor.

Under the leadership of the state MoH, CMAM has been successfully scaled up with limited resources and has achieved over 60% coverage, as reported by two successive coverage surveys in selected localities. There are currently 129 OTPs distributed across all nine localities in the state, treating more than 14,000 children annually since 2016.

By end of the first half of 2017, Red Sea was among the states identified to have difficulty in achieving annual targets. As a result, UNICEF supported state capacity through decentralised analysis, identification of low-performing sites and localities, bottleneck analysis and development of action plan for progress acceleration. The state MoH was fully engaged, providing access to the state CMAM database for analysis and coordinating and actively participating in the entire process. The steps in this process were:

Step 1: Setting of annual state target

In 2017, 17,490 children suffering from SAM were targeted for treatment through the CMAM programme in Red Sea State. This target was set by the state MoH and partners and supported by UNICEF, based on SAM prevalence from the 2013 S3M.

Step 2: Tracking progress and identifying low-performing states

Progress towards the target was quite slow in 2017; by the end of July 2017, only 6,837 were admitted, representing only 39% of the annual target. The coverage was also 777 fewer children than the previous year. Cure rates, defaulter rates and death rates were within SPHERE standards. UNICEF’s technical team supported the nutrition team of the MoH with decentralised monitoring to accelerate progress.

Step 3: Centre-level joint analysis of performance and bottlenecks

UNICEF, FMoH and local staff discussed and agreed on criteria for classification of OTP performance as high, average or low (see Table 1). This was informed by the total admissions for the state at the time, total number of OTPs, demographic factors (nomadic activity and seasonal migration from heat waves) and admission trends in previous years. Using this benchmark, UNICEF and MoH monitoring teams jointly analysed CMAM data.

Table 1: Classification of OTP performance in Red Sea, July 2017

Pivot tables (see Figure 2) were used to present performance and trends per CMAM site, locality and implementing partner. On flip charts, OTPs were grouped by performance according to the agreed criterion. For each OTP, participants who were familiar with the site gave detailed answers to the question, “Why is the OTP performance at the current level?”. This qualitative analysis provided thorough insight to all OTPs on issues such as staffing, local government support, supplies shortages, community acceptance and capacity of primary healthcare. These discussions were facilitated by interactive tools such as visualisation in participatory planning (VIPP) cards or StickIt to bring out the real strengths and bottlenecks of OTPs, as opposed to programme reviews that provide generalisations. Summary characteristics of high- and low-performing sites in the Red Sea are provided in Box 1.

Figure 2: Example of admission trends for Karary site, Nyala South, Jan-Sept 2017

Step 4: Development of joint action plan

Box 1: Summary characteristics of high- and low-performing centres in Red Sea

High-performing centres

1. Regular screening at community level
2. Highly experienced and committed staff
3. High acceptance by community members
4. Provision of integrated PHC services (than standalone OTPs)

Low-performing centres

1. Inadequately staffed
2. Centres run by volunteers without incentives
3. Low level of education and understanding of treatment protocol, in spite of several training sessions attended
4. Cadres seeking income from other activities, such as the port.
5. There is no form of case-finding
6. Low community acceptance
7. Scattered population and mountainous terrain
8. Frequent movement of population to other locations for agriculture and other income-generating activities
9. Irregular functioning of mobile OTPs (UNICEF fund only four localities)

 

From the deep-dive performance analysis and identification of facilitating factors and bottlenecks, an accelerated action plan was developed, tailormade to accelerate the performance of low-performing OTPs in the Red Sea State (see Box 2). 

Step 5: Follow-up review of progress

The action plan was implemented immediately by the state MoH nutrition team with support from the UNICEF team based in the Red Sea. The UNICEF Country Office in Khartoum provided regular follow-up to track progress in the implementation of the action plan. This follow-up support has proven critical to ensure that plans are not shelved but are implemented in a timely and systematic manner. Many of the actions have been institutionalised and implementation continued beyond 2017, including monthly local CMAM meetings, joint quarterly monitoring, on-the-job training and quarterly community mobilisation sessions. 

Box 2: Actions to improve performance of identified poor-performing  

1) MoH/UNICEF to conduct joint quarterly monitoring/mentoring (and on-the-job training) visits to the 27 high-need OTPs in Sinkat, Haya, Ageeg, Haleib, Jabetalmaadin and Sawakin localities.

2) Organise monthly locality-level meeting for staffs of 27 high-need OTPs in the six localities to provide closer review of progress.

3) Screening by mothers of SAM children

• Distribute MUAC tapes to all OTPs and facilitate provision to all attending mothers and orient them on how to take MUAC.
• Conduct weekly orientation to mothers of SAM children on how to measure MUAC and encourage them to screen children in their neighbourhood.

4) Screening by mother support groups (MSGs)

• Distribute MUAC tapes to MSGs and conduct on-the-job training/refresher on MUAC screening.
• MoH to encourage MSGs to conduct MUAC screening in their neighbourhood and refer those with SAM to the OTPs.

5) Community mobilisation through key community leaders

1. Conduct quarterly community dialogue/sensitisation (nadwa) in OTP catchments targeting 25 key community comprising:

a. 3 OTP staff

b. 2 Sheikhs

c. 5 Shabab

d. 2 Women Union

e. 5 mothers of SAM children

f. 5 Community Volunteers

g. 1 Traditional healers/drug peddlers/tea sellers

h. 1 Local authority

i. 1 Imam

6) State MoH to prepare laminated letters of appreciation to OTPs that are performing very well in recognition of a job well done and to encourage them to continue good practices.