Development of multi-cluster rapid and in-depth assessment methodologies in Afghanistan
By Anna Ziolkovska, Hassan Ali and Baseer Qureshi
Anna Ziolkovska is currently the Nutrition Cluster Coordinator in Yemen. Anna has a PhD and over ten years’ experience in nutrition, humanitarian assistance, cluster coordination, information management, capacity building and project management at HQ and country levels, including in South Sudan, Somalia, Philippines, Ukraine, Afghanistan and Yemen.
Hassan Ali is a nutrition survey and surveillance expert with over three years’ experience in SMART methodology in Kenya, Somalia, Afghanistan and Yemen, working with Ministries of Health and Nutrition Clusters.
Baseer Qureshi is a medical doctor (MD, MPH, MBA) working with the public nutrition directorate (PND) of the Ministry of Public Health (MoPH) in Afghanistan as a national nutrition surveillance consultant.
The authors would like to acknowledge the support of the Common Humanitarian Fund (CHF) in funding the workshop. The Nutrition, Health, Food Security and Agriculture (FSAC), and WASH (water, sanitation and hygiene) Clusters; their respective cluster coordinators and assessment working groups; and the relevant government ministries, including MoPH, and the United Nations Office for the Coordination of Humanitarian Affairs (OCHA) are also acknowledged for their role in the development of multi-sector assessments.
Location: Afghanistan
What we know: Joint (multi-sector) data collection and analysis is needed to inform nutrition-specific and nutrition-sensitive interventions.
What this article adds: Due to limitations in inter-sector assessment frameworks and experiences, the Nutrition Cluster took the lead in improving joint assessment tools and their implementation in Afghanistan. A nutrition data clinic (to define standards for assessment) was followed by a multi-sector workshop (Nutrition, Health, FSAC and WASH) to develop a multi-sector method for rapid and in-depth assessments for Afghanistan. Two SMART-based methodologies and questionnaires were developed, with a rollout plan. Nine SMART assessments have since been undertaken. The Nutrition Cluster and MoPH have overall responsibility for joint assessments. The Afghanistan National Disaster Management Authority (ANDMA) has a key role in assessment coordination. The Nutrition Cluster will undertake joint assessments and share information with working groups/authorities. Protection and Shelter Clusters will be brought into the process. This successful initiative is catalysing multi-sector analysis and programming through development of a Nutrition Integrated Phase Classification (IPC).
Context
The UNICEF conceptual framework, used by the nutrition community for several decades, identifies three levels of causes of malnutrition: immediate causes, which operate at the individual level (inadequate dietary intake and infection); underlying causes that influence households and communities (inadequate access to food and/or poor use of available food, inadequate child care practices and poor water and sanitation and inadequate health services); and basic or root causes around the structure and processes of societies (economic, cultural and religious systems). This framework is an important tool that helps understanding and causal analysis and reflects the importance of joint programming between sectors. This highlights the need for joint data collection and analysis to fully inform the range of nutrition-specific and nutrition-sensitive interventions required.
A coordinated approach to emergency assessment and prioritisation of the needs of affected people lays the foundation for a coherent and efficient humanitarian response. For protracted crises, the depth and volume of information needed for an effective response increases as the crisis evolves. This often translates into a requirement for in-depth cluster/sector, thematic or agency-specific assessments to inform planning and operations, which in turn necessitates a harmonised assessment approach with joint needs analysis.
According to the Inter-Agency Standing Committee (IASC) Operational Guidance for Coordinated Assessments in Humanitarian Crises,1 the Humanitarian Coordinator (HC), supported by the United Nations Office for the Coordination of Humanitarian Affairs (OCHA), is responsible for coordinating emergency assessments across clusters/sectors at the country level. In turn, clusters/sectors at the country level are responsible for engaging in all relevant aspects of multi-cluster/sector assessment coordination. OCHA is mandated to ensure that each cluster/sector is provided with the necessary common services and tools for effective inter-cluster/sector collaboration, including inter-agency needs assessments. Coordination mechanisms applied to needs assessments differ depending on the phase and nature of a crisis. A multi-cluster/sector initial rapid assessment (MIRA) is recommended during the first two weeks following a disaster, followed by joint or harmonised intra-cluster/sector in-depth assessments as the crisis continues.
There has been a ‘silo-minded approach’ to conducting assessments in Afghanistan for many years there. No inter-sector assessment framework existed at government level; all government agencies/ministries worked in isolation. The Humanitarian Coordinated Assessment Working Group (HCAWG) was established in January 2016 under the Afghanistan Humanitarian Country Team (HCT). However, membership was at a management rather than a technical level. Formerly the Household Emergency Assessment Tool (HEAT), developed in Afghanistan and updated in September 2016, was the primary multi-sector tool used in the country. While it was nominally developed in consultation with the clusters, in-country implementation was fraught with challenges in terms of methodology, data collection and analysis, and the tool faced ongoing criticism from WASH, Nutrition and Health clusters. For example, HEAT assessments were overly long, with too many questions, yet failed to capture important information necessary for individual and multi-cluster analysis. Furthermore, since there was no sampling method but an exhaustive assessment of households instead, data was too bulky and difficult to analyse.
In response to these limitations, and considering the UNICEF conceptual framework for malnutrition, the Nutrition Cluster made the logical choice to take the lead in improving joint assessment tools in Afghanistan. A workshop was proposed to develop a new, multi-sector framework as a collaboration between the different relevant ministries. This process is described in this article.
Process of the development of the multi-cluster assessments
In March 2016 the Nutrition Cluster and the Ministry of Public Health (MoPH)/Public Nutrition Department, with financial support from the Common Humanitarian Fund (CHF), organised a nutrition assessments data clinic to try to standardise how nutrition assessments are conducted by defining and adapting standard, globally accepted tools and methods for country-wide adoption. Following the success of the nutrition data clinic, the Nutrition Cluster and its Assessment and Information Management Working Group (AIM WG) organised a multi-sector workshop in November 2016 through Action Contre la Faim (ACF) Afghanistan with the purpose of developing a multi-sector method for rapid and in-depth assessments for Afghanistan. The project was first discussed in the AIM WG before being taken forward for implementation. Guided by the UNICEF conceptual framework for malnutrition, information from Nutrition, Health, FSAC and WASH sectors was considered the most important for defining nutrition response; therefore representatives from these four sectors were invited to participate.
In the initial stages the Nutrition Cluster Coordinator and an ACF representative approached the Cluster Coordinators of the WASH, Food Security and Agriculture (FSAC) and Health Clusters to discuss the aim, objectives and modalities of the workshop. After general agreement and buy-in from all three coordinators, a workshop preparation task force was created that included one or two representatives from each cluster (usually the Cluster Coordinator and a chair of the relevant Cluster’s Assessment Working Group (AWG)). The taskforce met twice to discuss the agenda and the remaining work was carried out through emails. The main functions of the task force were to develop an agenda for the workshop and agree facilitators for each section; nominate participants from each cluster with expertise and decision-making responsibility in assessments; and prepare workshop materials (including a presentation of the current rapid and in-depth assessments used by the clusters in Afghanistan).
Participants of the workshop came from all four Clusters (Nutrition, WASH, FSAC and Health) and a representative from OCHA was invited to observe. (Given that it was a technical activity, they were not in the position to directly participate.) Clusters were represented by Cluster Coordinators, local non-governmental organisations (NGOs), international NGOs and United Nations (UN) agencies. Where a cluster had an AWG, representatives were prioritised for participation. In addition each cluster invited a counterpart from the government (from each relevant ministry) to ensure government engagement and buy-in from the beginning of the process. In total, 30 participants attended the three-day workshop, including MoPH and Ministry of Rural Rehabilitation and Development.
In working towards its objectives, the workshop focused on:
- Clarifying what is meant by multi-cluster assessments: Why and when are they conducted?
- Understanding assessments after an emergency: What do we know, what should we know, and how do we make it happen?
- Analysis of existing coordinated and sector assessments to make structured observations about what worked and what did not; distillation and prioritisation of the most important lessons learned; and capture of recommendations on how these can be addressed.
This was done largely in group work, with each group focusing on key elements of multi-sector assessments as follows: preparation and planning; data collection and field work; and analysis, interpretation, preparing findings/reports and dissemination.
The first day was dedicated to rapid multi-sector assessments. Current methodologies in use at the inter-cluster level were presented (HEAT tool) and gaps and key challenges discussed. This was followed by a presentation by each cluster of their own rapid assessment methodologies. In general, the methodologies, frequency, indicators and tools used by each cluster varied significantly. For example, the WASH cluster used a one-page questionnaire to collect information from 210 households, regardless of the size and population as needed, while the FSAC cluster conducted a national food security assessment of food security twice per year. In order to ensure that the rapid multi-sector assessment method proposed at the workshop could meet the diverse requirements of each cluster and could be undertaken in different emergency scenarios as witnessed in Afghanistan, participants developed the methodology, indicators and questionnaire collaboratively. This included a joint timeline of assessments, potential tools, sampling methodologies, representativeness, quality assurance, limitations and type of indicators.
A similar process was undertaken for in-depth assessments on day two. At the time of the workshop no in-depth multi-sector assessments existed; each cluster worked in isolation to collect and analyse its data.
Day three focused on next steps, including how to ensure that the recommendations of the workshop were implemented; designing the multi-sector assessment framework; developing a workplan; defining main tasks for the inter-cluster AWG; developing agreement on how to move forward with joint programming following the assessments; and agreeing how to conduct multi-sector programme analysis.
Multi-cluster assessments methodologies
Two methodologies and questionnaires were developed at the workshop for integrated Nutrition, WASH, FSAC and Health Cluster rapid and in-depth assessments, as well as a plan for rollout in Afghanistan.
Multi-sector rapid assessments
For multi-sector rapid assessments, the main challenges identified with the HEAT assessment tool in Afghanistan, as outlined earlier, were the absence of a sampling methodology and limited relevance of data collected for the four clusters (due to lack of consultation and limited use of cluster feedback by OCHA in its development). To address the first challenge, it was agreed to use an improved simplified methodology, similar to Rapid SMART. (A Rapid SMART survey is most often used in emergency contexts with high insecurity, which limits the team’s access to survey areas. Rapid SMART has limitations compared to the standard full SMART methodology: it cannot be used to assess global acute malnutrition/severe acute malnutrition (GAM/SAM) for larger geographical areas, cannot be extrapolated beyond the zone of assessment, and can only be used to inform emergency responses, not long-term programmes. http://smartmethodology.org/survey-planning-tools/smart-methodology/rapid-smart-methodology/)
Consultation on the adoption of the SMART methodology started at the nutrition data clinic, which was technically supported by the ACF Canada SMART team. Further consultations on the adoption of the Rapid SMART sampling approach were undertaken through the ACF HQ technical contacts to ensure a representative, random, rapid approach. Key considerations regarding Rapid SMART sampling methods are outlined in Box 1.
During the group sessions each cluster was asked to develop up to four indicators and questions to collect in rapid multi-cluster assessments. These were then discussed in plenary, but it was subsequently agreed post-workshop that only two indicators should be selected per cluster, as follows (see Table 3 for details):
- Nutrition: Wasting based on mid upper-arm circumference (MUAC) (children and women of reproductive age) and exclusive breastfeeding of infants aged 0-5 months (six completed months, equivalent to 0 - < 6 months).
- WASH: Proportion of people who have access to safe drinking water and proportion of people who have access to sanitation facilities.
- Fragile and conflict-affected states (FCAS): Household Dietary Diversity Score (HDDS) and food stock availability.
- Health: Prevalence of acute respiratory infection (ARI), diarrhoea and measles and access to basic health services.
Multi-sector in-depth assessments
After evaluation of different methodologies used by clusters, it was agreed that the Modified SMART methodology was applicable for the development of the multi-sector in-depth assessment. (Modified SMART (also known as SMART++) is where additional indicators from other sectors are included. This does not affect the sampling approach, since only anthropometry and mortality (the most critical indicators) are considered when determining sample size.
Modified SMART was adopted without substantial changes to the methodology; however the standard questionnaire was expanded to include additional indicators from other sectors. The nutrition SMART surveys being implemented in Afghanistan at the time were already taking into account additional indicators from each cluster. This was the result of a standardisation exercise conducted in March 2016, during which members of the Nutrition Cluster consulted with WASH, FSAC and Health Clusters to determine a broader range of indicators to include (albeit restricted to those with a direct impact on child nutritional status). As this process had already taken place, and in general the indicators being used were the same as those proposed in the workshop, little work was needed to come up with the list of indicators and questions to be included in the final questionnaire (Table 3).
Multi-sector assessments in practice
Since January 2017 nine SMART assessments have been conducted in the provinces of Kandahar, Farah, Saripul, Samangan, Jawzjan, Takhar, Bamyan, Kapisa and Dykundi. Joint inter-sector analysis was constrained by a capacity gap in the position of Nutrition Cluster Coordinator for several months. At a follow-up and lesson-learning workshop conducted by ACF, and to ensure government ownership of the process, it was decided that a main coordination body should take the responsibility of multi-cluster assessments that should involve the Afghanistan National Disaster Management Authority (ANDMA).
Box 1: Choice of sampling methods in Rapid SMART methodology
In this methodology, the choice of sampling method depends on the geographical zone and population group to be assessed.
Where there is one settlement to assess (e.g. one camp, one block of houses in a city, one village) and:
- the population is less than 200 households, then an exhaustive assessment on all eligible children living in the area should be carried out. In this case, the team must exhaustively visit all households in the settlement.
- the population is above 200 households, then follow the decision tree (Figure 1) for a simple or systematic method of random sampling. In such cases, a minimum sample size of 150 households is enough to gather relatively meaningful prevalence. A design effect of one is assumed. The precision of the results varies in this case, as shown in Table 1 (third column).
Where there is more than one settlement and if the population is dispersed, cluster sampling should be used. For rapid multi-sector assessments, at least 25 clusters should be selected using probability proportional to size (PPS) in order to cover all the target population in the geographical zone. To select the clusters, an estimate of the number of the population living in each settlement is necessary. Random selection of clusters can be made using emergency nutrition assessment (ENA) software. For cluster random sampling, a minimum sample size of 250-300 households is enough to estimate GAM prevalence. A design effect of 1.5 is assumed. The precision of the results varies, as described in Table 2.
Figure 1: Decision tree for household selection at last stage of sampling
HH: households
Table 1: Sample size and precision for one-settlement rapid multi-sector assessments
HH: households
Table 2: Sampling strategy and precision for more than one settlement/dispersed population rapid multi-sector assessments
HH: households
Table 3: Indicators and questions included in the multi-sector in-depth assessment questionnaire
Key lessons learned
The previous multi-cluster assessment methodology had many limitations, which reflected the limited technical capacity of OCHA. The approach to improving the methodologies, led by the Nutrition Cluster, helped overcome this constraint and was regarded as an improvement by all involved, including OCHA, who agreed to continue using the approach and to engage additional clusters (such as Protection and Shelter) in this collective effort. Certain difficult-to-achieve aspects of coordinated multi-sector assessments – such as multi-stakeholder buy-in and participation and a degree of joint analysis – have been realised through the process. Furthermore a broad range of stakeholders contributed to the workshop, which created a positive environment for meaningful discussion and agreed actions going forward. Finally, the process benefited from a great willingness among stakeholders to tap into and strengthen existing structures (AWGs of the clusters under the leadership of technical experts from UN/NGOs and relevant ministries) in order to fast-track the process of multi-sector assessments. This approach also facilitated strengthening of government capacities, securing government buy-in and ensuring its ownership of the process and willingness to take it forward.
Agreed next steps
Involve Protection and Shelter Clusters: For rapid multi-sector assessments, the clusters agreed that the two remaining clusters, Protection and Shelter, should be included going forward. This will ensure the development and use of a single joint tool. It is important to note that it was the consultative process and constant engagement with the four clusters that ensured their buy-in to the rapid multi-sector assessment tool. This same process must now be applied to the Shelter and Protection Clusters to ensure their buy-in, too. OCHA has agreed to support this process.
Operationalise the HCAWG: Following discussions at the 2016 workshop, the HCAWG was reconvened in January 2017 under the Inter-Cluster Coordination Team (ICCT) at a technical level. This will ensure a more operational focus for the group, with clear Terms of Reference (ToR) that focus on coordination of joint assessments and joint data analysis.
Nutrition Cluster to conduct future joint assessments: Given that the Nutrition Cluster has experience and capacity on SMART assessments, it was agreed that it will be responsible for undertaking joint assessments and sharing information with other clusters, the HCAWG and the ICCT. Data sharing and multi-sector analysis will also be facilitated through the inter-cluster HCAWG.
Nutrition Cluster and MoPH to have overall responsibility for joint assessments: Once all clusters agree on the ‘multi-cluster’ questionnaire, including Protection and Shelter, the Nutrition Cluster will officially update the current SMART guideline. MoPH will then approve this as the new standard methodology for assessments. Overall responsibility to undertake the assessments and prepare the final reports will lie with the Nutrition Cluster and MoPH, owing to the alignment of the new assessment methodology with the Nutrition Cluster’s existing ways of working.
Action joint data analysis in future: The clusters have agreed on the need to collaborate not just on data collection, but also on data analysis and there is impetus to realise this: extracting individual, cluster-specific data for siloed analysis defeats the purpose of the exercise.
Explore multi-cluster programming: Clusters would like to use the momentum of this effort to explore how they can work together operationally. One proposed way of doing this is to conduct a Nutrition Integrated Phase Classification (IPC). This is planned for November 2017 and will be led at country-level by the Nutrition Cluster AWG, with input from experts in other clusters as necessary. Another proposed activity is a four-cluster workshop to explore ways to plan and work together. Both activities are currently included in the Nutrition Cluster work plan for 2017; however this has not been implemented yet due to a seven-month capacity gap in the position of Nutrition Cluster Coordinator.
Conclusions
The development of rapid and in-depth multi-sector assessment tools, led by the Nutrition Cluster, has been a success both in generating fit-for-purpose tools for the collective and in creating momentum and commitment for joint analysis and programming. The tools produced are largely relevant for other countries; however it is important that the full consultative process is implemented in each country, rather than merely adapting the tools. This will ensure buy-in from all relevant clusters and government ministries, which is fundamental to the success of joint assessments. Continued engagement and advocacy is needed across sectors. The Nutrition Cluster is an obvious choice for leading such processes, given the need to apply a nutrition lens to multi-sector analysis and response.
For more information, contact: Anna Ziolkovska
Endnotes
1www.humanitarianresponse.info/system/files/documents/files/ops_guidance_finalversion2012_1.pdf