Boosters, Barriers, Questions: an approach to organising and analysing SQUEAC data
By Andrew Prentice (VALID), Balegamire Safari Joseph (VALID), Esther Ogonda McOyoo (Concern), Faith Manee Nzidka (ACF), Hassan Ali Ahmed (Mercy USA), Jackson N Chege (Islamic Relief), Jacqueline Wairimu Macharia (ACF), Kennedy Otieno Musumba (ACF), Lilian Mwikari Kaindi (ACF), Lioko Kiamba (ACF), Mark Murage Gathii (IMC), Muireann Brennan (CDC), Samuel Kirichu (Concern), Salim Athman Abubakar (IMC), Stephen Musembi Kimanzi (IMC) and Mark Myatt (Brixton Health)
This article outlines an approach to organising and analysing collected data and for planning further data collection during a SQUEAC coverage assessment. The approach, known as ‘Boosters, Barriers, Questions (BBQ)’ involves examining the collected data for boosters (i.e. anything that might act to support coverage) and barriers (i.e. anything that might act to undermine coverage) - see Figure 1. The approach was developed during a Coverage Monitoring Network (CMN)1 training on the SQUEAC coverage assessment methodology in Kenya in October and November 2012.
The BBQ approach uses three panes to record (1) boosters, (2) barriers and (3) issues arising that require further data collection (questions). A fourth pane acts as a key to symbols that are used to indicate data sources and data collection methods. Figure 2 shows the parts of the BBQ tools and explains their purpose.
The town of Meti
A large hand-drawn BBQ tool, such as is shown in Figure 3, proved useful for managing a SQUEAC investigation. The BBQ tool provides a summary of the current state of the investigation and serves as a focal point when deciding data collection needs and dividing tasks between team members. The collaborative focus provided by the BBQ tool facilitates team building and improves the quality of the investigation.
When using the BBQ tool, each of the listed boosters and barriers is tagged with symbols that indicate the different sources of data supporting each finding (e.g. programme staff, carers of severe acute malnutrition (SAM) cases, community leaders) and the different methods used to collect the data (e.g. structured interviews, semi-structured interviews, informal group discussions). The use of these symbols allows the easy identification of findings that have, or have not, been validated using triangulation by source and method - see Box 1, Box 2, and Figure 4.
Box 1: Triangulation by source and method
It is important that the collected qualitative data are validated. In practice, this means that data are collected from as many different sources as possible. Data sources are then cross-checked against each other. If data from one source are confirmed by data from another source, then the data can be considered to be useful. If data from one source is not confirmed by data from other sources then more data should be collected, either from the same sources or from new sources, for confirmation. This process is known as triangulation.
There are two types of triangulation: Triangulation by source refers to data confirmed by more than one source. It is better to have data confirmed by more than one type of source (e.g. community leaders and clinic staff ) rather than just by more than one of the same type of source. Type of source may also be defined by demographic, socio-economic, and spatial attributes of informants. Lay informants such as mothers and fathers are sources of differing gender. Lay informants from different economic strata, different ethnic groups, different religious groups, or widely separated locations are also different types of source.
Triangulation by method refers to data confirmed by more than one method. It is better to have data confirmed by more than one method (e.g. semi-structured interviews and informal group discussions) than by a single method.
You should plan data collection to ensure triangulation by both source and method. The BBQ approach is designed to help you do this.
Data collection using triangulation is a purposeful and intelligent process. Data from different sources and methods should be regularly and frequently compared with each other. Discrepancies in the data are then used to inform decisions about whether to collect further data. If further data collection is required, these discrepancies help determine which data to collect, as well as the sources and methods to be used.
Box 2: Example of using the BBQ tool for triangulation by source and methods and hypothesis formation and testing in SQUEAC assessments
On the first day of collecting qualitative data, one of the teams found that carers of young children living in villages near health facilities delivering integrated CMAM (iCMAM) services were more aware of the iCMAM programme than carers of young children living in villages further away from health facilities delivering iCMAM services. This information, collected using informal group discussion with carers of young children in their home villages, was not confirmed by information collected by other teams. This finding was, therefore, placed in the ‘Questions’ section of the BBQ tool. To confirm this finding, questions were developed and incorporated into interview guides for semi-structured interviews intended to be administered to other sources (i.e. teachers and villages leaders and elders) on the following day.
On the second day of collecting qualitative data, information collected using semi-structured interviews with teachers, village leaders and village elders confirmed the original finding that distance was negatively associated with awareness of the iCMAM programme. The finding had, therefore, been confirmed by triangulation by both source and method:
| Source | Method |
| Carers of young children | Informal group discussion |
| Village chiefs | Semi-structured interview |
| Village elders | Semi-structured interview |
| Teachers | Semi-structured interview |
The collected data led to the formation of two linked and formal hypotheses:
Carers of young children living in villages close (i.e. within 1000 metres) to health facilities delivering iCMAM services are aware of the iCMAM programme (i.e. know it exists, know that it treats malnourished children, know that entry is decided by mid-upper arm circumference (MUAC), and know that the programme delivers RUTF).
and:
Carers of young children living in villages far (i.e. further than 5 kilometres) from health facilities delivering iCMAM services are not aware of the iCMAM programme (i.e. do not know it exists, do not know that it treats malnourished children, do not know that entry is decided by MUAC, and do not know that the programme delivers RUTF).
To confirm these hypotheses, small studies were performed by two different teams on days three and four of the SQUEAC assessment. Each team travelled to two villages, one of which was located near (i.e. within 1000 metres) to a health facility delivering iCMAM services and the other located far (i.e. further than 5 kilometres) from a health facility providing iCMAM services. The EPI5 sampling method was used to select five households from each of the selected villages. The EPI5 sampling method was used because it is known to return a sample similar to a simple random sample of households. Carers of young children in each of the selected households were interviewed about their awareness of the programme. An in-depth interview guide was developed for this purpose. In addition, each team was given MUAC tapes and sachets of Ready to Use Therapeutic Food (RUTF) (two types) in order to test whether informants recognised them, reflecting an awareness of the programme.
The data arising from the small studies are summarised below:
| Study team | Village name | Distance class | Distance from iCMAM facility | Number of respondents interviewed | Number of respondents aware of the programme | Number of respondents not aware of the programme |
| 1 | Lakole | Near | 1 km | 5 | 5 | 0 |
| Mlandanoor | Far | 6 km | 5 | 1 | 4 | |
| 2 | Bilikomarara | Near | 1 km | 5 | 5 | 0 |
| Martaba | Far | 13 km | 5 | 0 | 5 |
The summary data were analysed using the simplified Lot Quality Assurance Sampling (LQAS) testing procedure with good awareness defined as more than 50% of carers of young children being aware of the programme.
The first hypothesis (i.e. good awareness if near to an iCMAM facility) would be confirmed if more than:
respondents were aware of the programme in the near villages. The study found ten respondents who were aware of the programme. The first hypothesis was, therefore, confirmed.
The second hypothesis (i.e. poor awareness if far from an iCMAM facility) would be confirmed if:
or fewer respondents were aware of the programme in the far villages. The study found one respondent who was aware of the programme. The second hypothesis was, therefore, confirmed.
Given these results, the SQUEAC assessment team concluded that distance was a factor affecting programme awareness and was likely to be a factor affecting coverage.
The approach outlined here is typical of the SQUEAC investigation process. That is:
- Qualitative data are collected and validated using triangulation by source and method.
- Validated qualitative findings are then used to develop formal hypotheses which are tested using simple quantitative techniques.
These are sometime referred to as Stage I and Stage II of a SQUEAC investigation.
A note on samples sizes and methods: Small sample sizes are common in SQUEAC. This is because the use of prior information acts to reduce both classification and estimation error. In the example small studies presented here, the association between proximity and awareness is very marked and a naïve frequentist analysis (i.e. an analysis that discounts all prior information) testing the null hypothesis that programme awareness was independent of proximity to the programme would return a p-value of p < 0.0001 (one-tailed Fisher Exact Test). This is very strong evidence against the null hypothesis. An estimation approach would return a risk ratio of 10.00 (95% CI = 1.56; 64.20) with proximity as the ‘risk exposure’.
Findings associated with different sources and/or methods can be treated as validated. Figure 5, for example, shows a barrier "Mothers go to traditional healers who are not linked to the programme" revealed by in-depth interviews with carers of SAM cases in the programme and several informal group discussions with traditional birth attendants and traditional healers.
Findings associated with few sources of data and/or few methods of data collection are candidates for further investigation. Specific questions for further investigation are listed in the central ‘Questions’ section of the BBQ tool. Figure 5, for example, shows a potential barrier "Only person who can identify malnutrition is the Community Health Worker" revealed by a single source/method (i.e. informal group discussion with carers of young children in communities) and required, therefore, further investigation by collecting data from different sources and/or similar sources using different methods.
As the investigation proceeds, the BBQ tool is redrafted to (e.g.) combine similar findings and remove invalidated findings. Figure 6 shows the result of redrafting the BBQ tool shown in Figure 3 to combine similar findings. Note how some of the findings related to barriers have been combined using diagrams showing cause and effect linkages between barriers.
Grouping findings by consequence helps with building concept maps that describe the relationships between boosters and barriers in a programme. For example, a programme's failure to recruit traditional healers as community-based case-finders may lead to late admissions, complicated cases requiring long stays or inpatient care (which may lead to defaulting), poor outcomes and negative opinions of the programme. Figure 7 shows a fragment of a programme concept map illustrating these relationships.
Figure 8 shows the first draft of a programme concept map from day five of the SQUEAC investigation. This illustrates the richness of data that arises from SQUEAC investigations and the ability of the BBQ tool to assist with data-analysis and presentation.
Sorting the lists of boosters and barriers into three categories with regard to the likely size of their effect on coverage (i.e. large, moderate, and small effects on coverage) helps with building the prior for the stage three survey.
The BBQ tool proved useful during the CMN training in Kenya and helped trainees make sense of large quantities of data from many and disparate sources. The boosters and barriers model helped trainees maintain the focus of the investigation and to plan data collection. The BBQ tool may be used as an alternative to mindmapping or as a complement to mind-mapping.
For more information, contact: cmnproject@actionagainsthunger.org.uk
1See news piece in this issue about the CMN Project
Imported from FEX website