Developing a mobile health app to manage acute malnutrition: a five-country experience

By Tine Frank, Emily Keane, Natalie Roschnik, Colleen Emary, Melani O’Leary and Laura Snyder

View this article as a pdf

Tine Frank is a consultant engaged by Save the Children with funding from DIFD through the Transform Nutrition research consortium. 

Emily Keane is Nutrition Advisor with Save the Children. 

Natalie Roschnik is Senior Research & Impact Advisor with Save the Children.

Colleen Emary is Senior Emergency Nutrition Advisor, World Vision.

Melani O’Leary is Regional Health and Nutrition Technical Specialist, World Vision.

Laura Snyder is Design & Development Manager with World Vision.

The mobile health project was made possible thanks to the financial support of USAID OFDA, Global Affairs Canada and DIFD (through the Transform Nutrition research consortium). The project could not have been successful without the support and contribution of the project country Ministries of Health, Dimagi (software partner) and the efforts and commitment of our country team staff and participating health workers, who work to improve the health of their communities. The project would like to thank the Transform Nutrition research consortium for funding a learning paper (Frank, 2017), on which this article is based, and the World Vision, Save the Children and Dimagi staff who agreed to be interviewed.

Context

Community-based management of acute malnutrition (CMAM) is a proven high-impact and cost-effective approach in the treatment of acute malnutrition. However, its success can be limited by a number of factors, including lack of protocol adherence by health workers and inaccurate record keeping. Limitations in paper-based systems, typically used to support referrals, make it difficult to track cases through the different levels of treatment to recovery and discharge. Missing or poor quality information that is not accessible to decision makers within a reasonable timeframe hampers surveillance, logistics management, alerts to stock-outs and response to surges in caseloads. 

There is strong evidence that mobile device-based (mHealth) apps can improve frontline health workers’ ability to apply treatment protocols more effectively and to improve the provision of supply chain management. Currently, the majority of frontline health workers supporting CMAM programmes are trained in standardised protocols but do not have on-site quality-assurance supervision or job aids to support protocol adherence and accurate treatment. Tracking and site-monitoring systems are paper-based and subject to error and misreporting. An innovation that secures a dynamic link between frontline patient treatment data with programme performance, reporting and stock management has the potential to dramatically improve monitoring, evaluation and real-time decision making for CMAM. Effective job aids and a performance-monitoring system are needed to improve the quality of CMAM programmes globally.

In 2013, World Vision (WV) initiated the development of a mobile phone-based application for CMAM. Since then, the CMAM mHealth app has been piloted in five countries: Niger, Afghanistan, Mali, Chad and Kenya. This article shares experiences and lessons learned through interviews with World Vision, Save the Children and Dimagi staff at country and headquarter level. It draws on a learning paper to be published later this year which gathers lessons learned from developing and rolling out the app in these five countries with considerations for scale-up and future mobile health projects.

The CMAM mHealth project 

The first CMAM mHealth app was developed in Chad in 2013, based on Chad national protocols with support from WV nutrition technical specialists. A global call for partners by WV secured the engagement of Save the Children (SC) and International Medical Corps (IMC). Five months were spent testing and refining the prototype app. Funded by the USAID Office for Disaster Assistance (OFDA), the app was subsequently piloted in established CMAM programmes in Chad, Kenya, Mali and Niger by WV, SC and IMC, working closely with technical software partner Dimagi and local ministries. In November 2014, WV Afghanistan introduced the app into a Global Affairs Canada-funded Maternal and Under 5 Nutrition and Child Health (MUNCH) project. In 2015, SC added an impact evaluation to the Kenya pilot via the DIFD-funded, IFPRI-led Transform Nutrition research consortium to evaluate the impact of the app on the quality of CMAM reporting and care. The results of the impact evaluation will be available in late 2017. 

The overall objective of the mHealth application was to improve CMAM treatment, reporting, monitoring and supply management for improved quality of care for children suffering from acute malnutrition. The app has a dual purpose: first to improve the quality of CMAM programme delivery, as health workers are more likely to correctly follow the treatment protocol, assess a child’s nutritional and medical status more accurately, provide the correct number of ready-to-use therapeutic food (RUTF) sachets, routine medications and identify defaulters or non-responders; and second, to provide more accurate and timely data for district-level CMAM management and decision making.

Public-private partnership

Over the last nine years, WV has collaborated with the Bill and Melinda Gates Foundation, Grameen Foundation and Dimagi in a public-private partnership to develop a common set of apps built within the MoTECH Suite. Gates Foundation and Grameen Foundation provided the funds and strategic direction, WV was the lead implementing partner and Dimagi the software partner responsible for developing and testing the app prototype. Dimagi is a US-based software social enterprise that develops technologies to improve service delivery in underserved communities. Its technology platform, CommCare (www.commcarehq.org), is the most widely adopted mobile platform for low-resources settings and is used by WV, the World Health Organization (WHO) and others in over 50 countries. CommCare can manage and monitor health records of children and pregnant mothers; support diagnosis and treatment; collect real-time data; disseminate audio and pictorial behaviour-change communication messages; and refer complicated cases. The CMAM prototype app was developed under the WV-Dimagi partnership and contextualised to each country. As such, Dimagi was the natural choice of technical partner for the CMAM mHealth project.

Adapting the app to each country

The contextualisation of the app in each country began with a rapid assessment of mobile networks, technology coverage and usage levels in the CMAM project area and intended pilot-testing site. Information was also collected on existing CMAM data management practices and processes. This data was used to inform project design and as the basis for end-of-project evaluation, reflection and learning.

The app was first contextualised in Niger, which required French translation, adaptation to the Ministry of Health (MoH) national protocol for the management of acute malnutrition, addition of local multi-media (visual and audio), discussion with mobile network operators, adaptation of project monitoring and evaluation tools, testing with users and re-refining the app. Much of this was done remotely with in-country visits by Dimagi. District health staff were engaged in the testing process. A similar approach was used in all five countries, with context-specific variations where necessary.

Rollout

In preparation for rollout, a training of trainers was conducted jointly by Dimagi with the implementing non-governmental organisation (NGO) (WV, SC or IMC), followed by a training of health workers and field observations. After deployment, implementing partner staff provided ongoing support and supervision. Regular user observations were undertaken and project staff monitored app use through the server, recording issues and prompting app updates from Dimagi or reminders to health workers to upload data.

The number of health facilities provided with the mobile app ranged from eight in Niger to 50 in Afghanistan; the number of health workers trained ranged from 20 in Niger to 52 in Afghanistan. Trainings and rollout were staggered, with the first taking place in Niger in May 2014, followed by Afghanistan in November 2014, Mali and Chad in February 2015 and Kenya in October 2015. Project close-out was staggered, with Chad ending in June 2016, followed by Mali and Afghanistan in September 2016, Niger in October 2016 and Kenya in December 2016. 

Assumptions and compromises

It was assumed at project design stage that the app would be fully developed in Niger before being adapted to other countries in a phased approach (country by country) and the design and reporting function would be added once the app was deployed and in use. Dimagi were expected to provide at least two in-country, on-site support visits to develop and test the app, then return for the trainings of trainers and end users. A WV Africa Regional Nutritionist would provide ongoing support and supervision. 

In practice, there were compromises. Security issues in Chad and Afghanistan, along with the Ebola threat in Mali, restricted travel; visits were largely replaced by workshops in Senegal (Mali and Chad) and Dubai (Afghanistan), along with remote technical support. Adapting the app to national protocols took longer than anticipated, which had considerable impact on software programming; these delays subsequently affected the scheduled availability of the Dimagi team. This also meant that country introduction occurred almost simultaneously, rather than in a phased approach, limiting time for lesson learning and adaptation between pilots. The majority of technical support was remote rather than on-site. As a result, the reporting functionality was not completed for West African countries, limiting the ability to assess impact of the app on CMAM performance. The reporting function was developed for the final Kenya pilot; looking ahead, this will serve as a template for others.

Key experiences and lessons learned

Health worker acceptance of the app

In most countries, health workers were initially resistant to using the new mobile app and reluctant to give up the paper-based reporting system. They did not trust the app’s capacity to store and send data via the ‘cloud’ and worried that their data would be lost. The app also ‘forced’ health workers to follow the treatment protocol, which took more time. Health workers would previously take short cuts, dropping certain steps like the appetite test and health and nutrition counselling. This was no longer possible with the app, which some found tedious and so would revert back to the paper-based registers. In some countries, health workers were still required by the MoH and their supervisors to complete the paper registers (due to the reporting function not being completed on time), which in effect doubled their workload.

With time, practice and support, health workers in all five countries learned to use and trust the app and saw many benefits, including better quality information; stronger communication with remote supervisors; valued use of pictures and counselling messages to communicate with caregivers; reduced need to refer to guidance manuals (Afghanistan); reduced errors and improved quality of care (Mali); reduced conflict between healthcare workers and caregivers over treatment (Chad); and improved case management, including minimising defaulting (Kenya). Barriers to app use included unfixed bugs, network issues and power problems (e.g. inadequate battery life and non-functioning solar chargers were the main challenge in Chad).  Independent of the mHealth app, the lack of a consistent supply of nutritional commodities was a constant challenge; health facilities frequently ran out of stock of supplies and essential drugs, leading to a high default rate.

Technology landscape 

Procurement of suitable quality devices, as well as reliable network operators and data packages within budget, was a challenge. Battery-life issues, screen size, phone inauthenticity, network speed and coverage all had a negative impact on health workers’ motivation and uptake of the app. Key country challenges included purchase of fake phones with compromised functionality (Niger); poor network coverage, problems with solar chargers requiring health workers to travel to upload data, increasing time and travel costs, data upload requirements exceeding data bundle capacity; personal use of phone data (Afghanistan); and low network coverage that delayed data submission by months (Kenya).

Local government involvement

The degree and nature of local government involved varied by country. In Mali, the MoH (ANTIM/Agence Nationale de Télésanté et d’Informatique Médicale) provided considerable expertise and support and was involved in all the stages of the pilot, which led to considerable buy-in and commitment in the regulation and acceptance of the app. However, ensuring continuity of the Mali CMAM mHealth project was a challenge, since MoH prioritises resources for MoH rather than those developed externally. In Chad, district health officials were not sufficiently involved during implementation. This limited MoH leadership in moving the mHealth project forward and linking it to the MoH health information system.  In Afghanistan, regional and national MoH officials were involved from the beginning and have showed interest in scaling up mHealth interventions (it is seen as cost effective, facilitating easy collection of accessible data). The decentralised health system in Kenya, coupled with high workloads and competing demands, meant buy-in took longer than anticipated. In particular, the time and motivations needed to impact health workers’ behaviour change were not properly understood and accounted for. However, the MoH in Kenya has proved to become a particularly strong partner in the project. 

Adapting to national systems

In Chad, there were significant delays in contextualisation: terminology was not easily understood by users and the need to translate to local languages was beyond the scope of the project. In Kenya, use of the app was not authorised until the summary data report was aligned to the country health information system, which caused further delays.

Software development, testing and technical support

Delays in sorting software and programming issues had a negative impact on users and project staff in all countries. For example, the Kenya app was developed based on the Niger version; this process took four months, largely because the Kenya protocol and case management guideline differs considerably from that of Niger. The Kenya app experienced consistent bugs throughout the project due to programming issues, which had a marked influence on user confidence.

Pilot country selection

The contexts in which this type of mHealth innovation is most needed tend to be complex and challenging environments. In Mali and Kenya, high turnover of health facility staff meant continuous training was needed to ensure uptake among new staff. In Afghanistan, the political environment, weather conditions and security concerns meant that Dimagi was not able to provide onsite support, delaying mHealth activities by over one year. In Chad, there was insufficient budget to fund MoH officials to join field visits and trainings, limiting buy-in and supervision capacity. All project country teams were understaffed. Due to initial delays in the project, the country phases were mostly conducted almost simultaneously, resulting in deployment of an app that had not been sufficiently tested, causing frustration among both users and project teams.

Health worker protocol adherence

The use of the app seems to have improved adherence to the treatment protocol and the quality of care compared to the paper-based job aids and reporting tools. However, as the general treatment protocols for CMAM are complex and time-consuming, and with the app being based on standard national protocols, the app itself was relatively complex and data-heavy. As outlined above, the app was more time-consuming as steps could not be skipped and some health workers were using dual systems (electronic and standard government paper forms) to meet data reporting requirements. 

A number of developments took place in countries to improve adherence. In Niger, phones were blocked for personal use from the outset to minimise wasting of data, battery and memory card; this was well accepted. In Kenya, a WhatsApp group was created for all users to interact with each other and the project team and share experiences; there was a marked effect on compliance and usage. In Afghanistan, users who had lower competency scores in post-training assessments were monitored and supported.  

Recommendations 

The lessons from the CMAM mHealth project around planning, implementation and evaluation are valuable for anyone considering future app development and adaptation and for scaling up by MOH and other partners. Recommendations include:

When introducing a new mHealth app to health workers, particularly in remote locations, significant on-site support is necessary for both health workers and supervisors. This has budget implications for fuel and travel costs to remote clinics. Rollout should only be considered once an app has been fully tested in non-remote health facilities, all major bugs have been fixed and the system for recharging the devices has been fully tested. Rapid assessment information on networks and power should be used more strategically to develop contingency plans in the event of failure. These should be jointly developed and agreed to with MoH partners, as well as mobile network operators.  

Health workers are more likely to include the app in their daily routines where it is incorporated into national policies and strategies. To enable this, it is critical to plan and budget for local and national government engagement and uptake from the start. This includes regular meetings with key stakeholders to plan, discuss progress and generate recommendations; field visits; and clear communication of the relevance of the mHealth app to support national strategies and goals.  Issues such as stock-outs of nutrition supplies should be discussed in relation to the app functionality. For scalability, the app should be linked to a wider continuum of care, related to integrated maternal and child health services.

It is imperative that the technology partner provides ongoing support for troubleshooting, bug fixing and capacity building. The ideal technology partner would provide considerable in-country presence and support, either through country representatives or frequent site visits, to build national capacity, especially regarding capacity to fix bugs and update the app. Pre-field testing, a workshop attended by all stakeholders and users could be used to test the app and identify errors and inconsistencies. In challenging contexts, significant flexibility on timelines, budgets and tasks are inevitably needed.  

The feasibility of piloting projects in challenging contexts should be examined closely to resource and adapt the deployment accordingly. It is worth considering pilots in lower-risk, more stable and accessible areas/countries with more reliable infrastructure. This would minimise delays, frustrations and limited uptake caused by lack of network, electricity and access for sufficient support supervision. Once app development and testing is at a satisfactory level and training and support requirements have been identified, a more stable version of the app could be deployed to high risk/poor infrastructure areas. 

There is a need to review and better understand what is feasible in terms of capacity and time for health workers in low-resource settings, with high patient caseloads (and resulting long waiting times for children and caregivers). Simplification of management protocols can be reflected in the app, which can help accuracy in case management and, with reporting functionality developed, could save health worker reporting time.

Conclusions and looking ahead

The collaboration between the implementing partners across the different project countries has been highly successful and important to the project, with mutual sharing and learning having a positive effect on all implementation sites. Based on this experience, the mHealth app has great potential. For scale-up, several priorities have been identified, including completing reporting functionality and developing stock monitoring functions (Mali); expanded use of the application in current user countries (national deployment in Afghanistan); linking the application to HMIS systems (planned for Mali) and national mHealth platforms; further capacity building to support uptake and use of application; exploration of scale-up opportunities with national and county ministries (Kenya); linking multiple child health-focused apps on an integrated platform (e.g. GMP, iCCM); and connecting the application to other platforms (e.g. CMAM Report, World Vision CMAM database). 

For more information, contact: Emily Keane, email: Emily Keane, email: e.keane@savethechildren.org.uk or Colleen Emary, email: Colleen Emary, email: colleen_emary@worldvision.ca

To see a short video clip on the experiences of staff in Wajir, Kenya, visit: https://kenya.savethechildren.net/media/videos#

References

Frank T. 2017. Developing a mobile health app to manageacute malnutrition. World Vision/Save the Children Learning paper.