Beyond the indicators - Assessing project impact on children’s lives

By Holly Welcome Radice, Maria Ruiz-Bascaran, Abebe Zewdu, Oumar Mohamud, Mekdes Asfaw, and Mahlet Bezu

Pictures from top to bottom: Holly is the Head of Livelihoods & Nutrition Information Systems. Maria is the Senior Programme Manager of the RAIN project and former programme manager for PILLAR II. Abebe is the DRR Coordinatorm Oumar is Project Coordination-Somali and Mekdes is Project Coordinator- Afar. All work for the PILLAR PLUS project and with Save the Children UK Ethiopia. Mahlet was previously interned with Save the Children UK Ethiopia and currently is pursuing a masters at Lund University.

Save the Children UK would like to thank the families in the Afar and Somali Regions that participated in these interviews and ECHO (European Commission Humanitarian Aid Department) for funding this project.

In humanitarian work, how do we know that we are positively reaching distinct members of a community? When we calculate beneficiary households, can we be sure that everyone in the household is really benefiting? And is it clear to what extent they are benefiting? These are questions that all nongovernmental organisations (NGOs) struggle with in programme evaluation. Save the Children UK Ethiopia recently sought to find answers to some of these questions in one of its projects.

For nearly 20 years, Save the Children UK has been providing support to pastoral communities in Ethiopia, with current programmes in the Somali and Afar regions. One project is a drought preparedness and risk reduction project, called PILLAR (Preparedness Improves Livelihoods and Resilience), funded by ECHO's Regional Drought Decision (RDD). The project was implemented in two phases between April 2008 and June 2010 in selected drought districts of the Afar and Somali Regions. PILLAR seeks to contribute to community level drought risk reduction through prevention, mitigation, preparedness and response through a range of activities focusing on the three pillars of pastoralism- people, livestock and natural resources.

PILLAR has performed well against its objectives and indicators, with some 18,000 households reached across two regions. This is good news for the community, the NGO, and the donor. Being a child focused NGO, Save the Children UK wanted to understand and document to what extent PILLAR has impacted the lives of children.

Impact assessment involving children

Internationally, Save the Children UK aims to ensure that child participation-informed and willing involvement of children-is meaningfully incorporated into all its work. Some direct benefits for children had already been revealed in the communities. A local teacher in Afamana Tibedha in Afar had mentioned that since the construction of a water point in the village, students were attending classes more regularly and enrolment had increased by 40%. To get a sense of some of PILLAR's impacts on children, Save the Children UK conducted a rapid field assessment in six beneficiary communities and case studies of six beneficiary children were completed. The case studies were based on focus group discussions and individual interviews with selected beneficiaries and discussions held separately with adult beneficiaries and their children.

Children who were interviewed belonged to families that were involved in water point construction, small scale irrigation and income generating activity micro-projects. According to those children and their caregivers, project impacts were seen similarly. Broadly the answers of the interviewed children were centred on two categories, household and personal for children, as shown in Table 1.

These findings show that the interventions went well beyond what was measured in the indicators-availability of water in the dry season and protected / diversifying livelihoods- and that the interventions had layers of effect. Immediate results such as better nutrition, access to basic needs, access to clean water were valued by the children. Most heartening were the reports of increased school attendance and reduction in time spent on chores and even more time to play. These indirect benefits are positive and encouraging for long term community resilience as education is a key component to improving livelihoods.

A fodder production group has decreased Sead's time looking for pasture

Children as direct beneficiaries

Such an assessment is not scientific by any means, but the value is obvious. It serves as a singular opportunity to 'ground truth' the effect of interventions on different groups within a population. For Save the Children UK's implementation team, this exercise was helpful to see the layers of impact that the project has had. Children in the interviewed beneficiary households clearly directly benefited from the interventions on personal, household and community levels.

Women targeted by income generation activities in Fik, Somali region

In a project focusing on community resilience, children may normally be viewed as indirect beneficiaries. Additionally, the humanitarian nature of the project could be mistakenly viewed as only providing short term insurance against shocks. However, this rapid assessment has discounted both of these theories. Children in these communities have received direct, immediate benefits. If these benefits identified are maintained, the investment will surely have a profound affect on the future of individual children's lives and ultimately on community resilience.

Ultimately, the assessment gave some ideas regarding what interventions to build on, better to target children in disaster risk management (DRM). The next phase of PILLAR will work directly with the children in Alternative Basic Education schools. This scheme will pilot children- led DRM committees that will look to build the capacity of children and their caregivers to be better able to reduce disaster risks within their own communities.

Finally, such an assessment is an opportunity to listen to and learn from children. Fostering the practice of voicing opinions and contributing to community matters can be an empowering process. In pastoral communities, such as those targeted in PILLAR, being able to speak on behalf of one's community is an essential tool in reduction of vulnerability to drought. It can encourage a generation to be proactively contributing to their own community's involvement in early warning, early response and community preparedness.

For more information, contact: Holly Welcome Radice, email: Holly.R@scuk.org.et