Evaluation of Concern’s response to the Haiti Earthquake

By Andy Featherstone

Andy Featherstone is an independent humanitarian research and policy consultant. With significant experience working for NGOs and interagency networks, he specialises in helping agencies learn from their work and in bridging the gap between policy and practice.

The earthquake that struck Haiti on January 12, 2010 did so with devastating consequences. More than 200,000 people were killed, 300,000 were injured and over one million were left homeless. The cocktail of extreme vulnerability coupled with the huge loss of life and massive destruction wrought on Haiti’s largest urban area and political and commercial hub effectively decapitated the state. It left hundreds of thousands of people traumatised and without the means necessary to sustain life and livelihood. It was this that precipitated the tremendous generosity that saw Concern raise 28 million euro and embark on its largest singlecountry humanitarian programme since it was established in 1968.

An evaluation was undertaken eight months after the earthquake¹ and followed an unprecedented expansion of the programme and staff. The purpose of the exercise was to review the appropriateness, timeliness, efficiency and effectiveness of both the interventions carried out and operational support systems with an important focus being placed on documenting lessons learnt.

The evaluation findings were as follows.

Timeliness

The response was timely, particularly the early support to water, sanitation and hygiene (WASH). An area where Concern performed particularly strongly was in quickly moving beyond a focus on support to urban areas to meeting the needs of displaced and host communities in rural areas.

While Concern was successful in providing timely assistance in important sectors of its response, the organisation lacks consensus over the use of its surge capacity mechanisms, the Emergency Response Team (ERT) and Rapid Deployment Unit (RDU). It is urgent that agreement is reached over how to manage and deploy these assets to most effect in the future.

Signs at the water stand encourage people to drink chlorinated water - prior to the painted signboards being erected, people were not convinced that the water was safe to drink

There is a need for greater fluency within the organisation in applying streamlined human resources (HR) systems that better support emergency recruitment (like the use of generic terms of reference and the quicker adoption of recruitment systems that can fast-track staff appointments, inductions and entry to the programme).

While the organisation has a comprehensive financial management system, the experience of the team in trying to link programme outputs with donor contracts to facilitate contract management has been a challenging one and would benefit from review.

Coverage

Concern has performed well against the coverage criterion, choosing to launch a multi-sectoral response which targeted both urban and rural communities. Given the extraordinary nature of the crisis, an extraordinary response was required. While it means that Concern has had to work hard to deliver against such a large programme, it has done so successfully.

Relevance

The programme has maintained its relevance throughout the response by providing a mix of targeted assistance (through cash for work and livelihoods programmes) and blanket distribution of basic commodities. While the context of the crowded urban environment has made it difficult at times to attain minimum standards, interviews with project participants suggest that the services being provided by Concern are meeting their needs.

The use of SPHERE standards and indicators in Concern’s programmes has provided an important yardstick for success. Given the clarity that the WASH team now has about progress that needs to be made against cluster standards, it will be important that swift progress is made to achieve these.

Connectedness

Concern’s established presence in the country and the strong links it has with communities provided an important platform for the earthquake response. This has served both to provide much needed assistance and further cement Concern’s relationship with project participants. Since the organisation is also working in sectors it has established a competence in, interventions in these areas often benefit from strong analysis.

While considerable progress has been made in building a coherent Concern team in Haiti, it will be important to continue to strengthen ways of working to ensure strong integration between all parts of Concern’s mandate, whether long-term development or humanitarian response.

Out-patient nutrition clinic at Place de la Paix, Port-au-Prince

Accountability

Good progress has been made in cascading key accountability principles such as the provision of information, consultation and participation of communities throughout the Concern programme. An important area for Concern and the broader humanitarian community will be to ensure that camp committees are consistently working in the best interests of the people they represent.

There is some urgency in establishing accountable and representative camp committees and Concern should continue to work with the cluster to find workable solutions.

Effectiveness

Concern has been effective in mounting a large multi-sectoral humanitarian programme in Haiti. In particular, the breadth of the programme, the timeliness of many of its early interventions and the prioritisation of meeting rural in addition to urban needs has been impressive in such a complex context.

Protection and peace building are now being mainstreamed across the humanitarian programme. In the absence of government policy on durable settlement solutions and with elections planned for November 2011, it is likely that these cross-cutting areas will become ever more relevant. It will be important that there is sufficient capacity and that activities are fully integrated across all of Concern’s humanitarian work in Haiti.

Two resources are highlighted - a review of the excellent organisation-level meta-evaluation conducted in 2009 and the Preparing for Effective Emergency Response (PEER) document which summarise organisational knowledge and learning. Condensing these into a set of succinct (1-2 pages) documents, highlighting key lessons for programme design and delivery and organisational systems and ways of working, would be a wise investment for the future.

Identifying best practice

The evaluation highlighted the significant contribution which Concern has made in meeting the needs of earthquake-affected communities in a timely and effective manner. A number of these are worthy of particular mention as they demonstrate significant innovation, achieve a level of excellence in response, or show proficiency in a particular area of response.

Eighty per cent of schools in Port au Prince were damaged or destroyed. Concern has hired ‘animators’ to facilitate play and learning.

Responding to the needs of rural and urban communities

Concern has built on its established presence in the country to extend both relief and recovery activities to those living outside the immediate environs of Port-au-Prince. Its operations in La Gonave and Saut D’eau have been backed up by a robust analysis of the numbers of displaced and the impact this displacement has had on the local infrastructure and economy. In the first three months of response, Concern provided cash, tents to meet emergency shelter needs and non-food items to targeted beneficiaries in these areas. Given the propensity for aid to be targeted at the most visible and most numerous claimants, the targeting of rural areas by Concern is noteworthy. Not only does it have the potential to ensure that those affected by the earthquake and subsequent displacement are supported, but it goes some way to slowing the inevitable return to Port-au Prince which the limited services available in the city would have struggled to accommodate.

Transitional shelter design and delivery

The roll-out of the T-shelter programme has been considered by many (including the shelter cluster coordinator) to be exemplary. While it took some time to conduct the baseline survey, to assess needs and to procure the materials, the programme has benefited from sound targeting and excellent organisation of work processes which has allowed for swift production of the shelters. The design is innovative and has taken account of the needs for earthquake- and hurricane-proofing. The approach taken towards the use of contractors to manage shelter construction using labour sourced from the camp has allowed the work to progress quickly, while fostering ownership and transferring important skills to members of the camp population.

Temporary shelters nearing completion at Tabarre

Humanitarian leadership & coordination

Throughout the response, Concern staff have shown a commitment to participating in humanitarian leadership and coordination forums. While the Country Director is part of the Humanitarian Country Team (HCT), sector staff members have also played prominent roles in clusters and sub-cluster groups.

Interviews showed they had an astute understanding of both the resource requirements that coordination commitments place on the organisation but also the opportunity this provides to influence the humanitarian response far beyond what Concern could achieve through other means. While some members of the humanitarian community may dismiss the value of coordination as being too time-consuming, it is the way of doing humanitarian business. In contexts such as Haiti, it is essential for prioritising assistance and avoiding duplication. The team has used the forum that the clusters offer for raising issues of concern to the wider humanitarian community such as protection, shelter design issues and the threat of forced evictions. Interviews with Cluster Coordinators highlighted the value they placed on Concern’s participation, as well as the important contribution that staff members have made to the work of the clusters.

Lessons learned

Concern’s response in Haiti has necessitated interventions that have either been unprecedented or provide the potential to lead the organisation in new directions or challenge current ways of working.

Responding to urban disasters

Humanitarian organisations in Haiti have struggled to make their earthquake responses relevant to the urban environment. While ‘camps’ are often administratively more easy to support, the lack of space and the infiltration of powerful gangs into the over-crowded urban environments has created a significant challenge to agencies who are more used to working in peri-urban or rural environments. The lesson here may be to stretch humanitarian comfort zones and look at methodologies to support smaller and more decentralised settlements that focus on the importance of community, that benefit from strong links with local authorities and which have strong links with the private sector to ease the process and sustainability of handing over services. In seeking to address these issues directly, the approach taken by Concern in Tabarre Issa camps has much to offer. In working with both resettled and host communities, and in trying to plan the settlement less as a regimented camp, Concern’s programme has taken a ‘neighbourhood’ or ‘community’ approach which is considered as best practice by many in the cluster.

Cleaning drains as part of Cash for Work

Innovative approaches to addressing vulnerability: The Baby Tent Programme

Haiti is the first humanitarian response where the concept of ‘baby tents’ has been delivered to scale, in a context where women had suffered significant trauma and where the use of infant formula was often prioritised over breastfeeding practices. The Baby Tents were a space which offered privacy, care and counselling and which could advocate for, educate and support women on breastfeeding. Where infants were not breastfed, the baby tents monitored the use of infant formula. Thus they provided potentially life-saving services for both breastfed and non-breastfed infants.

A high value placed on an independent procurement capacity

An important lesson has come from trialling the United Nations Humanitarian Response Depot (HRD) which is available for both prepositioning of stock items and procurement. Experience from the earthquake response strongly suggests the need for Concern to retain an independent procurement capacity as quotes from the HRD procurement agency were found to be uncompetitive and lead times were considered to be lengthy. While the initial reliance on air freight has a significant cost attached to it, it did ensure that the programme could scale-up swiftly and ensured that minimum quality standards for procured items were met.

¹Featherstone. A (2010). Evaluation of Concern’s Response to the Haiti Earthquake. October 2010