ALNAP Annual Review 2002

The ALNAP Annual Review 2002¹ provides a synthesis of the principal findings and recommendations of evaluations of humanitarian action, completed and made available to ALNAP in 2000-2001. Encompassing 46 evaluations and 9 syntheses, it identifies crosscutting issues and trends and the learning and accountability challenges for those within the humanitarian sector, whether practitioner, policymaker or evaluator. The synthesis also highlights the humanitarian sectors repeated failure to learn and implement lessons from past experience.

The central theme of the review is learning within the humanitarian system, including mapping of current learning mechanisms, an analysis of their strengths and weaknesses, and key constraints to learning. It also reviews learning mechanisms from other sectors that might be adopted or adapted to meet the specific characteristics and learning needs of the humanitarian sector. The quality of individual evaluations is assessed using the 'ALNAP Quality Proforma'. This highlights the strengths and weaknesses of current practice from conception and design, through the evaluation process, to the dissemination of findings and implementation of recommendations.

One of the main sectors evaluated was food aid and emergency agriculture (19% of the reports). In general this sector met the short-term objective of 'feeding mouths', and successes were usually qualified. Problematic areas identified were ration levels and inadequate attention to disadvantaged groups, such as women and children. Water and sanitation, and health interventions were also successful in meeting their short-term objectives. Shelter, and in particular housing, was the least successful sectoral emergency intervention.

In general, the evaluation reports tell of a job well done, however a number of generic weaknesses within the humanitarian system are described by the review. Overall, the quality of evaluation reports was unsatisfactory. In particular, evaluation reports were poor at explaining why interventions were successes or failures. Other problematic areas included failure to integrate short-term emergency objectives with longer-term plans, poor co-ordination, limited exploration of building on indigenous coping strategies and the need to improve monitoring.

Some keys to success were identified, including dedicated staff, good needs assessment, and the ability of international non-governmental organisations (NGOs) to work with well-established partners. Dedicated staff were identified as central to the success of humanitarian action. However they were often hindered by agency institutional systems typically fraught with poor communication, ineffective staff capacity building and inadequate training.

A number of examples of good learning practice were highlighted in the review, such as the development of common networks and sources of learning, e.g. Reliefweb, Humanitarian Practice Network and ALNAP. However, while the evaluation mechanism is well established in the humanitarian sector, weakness lies in a failure to distinguish between learning and accountability approaches to evaluation. For example, external, independent evaluations (which may be undertaken primarily for accountability purposes) are often not considered conducive to learning at individual and team levels, while outside criticism is often handled defensively rather than constructively.

Several constraints to learning in the humanitarian sector were identified, in particular lack of clarity as to intervention objectives and desired outcomes, and poorly defined responsibilities and relationships between individuals, teams and organisations. In addition, high rates of staff turnover create a severe obstacle to learning and knowledge transfer, while mechanisms for cross-organisational learning are poorly developed. Short-term funding, pressure to maintain low overheads and competitive behaviour amongst organisations vying for profile and fund access, were considered the major barriers to effective learning by, and within, the sector.

Possible areas for action identified in the review include:

Within organisations

• Assessments of current organisational practices in learning, knowledge management and training
• Development of programmes to support learning and knowledge management
• Need to commit to and prioritise learning, and work towards culture of learning and self-critical reflection.

Across sector

• Concerted action to reduce high staff turnover
• Improved resource provision for learning and development of mechanisms to increase crossorganisational learning, such as the Groupe URD (Urgence Réhabilitation Développement) approach used in Central America and ALNAPs Learning Support Office in Malawi
• . Setting up of an annual sector award mechanism for instances of outstanding learning practice
• Development of an electronic library for the sector.

Copies of the ALNAP Annual Review 2002, Humanitarian Action: Improving Performance through Improved Learning can be ordered from the ALNAP Secretariat, ODI, 111 Westminster Bridge Road, London SE1 7JD or alnap@odi.org.uk at £15 per copy (£12 to ALNAP members).

¹Active Learning Network on Accountability and Performance (ALNAP). Established in 1997, ?is an international, interagency forum working to improve learning, accountability and quality across the humanitarian sector. Details of ongoing ALNAP activities, new developments and publications can be found on ALNAP's website http:\\www.alnap.org.