Malawi + 2 more

A stitch in time? Independent evaluation of the DEC's Southern Africa crisis appeal

Format
Evaluation and Lessons Learned
Source
Posted
Originally published

Attachments

July 2002 to June 2003
Executive Summary

Readers should note that the appendices to this report include some well-written and interesting material. This includes two beneficiary assessments for Malawi and Zambia, the Financial Management Report, as well as pieces on HIV/AIDS and Humanitarian Space in Zimbabwe.

Introduction

The Southern Africa Crisis Appeal was a new departure for the Disasters Emergency Committee. Here for the first time was an appeal that was intended to prevent a humanitarian crisis rather than respond to one. This led to a response that was a mixture of traditional relief activities together with activities that were more akin to rehabilitation or traditional development.

The evaluation team found a mixed picture. It encountered a large number of examples of better practice, with instances from all the DEC agencies. However the quality of the interventions varied between countries and between agencies, and sometimes between different country programmes belonging to the same agency.

On the positive side there was probably more beneficiary participation in this crisis than in similar crises in the past. Because of the political situation in Zimbabwe agencies were experimenting with feedback mechanisms that could have far wider application. Beneficiary participation was also very good in Malawi though participation here was largely limited to implementation as opposed to the design of the intervention. DEC agencies were important actors in highlighting to the world the difficulties faced by the region. Some of the nutrition surveys and analysis carried out by them, for instance, were instrumental in raising awareness of the impending crisis.

In general the DEC agencies carried out their emergency responses in the areas where they had their development programmes. The evaluation team considered this appropriate since these areas were often the poorest. It also meant that the programmes funded were often appropriate to the problems faced by communities, though timeframes were often shorter than optimal.

Some of the communities assisted by the DEC agencies in the crisis had previously been assisted by the development programmes of the DEC agencies. The evaluation team noted that such communities did not seem to be better able to weather the crisis than adjoining communities that had not had been assisted by DEC agencies before the crisis.

Programme coverage was generally good and co-ordination mechanisms in the region worked well, though there were major constraints to humanitarian space in Zimbabwe. Agencies were sensitive to the cultural norms of the communities with which they worked and made efforts to support local capacity. However, improvements could be made in some of the partnership arrangements that DEC agencies have with local organisations.

The crisis revealed that DEC agencies did not always have a deep enough understanding of the communities in which they worked. Underestimations of the importance of remittance or of coping strategies were common. Agencies were also sometimes slow to scale up in response to the crisis though this does not seem to have had any serious adverse effect on programme impact.

HIV/AIDS is a major factor in the region, with up to one third of adults affected in the worst areas. Despite this the links between HIV/AIDS, coping strategies, and food security were not well understood and further research is needed on these.

One of the biggest problems was that the DEC agencies, like the rest of the humanitarian community, lacked a conceptual model for dealing with the crisis. The crisis was overstated in terms of the threat of famine, but at the same time the chronic roots of the crisis were understated. While internal agency analysis was often more sophisticated and nuanced than the message presented to the media, even this did not capture the whole picture. The lack of an appropriate conceptual model led to some inappropriate responses.

DEC agency advocacy, though sometimes good, was nevertheless patchy across the region and across agencies. More should have been done in this crucial area.

Agencies had a strong framework in place to ensure good financial management, though narrative and financial reports were of a very variable quality. Agencies were also good at carrying out evaluations, but more could be done to standardise approaches, improve the quality of evaluation methods and promote inter-agency learning.

Methodology

The evaluation consisted of the following:

  • Initial interviews with agency headquarters in London.

  • Beneficiary assessments in Zambia and Malawi.

  • Evaluation team visits to Zimbabwe, Zambia, and Malawi.

  • A desk study of agencies. own evaluations using the assessment proforma developed by the Active Learning Network for Accountability and Performance in Humanitarian Assistance (ALNAP).

  • A review of agency financial management procedures based on a series of questionnaires.
The evaluation consulted over 1,300 people of whom more than three hundred were key informants.

Appendix 3: on page 80 has details of the persons met, and Appendix 14: on page 194 contains the evaluation team itinerary.

The DEC appeal supported the work of twelve agencies in seven countries. The evaluation team did not have the resources to analyse individual agency programmes in detail. The examples that have been given, both of good and bad practice, are inevitably somewhat random. Where examples of individual agency practice have been highlighted they are meant to be illustrative of broad themes and should be seen in that light rather than as representing detailed analysis of particular programmes.

Conclusion

The evaluation team concluded that the DEC Southern African Crisis Appeal was justified in that the DEC agencies contributed to prolonging lives and preventing suffering. However, the complexity of the underlying chronic problems means that the response offers large opportunities for learning for the DEC agencies.

Recommendations

Below are the key recommendations of the report. No time frame is given for the recommendations though the following mechanism is suggested:

  • After the agencies have had time to consider the recommendations and their implications, the DEC to hold a meeting in three months. time to discuss which of the recommendations the DEC agencies as a body accept, and how they plan to implement them. The plan from this meeting to be placed on the public area of the DEC site.

  • The DEC to hold a further meeting in twelve months. time to review the progress in implementing the recommendations. The minutes of this meeting again to be published on the DEC website.
The financial management report contains numerous other recommendations that are not summarised here.

(pdf* format - 494 KB)