Bangladesh

Bangladesh: 1998 Flood Appeal - An Independent Evaluation

Format
Evaluation and Lessons Learned
Source
Posted
Originally published

Attachments

AN INDEPENDENT EVALUATION
Roger Young and Associates

JANUARY 2000

FINAL REPORT

EXECUTIVE SUMMARY

From July to September 1998, Bangladesh suffered the most extensive, deepest and longest lasting flooding of this century. An estimated one million homes were damaged, the main rice and other staple crops were lost due to flooding, and some 30 million persons in 6 million families were affected by the floods.

In mid-September 1998, three weeks after the Government of Bangladesh had approved external assistance to deal with a national emergency situation, the DEC launched a public appeal for aid to those affected by the floods. A sum of =A3 3.8 million was raised and distributed to 11 of the DEC agencies best placed to respond to the disaster. The majority had a long-standing history of relief and rehabilitation activities in Bangladesh, working directly and /or through affiliation with local partners.

As a disaster prone country, vulnerable to recurrent flooding, cyclones and drought, Bangladesh has had considerable experience with disaster management. Government and non-governmental organisations have had significant disaster response experience over the past thirty years in Bangladesh, including famine in 1971, floods in 1974, 1987, 1988 and 1998 and cyclones of major proportions in 1971 and 1991.

An independent evaluation of the DEC activities was carried out in September-October 1999 with interviews of DEC agencies in Britain and Ireland, and visits to their offices and/or partners in Bangladesh. The evaluation team met with Government of Bangladesh and United Nations officials as well as with national NGOs, community leaders and beneficiaries. This report represents the findings and recommendations of an independent evaluation of the DEC "Bangladesh Flood Appeal" funded response to the 1998 flood.

The evaluation team was asked to review the effectiveness and efficiency of the DEC "Bangladesh Flood Appeal" funded disaster response, to assess the accountability of DEC agencies using public funds, to assess the value added of DEC funding, and to assess the level of coordination among DEC partners and agencies and other disaster response actors.

Accountability and Value Added

Do DEC funds provide additional funds to agencies and are these funds adequately reported and accounted for?

DEC financing for the Bangladesh flood appeal has provided DEC agencies with additional funds to undertake relief and rehabilitation activities. The scale of the 1998 floods in Bangladesh were so massive and long lasting that the need for humanitarian assistance was far greater than the supply of assistance resources. While many agencies received financial support from their own donors, and used internal finances, they also had to turn to official agencies such as DfID and the European Community Humanitarian Office (ECHO). The DEC financing has been additional to these funds and has permitted agencies to scale-up their relief and rehabilitation activities. There was no evidence that any DEC agency, or donors, had substituted DEC funds for their own financing.

Moreover, some agencies had not been successful in accessing EC and/or DfID funds at the time of the DEC appeal. Therefore, the evaluation team concludes that DEC funds did permit agencies to access funds that were not otherwise available. DEC funds facilitated an estimated 25% additional activities by the participating agencies.

Scaling up has consisted of either a greater coverage of geographical areas or a larger number of activities provided in the rehabilitation phase.

DEC funds are accounted for by the participating agencies in a transparent and accountable manner. The reporting requirements are adequate and include:

  • a statement of the agency's competencies and provisional plan of action to be submitted within 48 hours of a formal request to the broadcasters to support a national appeal;
  • a more definitive plan of operation to be submitted within 4 weeks of the appeal launch; the agencies' share of funds raised by the appeal is based upon a predetermined distribution formula;
  • a final narrative report due in the 7th month following the launch of an appeal, detailing actual operations and including an assessment of the agency's original statement of priority needs, and areas covered, the number of people assisted and a statement of funds received and expended.
Assessment Issues

How well did agencies and partners target vulnerable groups and households when they were assessing and selecting beneficiaries?

Most assessment procedures reviewed by the mission were adequately thorough and careful. Again, this can be attributed to the quality of the contacts with affected communities, since assessments relied heavily on information from partners or staff working in the field.

Several agencies mention that coordination at the local level (involving both government and NGOs) was good enough to enable them to prevent duplication, and/or to target households missed by other schemes.

The most frequently cited criteria for targeting were: households suffering severe loss; landless or assetless households; female-headed households; the elderly and individuals with disabilities. There are indications the effectiveness of targeting declined in that order, with the last category being the most difficult to identify and reach.

Some agencies targeted beneficiaries who were already part of their regular programming, who would not necessarily meet the above criteria for relief and rehabilitation. Others offset this bias by delivering flood relief by area and selecting beneficiaries within those areas with the assistance of village leaders, or local relief committees.

The evaluation team did not have sufficient field exposure to determine whether there were instances of relief going to non-affected or undeserving households. Most agencies did extend their relief and rehabilitation work beyond established target groups and beneficiaries.

For example, one agency that initially concentrated efforts with its eight core partners, provided funds to a further 43 local agencies and used its own staff to work in four severely flooded thanas where it had no partners.

Few agencies relied on local government sources to determine beneficiaries. Many said they by-passed VGF (Vulnerable Group Feeding) card-holders, unless they were sure union authorities had distributed cards only to those genuinely in need.

Some agencies noted that partners tended to direct rehabilitation interventions to their own programme participants, even though their initial relief coverage was more extensive.

Who Benefits from Disaster Responses?

This is a very difficult question to answer with certainty. The resources available to provide relief and rehabilitation could not meet the needs of those affected by the 1998 floods. Damages have been estimated at well over =A3 1.5 billion while estimated relief financing from all sources amounted to an estimated =A3 600 million. Many needs remain unmet.

Ideally appeal funds would go to those most in need, those most severely affected by the floods. However, assessment of need was imperfect in the context of the floods; homes were submerged and families had abandoned their homes. Many agencies were able to rely on communities themselves to identify the neediest. The evaluation team was told that most disaster relief went to communities affected by the floods, but not necessarily always to the most severely affected people within these communities.

There is some criticism that NGOs in general targeted their own group members disproportionately. Group members who were participating in the well-developed credit and savings programs are known to be among the poor, but not the poorest members of a given community. The most disadvantaged members of a community may not always have benefited from some NGOs disaster response.

Given the scope of this evaluation it is not possible to provide a definitive answer to the question. It would seem that the more effective efforts at appropriate targeting did include consulting local communities. Where the communities are well known to the partners, verification and monitoring that the disaster response was targeted to the most severely affected was more accurate.

To what extent did beneficiaries participate in decisions regarding targeting and activities?

Many of the agencies and their partners followed the participatory approaches used by their development programmes to shape the relief effort. There were quite varied interventions even within the programmes of single agencies, indicating they were reflecting local demands and assessments. Agencies without extensive community development experience were more inclined to deliver their programmes with less regard for community choices.

Examples of planned responses that were altered to meet beneficiary demands include removing unsuitable clothing from foreign relief packages and adding more food; including some cash, ORS, extra oil and women's sanitary napkins in relief packages.

The rehabilitation phase provided more scope for participatory inputs than the relief phase. The evaluation could not determine to what extent rehabilitation activities - for example housing - were determined by the recipients as opposed to the donors. Many agencies provided rehabilitation inputs as loans not grants, which was maybe based on their own needs or strategies.

Response issues

How well did the elements of relief activities (food, medical aid, shelter, water and sanitation, fodder, etc) match identified needs?

The DEC agencies and their local partners all have considerable experience with flood relief in Bangladesh, and it was not difficult for them to determine what was required and develop the appropriate procedures for delivering it. The mission did not learn of instances where relief packages contained superfluous goods or were missing essential goods, though there was wide variation in proportions and contents.

Several agencies reported they were able to deliver services, such as medical aid, which they did not normally provide, by hiring temporary staff or getting outside assistance. Some assigned head office staff to strengthen local capacity or to help manage coordination and monitoring.

Throughout Bangladesh there was a widespread mobilisation of volunteer assistance during the 1998 floods. The DEC-funded organizations also benefited from this response, getting help from the public or from their own networks.

Some agencies reported the supplies of relief goods in the affected areas were more than adequate. This meant local officials and politicians were less likely to commandeer or divert supplies. The local availability of relief supplies did contrast with the overall shortage of relief materials in a national context.

With a few exceptions the DEC agencies reported they were able to procure what they planned to distribute and to handle the logistics of distribution. Shortages of non-grain seeds appeared to be the principal procurement problem. Some complained there were cash flow problems caused by the banks' poor system for transferring funds to branch offices.

To what extent was standardisation an issue?

Although the basic list of requirements for both relief and rehabilitation were similar, there was no standardisation of the proportion of these in each overall package, nor was there much standardisation of the amount or design of the separate elements (e.g. in the size or content of food packages or the type and cost of houses). Different agencies did different things, based on organisational priorities, skills of their partners or policy decisions, given the available funds.

There is a continuing debate over whether rehabilitation disbursements should be grants or loans. There is a wide variation among the agencies on handling this choice. There is also variation on loan terms, and the disposition of funds made available from loan recovery.

Did the response activities build on lessons learned from past flood disasters?

There is general consensus that disaster relief was handled better for the 1998 flood than for the severe flooding ten years before in 1988. One crude indicator of this is the much lower fatality rate (1,376 in 1998 compared to about 6,000 in 1988).

There are some interesting comparisons to the findings of an ODI evaluation of the 1988 flood relief:

  • the 1988 report noted the housing interventions varied widely in design, cost, etc. This is still the case in 1998.
  • the 1988 report found the response was very top-down and there was little community-level participation. This appears to be less of an issue in 1998.
  • the 1988 report criticised the continuity and effectiveness of local coordination efforts. In 1998, DEC agencies were quite positive about the adequacy of communications, preparedness, and government-NGO cooperation.
Effectiveness

Agencies that implemented effective relief and rehabilitation activities demonstrated the following competencies:

  • effective disaster preparedness, both of the agency and its partners through recurrent training, and disaster manuals; also a preparedness of communities through prior and recurrent training, facilitating mechanisms to foster cooperation and community action at the time of the floods;
  • efforts to assess specific needs, and the degree of deprivation; this was especially difficult in a context where many were in need of humanitarian aid, living on rooftops or had had to abandon their homes for flood shelters;
  • an ability to coordinate agency efforts with other actors, including government at central and local levels, UN organizations and other NGOs to ensure that duplication of effort was minimised and that relief and rehabilitation activities were directed towards the most severely affected communities.
Cost effectiveness

There are several important examples of cost effective initiatives undertaken by the DEC agencies and partners. Three examples are cited here:

  • the use of NGO partner agencies, to extend the reach of the disaster response was an initiative taken by the majority of DEC agencies, as a result of lessons learned in the 1988 flood; this allowed agencies to reach more people in need at lower costs than establishing their own programmes;
  • a nutritional assessment carried out by one agency to assess the nature and extent of malnutrition, especially in children. This survey served to provide accurate and timely information that also allowed other agencies to develop appropriate food packages and targeting. With limited food resources to distribute, this rapid survey proved effective and informed several agencies' responses.
  • the construction of flood shelters for humans and livestock; with relatively modest costs for construction, these shelters permitted families to access shelter and a place to save their livestock, an essential asset for rural poor families.
Organization and Management Issues

Did the 1998 experience build on, or improve, coordination mechanisms among organizations providing relief, at the local and national level?

One good example of coordination was the sharing of the nutrition survey results, which led to modifications in plans and interventions for food relief. This led the agency itself to reduce its draw on the DEC funds.

Some thought the Government of Bangladesh (GoB) was late in declaring an emergency but there was little subsequent complaint about the government's role in coordinating the relief efforts. Compared to the 1988 floods (when there was no NGO Affairs Bureau) the government did not unduly delay NGO plans or withhold approval for specific activities; although, a few agencies complained that approvals for rehabilitation work through the NGO Bureau were delayed, compared to approvals that were provided for work in the relief phase.

Perhaps the most important lesson communities have learned from previous disasters is that they can influence what happens in a disaster effort delivered by government and non-government bodies. People have not only developed concepts of their right to be provided with relief and rehabilitation but also of the value of doing something for themselves. The extent of public participation and volunteerism was very impressive in the 1998 flood disaster. The scale and intensity of the 1998 flood and the reduced loss of life relative to previous floods suggests that the people have highly developed capacities to cope under difficult circumstances. The general public has also learned the importance of safe drinking water, as demonstrated by the widespread use of tubewell water, even during the height of the flooding.

The Mission concludes that there was adequate coordination among government and non-governmental actors during the flood response. As a result of this coordination, duplication of relief and rehabilitation efforts was for the most part, kept at minimum levels. On the whole, those who most needed relief and rehabilitation efforts were provided for, although to varying extents.

One key coordination forum for DEC agencies was the Disaster Forum, a body that brought all the major actors together in the 1998 flood response. At the national level, the NGO association ADAB, and the government's NGO Affairs Bureau were also coordinating NGO activities. DEC agencies believe these were useful in directing activities to areas of need. Government efforts at relief and rehabilitation were coordinated through the Ministry of Disaster Management and Relief and its operational body posted at District level, the Disaster Management Bureau. Most observers believe that the newly established Disaster Management Committees at the local level, comprising government officials, NGO representatives and locally elected officials operated adequately to coordinate the disaster response.

At the local level, all relief and rehabilitation activities were centrally authorised by government, through the approval of the Master Role by the TNO (Thana Nirbahi Officer or chief local official) at thana level coordinating committees made up of government officials and NGO representatives. NGOs were able to monitor government relief efforts and to negotiate with officials where abuses were present. Several DEC agencies worked together at the local level to ensure effective coverage, sometimes in areas where no other organisations were working.

Future Coordination Issues

Given the extensive experience of the DEC agencies collectively in disaster response in Bangladesh, there is an opportunity to document and exchange individual agency policy and implementation around specific activities, related to disaster preparedness and response. The following is a partial list of themes or issue areas that would benefit from further coordination among DEC agencies, by reviewing their experiences from the 1998 flood response:

Disaster preparedness and management - collectively, DEC agencies in Bangladesh have a wide experience of disaster preparedness and management, including the capacity to build a response approach at community level. DEC agencies themselves, have varying degrees of capability and priority. Agencies could share this knowledge and develop mechanisms to strengthen partners' preparedness and organizational capacity for disaster responses.

Targeting and coverage - agencies recognise the complexities surrounding appropriate targeting and ways to reach more of the most severely affected persons in a disaster; several initiatives and innovations by DEC agencies could be documented and reviewed.

Housing - given the extensive damage to housing during the flood and the wide variety of housing rehabilitation offered by agencies, it would be useful to document these experiences with a view to developing guidelines of appropriate housing interventions specific to different geographical locations, different types of disaster and varying community preferences; standardisation is not desirable but some greater uniformity in practice for similar situations would be feasible.

Loans, Grants, Local Disaster Funds, Emergency Replacement Funds - agencies had different policies and approaches to the use of grants or loans in the rehabilitation phase. Some loan funds are being used to develop future disaster funds for communities while others are being used to provide emergency credit funds for partner agencies. As this area of response is a new and somewhat uncharted ground for most DEC agencies, the donating public deserves further documentation and assessment by DEC agencies.

Full Report: (pdf.format)

* Get Adobe Acrobat Viewer (free)