Evaluation of Supporting flood Forecast-based Action and Learning (SUFAL) Project in the 2020 Monsoon Floods, July 2021

EXECUTIVE SUMMARY

Background: ‘Supporting flood Forecast-based Action and Learning’ (SUFAL) project was designed to contribute to reducing the adverse impacts of the increasing frequency of catastrophic flooding on the vulnerable and poor communities through Forecast-based Action (FbA). The project was funded by The Directorate-General for European Civil Protection and Humanitarian Aid Operations (ECHO) and was implemented through a consortium led by CARE Bangladesh, with Concern Worldwide, Islamic Relief and Regional Integrated Multi-Hazard Early Warning System for Africa and Asia (RIMES). The project was implemented in three northern districts of Bangladesh: Jamalpur, Gaibandha, and Kurigram. FbA contributed to disseminating Flood Early Warning messages with a lead time of 10 – 15 days with timely and accurate weather forecast information, while and it also helping to identify potential flooding areas.

Methodology: The primary purpose of the study was to “Evaluate the impact of early actions” applied through the SUFAL project on household and community beneficiaries in responding to the 2020 monsoon floods. Customized OECD-DAC criteria, Quasi-experimental design (Difference-in- Difference Method), Knowledge, Attitude and Practices (KAP) framework and Value for Money (VfM) framework were used as guiding methods and tools to design study instruments and evaluate the impact of early actions at every stakeholder level. The study covered a control group in non-project areas and three treatment groups in the project areas: Treatment group 1 (EWM support), Treatment group 2 (EWM + Evacuation + Shelter + WASH support), Treatment group 3 (EWM + Evacuation + Shelter + WASH + Cash-grant support). Treatment groups were categorized in three different groups to conduct cost-effectiveness analysis. The study areas were in the districts of Kurigram (Hatia, Begumganj, Buraburi, Shaheber Alga unions), Gaibandha (Bharatkhali, Saghata, Ghuridaha, Haldia unions) and Jamalpur (Kulkandi, Chinaduli, Noarpara, Shapdhor. The survey sample consisted of 224 control respondents (of which 153 were women) and 754 treatment respondents (of which 426 were women), among which Sample for treatment group 1, 2, and 3 were 293 (100 women), 292 (192 women) and 169 (134 women), respectively. A total of 118 of the 754 treatment households interviewed through the survey were women-headed households and 38 out of 224 control group households were women headed households. The team had conducted 7 FGDs with community members in the three implementation areas, and 27 KIIs with community volunteers, project staff, government officials, and other related NGOs.

Demographic information: In case of the treatment group, the average age of a female respondent was 44 years, and the average age of a male respondent was 46. In case of the control group, the average age of a female respondent was 43 years, and the average age of a male respondent was 45 years. The average annual income for the households was found to be BDT 110,732 for treatment group respondents and BDT 98,724 for control group respondents.

FINDINGS

Relevance and Timeliness: Consortium Partners identified 12 unions under three target upazilas based on level of vulnerability to floods and other factors for the pilot of FbA and subsequently the foundation training and inception workshops at regional and district level had been conducted to introduce key actions to stakeholders and to identify detailed strategies to implement the project. To reduce the gaps and address the capacity needs of the different stakeholder groups at institutional level related to FbA, the SUFAL team had drafted the forecast-based early action matrix with focal points from Department of Disaster Management (DDM) under the Ministry of Disaster Management and Relief (MoDMR) and Flood Forecasting and Warning Centre (FFWC) under the Bangladesh Water Development Board (BWDB).

There were quite a few pre-existing gaps at the institutional and service provision level that had been addressed through the SUFAL intervention. There had been a prevalent lack of clarity and accuracy of forecast information and actionable forecasts. To address this gap, SUFAL had increased the accuracy of EWMs through developing Upazila-level vulnerability maps, installing gauges, and developing Digital Elevation Model (DEM) with technical support from RIMES.

DMCs and LGIs also lacked organizational capacity in decision making and triggering early actions which was addressed by strengthening capacity of organizations in decision making and triggering forecast based early actions, sensitizing and building capacity through providing training to DMCs and LGIs to understand FbA concept and mechanism, and introducing concept of FbA to National level DDM.

There was a lack of Standard Operating Procedure, and clear direction of responsibilities, and DMCs and LGIs had lack of clarity and guidance in their roles and responsibilities related to flood response, and a list of early actions before the flood. These gaps were addressed by developing Early Action Matrix and List of Early Actions, defining flood triggers, Early Action Matrix and preparing list of early actions through discussions with national stakeholders, local government officials, and community people.

EW information and forecast based action supports failed to reach vulnerable stakeholder groups as public announcements were given when the flood already hit and public supports were mostly provided during and after the floods. This was addressed by disseminating early warning message to the relevant government and LGI officials, community volunteers, prominent community members and at household level. They had also provided cash grant to the most vulnerable community members, engaged community volunteers to enhance early warning message dissemination and forecast based actions, and targeting women and other vulnerable community members through the intervention.

Lastly, there had been inadequate budgetary allocations for Local Administrations for taking forecast based early actions. To address this gap, the team had supported LGIs/DMCs in increasing evacuation boats, shelter renovation, setting tents/ temporary shelters, repairment of roads and embankments, and building temporary walkways, shelter renovation and repairment was completed about 10 – 15 days before the first wave, temporary shade for livestock at flood shelters had been built, additional latrines and tube-wells had been installed, electricity supply had been ensured, and women in some shelters were given hygiene support and kits.

Effectiveness: Most of the activities had been conducted as per the plans of the intervention design, FFWC had introduced 15-day flood outlook in 2020 monsoon, with support of USAID-funded SHOUHARDO through CARE-RIMES, which was utilized by SUFAL project, the DMC and LGI officials had developed a good understanding of the importance and benefits of FbA, the DMCs/LGIs sat for coordination meeting to decide on strategies for their flood response after receiving EWMs, the LGI stakeholders had been able to present their required budget for flood response in 2020 flood around 4 to 7 days before flood, and LGIs/ DMCs were found to be using Digital Bulletin Board, Forecast-based Early Action (FbA) Matrix and List of early actions made in 2020 in upcoming flood preparation (2021) during evaluation. The DMCs/LGIs had strengthened their services to communities to respond to floods through FbA through arranging public announcements of EWMs to communities through miking and their ground level workforce, DMCs had deployed more boats for evacuation support, repaired roads and embankments, and built temporary walkways and some LGIs had increased their supports to some of the shelters.

There has been increased access to and acceptability for the early warning messages. A total of 8,800 households received EWMs through mobile phone voice messaging and 39,729 households were reached through the community volunteers and loudspeaker announcements. In 2019, most of the respondents did not receive accurate EWMs. Hence, most of them disregarded EWMs before the first wave, but the trust and perception on information accuracy significantly increased after seeing the accuracy of the information after first wave of flood. Around 97% of the treatment respondents and 37% of the control respondents said to have received the early warning messages. The tendency for taking early actions increased among treatment households as 51% of treatment households started taking early actions immediately after receiving EWMs and more than 80% after second wave and 100% of the respondents said that early actions they took after receiving EWMs were useful.

Impact: It was found through the study that less people in treatment group experienced damages compared to control households, treatment households saved more resources in 2020 than control households, and the average monetary values of assets saved by treatment group in 2020 were higher compared to the control group households. Due to the drawn-out duration and intensity of the flood in 2020, respondents reported that they were not able to prevent more damages although they took more early actions. Besides, treatment areas were the most flood affected areas. The early messages had helped the community to prevent damage to their assets and livelihoods. The percentage of damage prevented in agricultural sector for the treatment group had increased to 28% since the flood of 2019. The damage prevented in fisheries had increased significantly by 18 percent in 2020 in compared to that of 2019. The death of family members from waterborne diseases had decreased (except female members) in comparison to the previous flood in 2019. It is quite evident that the early warning message had enabled the males to take early actions regarding relocation of the vulnerable family members to higher grounds, relative’s houses, or to the shelters. The cash for work modality had also helped the community people to obtain a source of income by working for the embankment, roads, bamboo bridges, etc. Shelter renovations and upgradations reportedly encouraged the community people to evacuate faster. The average amount of loan taken by a treatment respondent and control respondent was found to have been Taka 20,194 and Taka 18,335 respectively. However, post flood loan burden was significantly less for the cash grant recipients (only 32% took loan after flood) as compared to other treatment groups (more than 50% took loan). The cash grants are said to have helped the recipients address their basic needs during the flood and also helped them to some extent to repair their house and pay for livestock treatment after the flood.

Efficiency: Two types of cost-effectiveness had been calculated- one with program cost (excluding management cost and staff cost) and another with total cost of intervention. For SUFAL project, €117 of benefit (calculated as the monetary value of damage prevented by treatment households due to taking early actions in 2020) was generated against each euro with respect to program cost. With total cost, each euro generated benefit of €38. The cost-effectiveness analysis also shows that the most vulnerable people are those who received cash grant from the project (treatment group 3) prevented the least amount of damage from flood. Therefore, these vulnerable group needs to be focused intensively on upcoming development projects.

As external factors to impede the efficiency of the project, the pandemic and ‘excessive relief dependence’ have been identified. The COVID-19 pandemic hit the project hard and substantially disrupted intervention activities and so the field level project staff got minimal time to prepare before flood because of the pandemic restrictions.

Sustainability: It is difficult to determine the level of institutional ownership and sustainability of changes at this stage of this pilot project. However, some early signs clearly show positive changes towards institutional ownership and sustainability. The LGIs, UzDMCs, UDMCs, DDMCs expressed satisfaction and appreciation with the FbA and accuracy of EWMs and expressed willingness to continue using project facilitated knowledge products - Forecast-based Early Action (FbA) matrix, List of early actions, digital bulletin board and EW voice messages. In 2020, all the UPs and DMCs reported that they had asked for flood response budget earlier than previous years and started flood emergency coordination meetings as soon as they received the EW messages. All the Union Parishads were found to be in the process of planning to create a new fund to respond to early actions. Some of the government stakeholders (Union Parishads, LGIs) reported that they are planning to allot separate budget, resources and activities to respond to next floods following FbA.

Replicability: As per the understanding of FbA and findings from the field, the model can be implemented and scaled up work for other flood prone areas in Bangladesh as well, considering ground level vulnerability mapping at ward level. Since the FbA approach can be customized along with its early action matrix and early action lists according to area-based needs, it has potential to be applied in other areas also. FbA model can be applicable for other disasters like river erosion if the possible time and area can be forecasted. Early warning messages regarding evacuation, health, sanitation, cash grant and other support are relevant to other disasters as well. Early sign of change in working modality of other NGOs is already surfacing in the target areas.

LEARNINGS AND RECOMMENDATIONS

Process and system level:

• Early actions have been already tested in SUFAL 1. Considering the success of piloted SUFAL in forecast based early actions, the project should be continued in existing and new areas as well. SUFAL should continue to advocate with DMCs to take ownership of this model and implement it in other new locations.

• Though treatment group 1 generated most benefits (more damage control), in terms of pro- poorness and inclusiveness, it is imperative to provide more focus and support to the more disadvantaged (treatment) group 3 and 2.

• To promote the FbA concept and boost contingency planning for early action by government administrative bodies, it is imperative that the fund flow from central to grassroots is strengthened electronically within both the government and private sector financial institutions.

• The project should support the technical service providers e.g., FFWC, BMD, DDM and RIMES in maintaining network and coordinated research on how to calculate and relay earlier lead times i.e. by strengthening and improving accuracy for the multi-hazard early warning system.

• SUFAL should also explore other areas of early actions in future. More attention needs to be given on inclusion of public health component in the FbA approach to decrease mortality and morbidity rates due to waterborne diseases in target areas.

• Besides, SUFAL should work more on early actions especially in agriculture and livestock subsectors as damages were higher in those subsectors.

Local administration, sectoral and central level:

• SOP development and finalization should be the main focus going forward for scaling up. SUFAL should support national task force (in developing protocols, guidelines, financing) and sectoral departments for the development of SOP and then to implement forecast-based action.

• Advocacy is required for dedicated allocation for early action from general disaster management funds both at national level (DDM, sectors) and at local level (administration, sectors). Most of fund allocation is now for relief based or post disaster damage management which needs to be changed.

• Horizontal coordination at upazila level among local public offices and vertical coordination between national and local level administrations should be strengthened through further advocacy and sensitization activities. This would enable more fluidity in decision making and mobilization of resources before, during and after floods that would immensely contribute to the purpose of helping the community people to be safer.

• Capacity building sessions for LGIs, DMCs, community volunteers should be conducted face- to-face and over a longer duration for maximum retention.

• Linkage between FbA and safety net programmes should be further explored (especially with SSNP’s employment generation programmes and cash for vulnerable people.

• Furthermore, it is recommended to advocate with the government and donors to establish a joint multi-year, forecast-based monitoring grant which will work towards integrating the FbA/EWS system into regular disaster preparedness work.

Community Level:

• The distribution of early warning message recipients had been uneven in many areas as a majority of the population of a certain area may have received the message while only a handful of people had received message in the neighboring area. This can be solved by selected small community unit, such as a neighborhood or para, first, and then ensure to select people from each neighborhood/para.

• To ensure more equitable access to EWMs of women, the intervention can include community level yard meetings and other such interventions to make the women aware and literate of basic mobile operating process.

• The intervention design should focus on institutionalizing and engaging the community volunteers throughout the year in phases instead of just engaging with them during floods for rescue and early warning message dissemination.

• The Flood Preparedness Program (FPP) is being implemented by national Resilience Program (NRP), and Community Preparedness Program (CPP) is being implemented by BDRCS. SUFAL can examine how it can utilize learning components from FPP and CPP for enhancing forecast based early actions, especially in strengthening community preparedness and institutionalizing community volunteers.

• The project could ensure the retention of the community volunteers through engaging younger people who are old enough be volunteers and are still studying in high school or college, thus staying in the locality.

• In addition, young married women (as there would be less risk that they would get married and leave their communities) with some literacy can also be engaged as volunteers who can easily access to women at households to people’s yard.

• The hands-on training sessions, yard meetings, showing docudrama, etc. would help to make the concept of early action more easily acceptable to the community people and would help them to sustain the activities after the project ends if the early actions would lead to reduced damage.

• Many people were found to have refrained from going to the shelters during the flood as they felt that their families, assets, and livestock would not be as secure as they/those would be in their relatives’ houses or at the embankments. Either the project could engage the police in the shelters (and also in the embankments where livestock are kept during flood) and offer patrolling and guarding of their livestock, assets, and especially the female members of their households. In summary, some form of patrolling and guarding services needs to be introduced into the shelters to motivate more people to avail of the shelters during floods.