Attachments
1. EXECUTIVE SUMMARY
Background
CARE, with the financial support from the Austrian Development Agency (ADA), has been implementing a three-year project titled "Water for Food Security, Women's Empowerment and Environmental Protection (SWEEP)" Project in East and West Belesa Woredas of Central Gondar Zone, Amhara Regional State. The SWEEP project was designed to address the socioeconomic and environmental problems causing food insecurity, including inadequate access to water supply and environmental degradation, social barriers and gender inequality, limited livelihood opportunities and low productivity. Therefore, the final evaluation aims to evaluate the project's impact in terms of changes for the intended beneficiaries and provide evidence for future decisions demonstrating accountability to the project beneficiaries, stakeholders, and donors.
Methodology
The final evaluation was guided by project intervention logic and the theory of change and employed the OECD DAC and ADA evaluation criteria. The assessment covered the two intervention woredas and ten kebeles consistent with the baseline and Mid Term Review (MTR) methodologies. A field team consisting of two senior researchers, two qualitative researchers, and ten enumerators were deployed. The evaluation employed a household survey of 869 households using a tablet-based data collection application to assess changes against key outcome and impact indicators. The team visited major stakeholder organisations, conducted 15 key informant interviews using pre-designed checklists, conducted 32 Focus Group Discussions (FGDs) with 234 participants, and conducted six case stories on selected project outcome areas.
Evaluation Results
(1) Impact - Measuring the project's impact at this stage might be difficult since impacts occur sometime after the project's implementation. However, an attempt is made in this evaluation to show the observed impacts of the project in food security, access to safe water supply, the capacity to adapt to environmental and economic shocks, and gender-based violence.
− Food security - The final evaluation result reveals that the project activities are interconnected and contribute to increased food security and marginalized target households' resiliency. This has been evidenced by the livelihood changes attained and reported by communities engaged in Income-Generating Activities (IGAs). The majority of the households (84%) in the final evaluation study said they ensured food security for eight or more months per year. The result is much higher than the MTR result (49%), and none of the households was able to feed their members for five months per year during the baseline.
− Capacity to adapt to environmental and economic shocks- The strategies to cope with economic shocks were selling firewood/charcoal (52% baseline, 18% MTR, and 14% end-line) and migrating to other localities to find work and earn money/food (23% baseline, 24% MTR, and 8% end-line). The reduction in the percentage of households selling firewood/charcoal shows a fundamental shift in protecting the vegetation, which is possibly the result of the awareness raised in environmental management. The decrease in migration could also be due to an increase in income level and improvement in livelihoods. The evaluation result indicated that the household’s capacity to adapt to both economic and environmental shocks has significantly increased – the ability to adjust to economic shocks has risen from 1% during the baseline to 71.4% in the endline survey. Similarly, the capacity to adapt to environmental shocks has increased from 15% during the baseline to 68.6% in the end-line survey.
− Gender-based violence - the SWEEP project has supported the local community to organise into Social Analysis and Action (SAA) groups and begin conversations about existing challenges and ways of mitigating harmful social and gender norms that affect women, girls and marginalized groups. The implementation of SAA has improved the local community's understanding and action in gender equality and in preventing Gender-Based Violence (GBV) and Harmful Traditional Practices (HTPs). There is a significant improvement from the baseline as nearly all the surveyed households (96%) believe early marriage (marriage under 18 years of age) is a harmful practice that affects girls' lives. Prevalence of HTPs and GBV, such as female genital mutilation (FGM), child marriage (CM), sexual harassment, and physical abuse, has shown a significant decrease from the baseline. The final evaluation result revealed that GBV (physical violence and sexual harassment) in the target community is 27.5%, 24% and 69% for baseline and MTR, respectively. Similarly, CM and FGM in the targeted community have shown a continuous decrease (baseline 22%, MTR 9%, and final evaluation5.7%).
(2) Relevance - The project is relevant in many aspects as it has aligned with national and international strategies and policies, and the project activities are consistent with community needs. The project has generally achieved its objectives and needs additional intervention to sustain project outputs and reach more venerable community members in the current non-SWEEP intervention kebeles of the two woredas.
(3) Effectiveness* - The SWEEP project generally achieved all three interrelated project outcomes and all the eight outputs as verified by the logical framework's indicators.
Outcome 1: Improved access to water resources for domestic consumption and productive use and enhanced and sustainable productivity of land for varied uses
Indicator 1: % increase of access to the safe water supply of households in the target Woredas and Kebeles
The project had facilitated access to safe water for the local community by constructing new water schemes and rehabilitation of existing water schemes. During its project lifetime, SWEEP completed the construction of 119 new hand-dug well, 12 solar pump water system, 3 R-masonry dams, rehabilitation of 208non-functional water supply schemes including a dam , and all of them are fully functional and accessible to the community. Also a total of 2,469 household water filtration kits are distributed to households. The project created access to safe water for domestic use and productive uses for 119, 794 target people in the two woredas, 165% of its original plan.
Access to safe water supply for domestic consumption has increased from 26% (baseline) to 55% (final evaluation) across the two intervention woredas. Not only access but also the distance from the sources has significantly reduced –households within less than 30minutes for the round trip from the water point were 22%, 31%, and 57% during baseline, MTR and final evaluation, respectively.
Indicator 2: increase of irrigated land size in 4 Kebeles
The project had a plan to construct four irrigation schemes (two in West Belesa and two in East Belesa) with a potential irrigation capacity of 110ha. While the construction of schemes in West Belesa remained with some finishing works, the projects in East Belesa are completed, started functioning, and 133.7 ha of land are being irrigated.
When all the irrigation schemes started functioning, a total of 329.7ha land will be irrigated.
Indicator 3: % increase of women and girls in targeted Kebeles who spend 8-10 hours/day or less on HH chores
The project attained a significant improvement in reducing women and girls engagement in household chores. The final evaluation result1 revealed that 67% of girls and women (47% and 87.1% for women and girls, respectively) spend less than 10 hours per day that significantly reduced the work burden compared with the baseline (24%).
Outcome 2: Marginalised groups empowered to contribute productively in the household and community
Indicator 1: income increase of marginalised beneficiaries in targeted Kebeles
According to the final evaluation survey result, the average income per household was 29,021 Birr, an increase of many folds compared with 3,400 Birr during the baseline study, and nearly twice the MTR result (15,493 Birr). The dominant source of income was from agricultural activities. The income level varies significantly between woredas,
West Belesa being much higher than the respondents' annual income from East Belesa.
Female Head of Household (FHH) and heads of households with a disability have significantly lower income level compared to the other project target groups. Nonagricultural income-generating activities contributed to about 15% of the household income. The average non-agricultural income differs significantly between woredas (2,211 at East Belesa vs Birr 5,653 at West Belesa).
Indicator 2: % increase of marginalised rural women holding a leadership position in local committees in targeted kebeles
The evaluation result showed that the intervention has significantly improved communities' perception towards women’s ability to hold and play a leadership role in the watershed, WASH and Irrigation Management Committees, and VSLAs in the targeted Kebeles. In comparison, none of the women held a leadership position during the baseline, 52% during the MTR and reached 55% during the final evaluation. They actively participated in the local committee, as witnessed in the end-line assessment.
Indicator 3: % increase of rural women who can equally participate in major income and expenditure decisions in the household in targeted Kebeles
While most of the target women who participated in the final evaluation reported having been consulted on how the income or product earned is to be utilised, only about a quarter of them participated in decisions involving selling and buying livestock (such as oxen, sheep and goats). On the other hand, a transaction involving chicken and eggs is dominantly left to women. Overall, 51% of the target women can equally participate in major income and expenditure decisions in the household, which is a remarkable improvement compared to the baseline (11%) and MTR (38%).
Indicator 4: % increase of improved attitude/perception in communities towards women’s ability to hold and play a leadership role in targeted Kebeles
The project intervention has contributed significantly to building women's selfconfidence to convey their messages in public meetings and their assertiveness in dialogues and decision-making processes. Concerning this, the final evaluation result revealed that 93% of the beneficiary households perceive that target women can hold and play a leadership role that was 56% and 87% during baseline and MTR, respectively.
Outcome 3:
Indicator 1: % increase of beneficiaries who have meaningfully participated in formal (government-led) and informal (civil society-led, private sector-led) decision making spaces
According to the end-line result, 94% of the respondents confirmed that local government considers their participation (10% all the time and 84% occasionally), which altogether shows significant improvement as compared to the situation during the baseline (25%) and MTR (80%).
Indicator 2: % increase of beneficiaries who report that government (Woreda) took their requests into consideration
The final evaluation survey also assessed if the local community's requests are heard and adequately answered by the local government. The majority of respondents confirm that the local government involves community members in planning, budgeting and monitoring for essential social services. Overall, 94% of the respondents who participated in such consultation mentioned local government considers their request and act upon issues accordingly (9% all the time and 85% occasionally). The finding indicates a significant improvement of community participation in the government’s lead planning, budgeting, and monitoring for essential social services compared to the situation during the baseline (30%) and MTR (67%).
Indicator 3: % increase of beneficiaries whose level of satisfaction for government service provision improved
The level of satisfaction of the communities with government service provision has visibly improved. The local community's overall satisfaction towards the government service provisions increased from 6% during the baseline to 42% during the MTR, and it slightly increased to 45% in the end-line survey.
(4) Efficiency- The evaluation has assessed the project's efficiency in terms of how well the various activities translated the available resources into the intended outcome regarding quantity, quality and timeliness. The application of diverse partners' coordinated effort has contributed to efficiency and minimised overlapping and duplication of effort. The project fully achieved joint coordination, technical supports, pulling human and other resources and minimising project implementation cost. Efficient utilization of resources enabled the project to benefit higher number of the initial target VSLA beneficiaries (152%). In relation to the water schemes, 30% of the resources were raised from community contributions.
(5) Sustainability - Several factors contribute to the project's sustainability after the phase-out of the project. The phase-out plan, community and related structures organised and strengthened to take over the project's roles and responsibilities. The active participation of project target groups, local communities, and government institutions in planning, implementing, and monitoring the project's activities. The project built capacities of major stakeholders involved in project. As a result, a sense of ownership on project outputs was created, which facilitates sustainability of the project activities after the phase-out. Detailed analysis of the sustainability of the project is provided in the analysis.