Evaluation of 3rd Call off of civil society support through umbrella organisations 2013–2017

Executive Summary

This report presents the findings, conclusions and recommendations from the evaluation of the third call of civil society support in Ethiopia through umbrella organisations. The evaluation was commissioned by Sida.

The Ethiopian context is challenging. The 2009 Charities and Societies Proclamation severely restricted the space for civil society organisations to work on rights-based issues and for donors to provide funding for human rights and democracy promotion.
In the third call, five umbrella organisations were funded with a total of MSEK 61 for three years. They in turn selected some hundred sub-grantees/implementing partners.
The projects were supposed to contribute to strengthening the role and capacity of civil society, address marginalised areas and groups, gender equality, women’s and youth’s economic empowerment and harmful traditional practices (HTPs) and gender based violence (GBV).

The main objectives of this evaluation were to document and analyse results – effectiveness, impact and relevance – and their sustainability and to analyse the relevance and effectiveness of these results and modes of working in relation to the new Swedish strategy for development cooperation in Ethiopia.
Findings

Effectiveness

There are many similarities with respect to project design, thematic areas and partners between the five umbrella organisations, but also some differences. Jerusalem Children and Community development Organisation (JeCCDO) exclusively works with community-based organisations (CBOs) that support OVCs, their caregivers/guardians and other vulnerable groups in their communities. Initiative Africa (IA) and its NGO partners promote female students’ access to education through a number of measures. Union of Ethiopian Women Charitable Associations (UEWCA) and Mothers and Children Multi-sectoral Development Organisation (MCMDO) support interventions against FGM and HTPs. UEWCA also engages in, among others, improving the economic capacity of women and in supporting OVC students and their guardians. The Center for Development Initiatives (CDI) attempts to curtail trafficking and GBV.

The evaluation team’s document review, interviews with management representatives of the umbrella organisations and field trips to four regions, Amhara, Tigray, Oromia and SNNPR, where sub-grantees, beneficiaries and local government representatives were interviewed, clearly show that there are results. Women have been assisted with business training and credits to start income-generating activities, OVCs were supported with school materials and girls from low-income families were enabled to aim for higher education. Awareness-raising and other activities addressed genderbased violence, harmful traditional practices and trafficking. The subgrantees/partners have acquired new skills, primarily relating to organisational capacity, in trainings and experience sharing exchanges.

The evaluation aimed to assess the effectiveness of the interventions, but magnitude and quality of the achievements were often difficult to assess. There were several reasons for this, mainly related to incoherent reporting, lack of systematic monitoring and limited analysis.
Some of the umbrellas claim to have achieved amazing reductions of HTPs.
Comprehensive evidence to back these claims is, however, lacking.

IA and its partners had developed a results matrix with objectives and indicators that were used to monitor achievements. However, as the results were at output level, more long-term effects were not captured. Due to the short programme period this would probably have been difficult anyhow.
An observation that seems to be relevant for all the five umbrellas and their subgrantees/partners is that the results have been achieved for the immediate target groups while there are few effects at overall societal level, benefiting more destitute women, OVCs that have dropped out of school and victims of HTPs than those who participate in the project activities. One example of such effects at system level is, however, IA and ETA’s work with a school violence index, which the Ministry of Education has agreed to use in all schools in the country.

Impact

Despite restrictive legislation, sub-grantees at local level have managed to establish relationships of trust and confidence with relevant government offices in woredas and kebeles. There are examples of this interaction having led to increased accountability of government officials at local level. Sub-grantees were also able to exploit grey zones of the legislation, for example through cooperating with local government bureaus on rights-related trainings. At the same time, however, some CSOs struggled to maintain their independence and avoid co-optation.

Relevance

In relation to the topics mentioned in the call for proposals, the support has been relevant. Almost 9,000 women have been economically empowered, about 28,000 girls have studied instead of being married off, 5,500 OVCs have been supported, and street children have returned to school. CSOs have been strengthened and especially the CBOs have developed, from funeral societies to development actors. Some organisations – and their beneficiaries - have been empowered to take on a more active role in their communities. Thereby they have contributed to voices of marginalised groups being heard and a more pluralistic society.

What a human rights-based approach (HRBA) entails is unclear, both due to the Ethiopian context and the confusion of human rights as an objective and the rights perspective as an approach. Many CSOs claim that they are applying a rights-based approach, but this mostly means that they work on some rights, framing the issue in a neutral language, or vaguely refer to equality, dialogue and related issues.

Relevance and Effectiveness for the New Swedish Strategy

The objectives of the organisations and the results they have achieved are relevant for the new Swedish strategy for development cooperation with Ethiopia 2016-2020, primarily for expected contributions under the area “Strengthened democracy and gender equality and greater respect for human rights”. Their modes of work are also relevant for the strategy areas “opportunities and tools to enable poor people to improve their living conditions” and “better environment”. The latter area has the advantage of being less sensitive in the restrictive legal environment in Ethiopia.
However, limited space for civil society is not the only obstacle to work for strategic results and policy change. Increased civil society interventions for such transformations would also require analysis and strategic thinking as well as a longterm perspective.

Recommendations

Among the recommendations to the umbrella organisations are that they should initiate a process where plans and strategies for influencing policy making are elaborated. Umbrellas and sub-grantees would benefit from more clearly trying to delineate their agenda from the government’s agenda. To better account for their resource use, as well as for learning purposes, the organisations should develop and implement monitoring and evaluation systems that are useful for all stakeholders, simple and easy to use. Efforts should also be made to operationalise the concept of HRBA.

The recommendations to the Embassy/Sida include considering the advantages and disadvantages with different types of partners - CBOs, mass-based and developmental and NGOs. Capacity development should be provided to the umbrellas and the programme period should be extended. Furthermore, efforts should be made to promote exchange with similar support programmes, such as the CSSP. The Embassy/Sida should also, together with the umbrella organisations, explore the possibilities of conducting a study on the long term effects of the support on different categories of beneficiaries.