Final evaluation report: Project: "Enhancing social protection by empowering CSOs in Bosnia and Herzegovina" (December 2020)

Consultants: Mirna Dabić Davidović & Zlatko Sarić

Executive summary

This report is the summary of the findings and recommendations by the team of the evaluators (Zlatko Sarić and Mirna Dabić Davidović) within the evaluation of the project “Enhancing Social Protection by Empowering CSOs in Bosnia and Herzegovina. “The main purpose of the evaluation is to capture findings of the process and make recommendations for potential future programmatic engagement in similar actions. The evaluation was carried out in the period October-December 2020.

The project implementation period is from May 2018 until December 2020 and it was carried out in different geographical areas across Bosnia and Herzegovina. The Project was implemented by CARE International in cooperation with 7 partner Civil Society Organisations that work with different marginalized groups.

Methodology of the evaluation: The evaluation process was broad and comprehensive. We tried to obtain a range of information and opinions as wide as possible about the project implementation and the results achieved. The main evaluation criteria were relevance, effectiveness, efficiency, impact and sustainability. The main methods used in the evaluation process were:

• Desk review-analysis of proposals and reports of realized and ongoing projects as well as general reports, policies, strategies, studies and survey about the project and consultancies, including the Baseline Study and regular monitoring data and existing documents from various resources (official, civil society data, shadow reports, media reporting etc.).

• Interviews,

• Focus groups,

• Survey assessing beneficiaries’ satisfaction.

Interviews with 34 persons were conducted during the evaluation process. In total 6 focus groups (28 persons included in focus groups) were conducted, primarily with the project beneficiaries from different backgrounds. With the intention to include as many participants as possible during the evaluation and to gain information from different sources and different surroundings and to conduct a survey assessing beneficiaries’ satisfaction with the services provided by each of the seven target CSOs, and to compare them with similar data from the beginning of the project the Evaluators created the questionnaire which was delivered by e-mail to all relevant beneficiaries. There were two types of questionnaires- one for beneficiaries and another one for organization’s management. The questionnaire assessing beneficiaries’ satisfaction was fulfilled by in total 143 beneficiaries of 7 partner CSOs and 11 members of CSO’s management team.

Findings: The project expected the following results:

1. Strengthened organisational capacity and management of 7 targeted civil society organisations (CSOs) in BiH
2. Improved service provision of 7 targeted CSOs to meet the needs of their target groups
3. Strengthened advocacy position targeted CSOs have strengthened advocacy position and improved policies in specific areas of their mandate.

1. Strengthened organisational capacity and management: In order to achieve the first result, it was necessary to implement different activities which required great engagement both by partner organisations and CARE International. The seven partner organisations were on different levels of capacity development which led to the fact that the results in capacity strengthening were not the same in different organisations. The organisations on lower development level achieved better results in the capacity strengthening process. Nevertheless, the project created space for the inclusion of individual needs of each partner organisation. This approach was well received by the partner CSOs. Strategic plans of all seven organisations were created or updated and they raised the level of their knowledge and skills in reporting process, financial management etc. Development grants were one of the crucial components of the project, as they directly supported the individual needs of each CSOs with an emphasis on their long-term functioning. It proved that the engagement of young graduates through the project was very well accepted. The fact that in five out of seven organisations these persons continued to work even after financial support of the project, shows how meaningful this initiative was. The visit to the Czech Republic provided a unique opportunity for partner organisations to network with similar Czech CSOs as well as to share good practice which could be replicated in Bosnia and Herzegovina. This also resulted in cooperation of some organisations from BiH and Czech Republic.

2. Service provision of 7 targeted CSOs improved: The survey showed that the scope of services expanded by 46% and the satisfaction with the quality of services provided by the CSOs rose by 52%. This increase is a result of the implementation of development grants as well as of the changes that occurred through the processes of strategic planning and the process of mapping of services offered by partner organisations. It is very important that some of the organisations introduced new services and some organisations approached new beneficiary groups for the first time. Four partner CSOs were supported to develop business plans for their social enterprises. This was an opportunity to become familiar with competition, market, market analysis etc. for the first time and to make long-term plans for their activities.

3. Strengthened advocacy position: All seven partner organisations created or updated their communication & advocacy strategies, skills related to different forms of communication improved with the support of hired consultants. The consultants played a significant role and they focused on the individual needs and context of the CSOs. As a result, there is a much bigger number of concrete initiatives and results related to public advocacy in which partner organisations participated. Also, the cooperation with the public institutions improved and some organisations managed to establish themselves as important partners in addressing the challenges in their communities.

Conclusion: According to the evaluation criteria (relevance, effectiveness, efficiency, impact and sustainability), the project achieved its goals to the fullest extent: The increased number of beneficiaries, the increased number of services provided by partner organisations to their beneficiaries, beneficiaries’ satisfaction with the services on considerably higher level, introduction of strategic thinking as a constituent part of the work of the organisations, creation of foundation for future progress through development grants, are results of this project.

Sustainability: The project as a whole has in itself the idea of self-sustainability. To strengthen the capacities of organisations, to think strategically, to meet the true needs of beneficiaries, to strengthen the quality of services provided, to be a part of the process of changes in the legislative framework and in the end to adequately communicate with the whole public are the key elements of self-sustainability. In this context the project drew the roads that lead to self-sustainability of both project results and the organisations themselves and the services they offer to beneficiary groups.

CARE International staff had an important role in the success of this project. With their flexibility and energy, they managed to establish full cooperation with partner organisations. Recommendations that are a part of this evaluation report should be understood as an additional incentive to think about possibilities for future development.

Main recommendations: The evaluation process resulted with several recommendations that can be very important for future projects. The most important ones are:

• The methodology on which the project was based, in fact strengthening of institutional capacities of the organisations and concrete help through grants, proved to be very successful. Having this in mind, the team of evaluators think that the methodological approach, with adequate adding, should be kept

• The recognition and the acceptance of Civil Society Organisations’ needs by CARE International and the donor itself through development grants is something that made this project unique. The evaluators encourage CARE International and donors to keep this approach which among other things means trust in the organisations themselves and accepting the fact that they know best what is necessary to them to improve their work conditions.

• Social entrepreneurship and its development should be one of the focuses of future projects. Marketing with all its specificities, work on legal regulations related to social entrepreneurship are only some of the areas in which there is considerable room for work

• It is still necessary to support the actions of public advocacy and the evaluators think that it is a very good practice to connect the actions with Communication & advocacy plan since by doing this strategic thinking about all such activities gains its value.

• Evaluation process showed that the organisations prefer coaching and mentoring over classic and usual trainings.

• Consultants’ services were one of the crucial elements of the project. But, considering the big number of consultants’ services and the fact that it sometimes caused additional confusion among organisations themselves, it would be good to think about less frequency of the consultants and where it is possible to join Terms of Reference and to engage fewer consultants but with a bigger scope of work.