This evaluation provides an independent assessment of the European Commission's Directorate-General for European Civil Protection and Humanitarian Aid Operations’ (DG ECHO’s) interventions in the humanitarian health sector during the period 2014 to 2016. The evaluation, launched by DG ECHO in November 2016, was carried out by ICF Consulting Services Ltd, with specialist inputs from humanitarian aid and health experts.
The purpose of the evaluation was to analyse DG ECHO’s portfolio of health interventions between 2014-2016, reporting findings against seven core evaluation criteria specified in the Terms of Reference (relevance, coherence, connectedness, effectiveness, efficiency, EU Added Value and sustainability), in order to provide conclusions and recommendations to inform DG ECHO’s future interventions in this area and feed into the comprehensive Humanitarian Aid evaluation currently being undertaken.
Evaluation data sources and methods
Findings presented in this report are based on analysis and triangulation of the following data sources:
HOPE database records for all 553 DG ECHO-funded humanitarian health actions in Third Countries, reported between 2014-2016;
Project reports and FicheOps from a sample of 100 projects;
A sample of 52 Humanitarian Implementation Plans (HIPs);
Background literature from 55 references;
44 semi-structured stakeholder interviews with DG ECHO officers and partners, international donors and development actors;
An online survey gathering feedback from 32 DG ECHO partners (106 respondents);
Three field visits (exploring the External Assigned Revenues – ExAR – programme in Ivory Coast, DG ECHO’s health interventions in Jordan in response to the Syrian conflict; and DG ECHO’s humanitarian health response in South Sudan); and
A research-based case study on the global humanitarian response to the earthquake in Nepal in 2015.
Validity of evaluation results
As with any evaluation, limited data and data inconsistencies in some cases, along with the vested interests of different stakeholder groups may affect the quality and strength of findings.
It was not feasible to conduct a review of the full portfolio of health-focused actions, due to budget constraints, and given the high number of funded projects identified. A purposeful sample of 100 was therefore selected to capture the diversity of healthfocused actions funded by DG ECHO and the diversity of contexts in which the intervention took place.
The HOPE Database was the principle source for extracting health-funded actions. A number of inaccuracies related to defining health sub-categories of actions were found when further analysing the data. Findings within this evaluation report reflect the data extracted from the database, however caution was applied when providing views on the amount of funding provided by DG ECHO to each of the pre-defined subcategories, due to this inaccuracy.
For practical reasons such as time and budget available for the evaluation, as well as security concerns, it was not possible to randomly select sites for fieldwork. The approach to selection of sites for fieldwork was therefore both purposeful and convenient, involving DG ECHO regional health experts and Headquarters (HQ).
As far as possible, methodological limitations were overcome by using complementary research methods to enhance the reliability and validity of the data collected, and to provide the basis for cross-verification and triangulation of the evaluation results.
Caution was exercised when interpreting data and reporting findings, and interests of different stakeholder groups were taken into account to address potential bias and to ensure objectivity. Input, review and validation with external thematic experts contributed to substantiate the validity of the evaluation results. However, in some cases it was not possible to make conclusive findings on the basis of existing evidence: where this was the case, it has been clearly highlighted in the report.
This section provides summary findings from the descriptive analysis of actions, followed by in-depth analysis by evaluation theme. The final component of this section provides a summary of the main findings observed during the field visit to the ExAR Programme in Cote d’Ivoire.
Overview of DG ECHO’s response in 2014-2016
Between 2014 and 2016, DG ECHO funded 553 humanitarian health sector actions in third countries (equivalent to € 616.9 million of funding).
Most funded actions were multi-sectoral – although they had a health focus or health component – combining health activities with nutrition, food security and livelihoods and/or Water, Sanitation and Hygiene (WASH) activities. Breaking down health activities into health sub-sectors, the most common health sub-sectors covered by projects were primary health (67% of projects) followed by medical supplies (51%), reproductive health (48%), community outreach (47%), and, prevention and response to outbreaks/epidemics (42.5%). Just over a quarter of projects (27%) included Mental and Psycho-Social support.
Looking at funding breakdowns for country, partner and target group, South Sudan received the largest amount of humanitarian health funding over this period (€ 73 million or 11.4% of the total budget) while globally, the International Red Cross and Red Crescent Movement was the partner organisation receiving the largest amount of funding (€ 121 million, 18.9%). In relation to target groups for funding, the majority of funding (73%) was provided to support IDPs and refugees. Natural disaster-affected populations received the smallest amount of funding. Nearly three quarters (72%) of DG ECHO funding was provided to projects that incorporated preparedness and response activities to epidemics (equivalent to 42.5% of projects).
DG ECHO’s humanitarian health actions have been relevant to a moderate extent, however better needs assessments would improve relevance of funded actions.
DG ECHO’s field network of Regional Health Experts (RHEs) were key in providing primary, up-to-date data, and, context-specific information to inform DG ECHO’s response strategies in the health sector. RHEs also engaged with DG ECHO implementing partners at design stage, by, for example, informing partners of thematic priorities and defining crisis-specific strategies. Evidence shows, however, that RHEs were not systematically consulted at the stage of the development of the HIPs or by DG ECHO TAs and partners on projects delivering health activities.