Kenya + 1 more

Humanitarian evidence systems mapping in East Africa

Attachments

Executive summary

In November 2014, Development Initiatives was commissioned by DFID to conduct a mapping and political economy study on the production and utilisation of humanitarian evidence in Kenya, Uganda and within relevant East African institutions. Based on interviews, literature sampling and financial analysis, this report presents a summary of a year’s work. It describes the humanitarian research landscape and which factors affect the production and uptake of research outputs. It makes recommendations for linkages that can be strengthened and for interventions that would strengthen national and regional research capacity on both the user and producer side.

Research and evaluation (R&E) appears to have a limited strategic function and value within the humanitarian landscape in the East Africa region. The R&E system tends to operate independently of host governments and local actors at all levels and is driven by donors.

However, there are signs that this is changing and that responsibilities for delivering long-term humanitarian response may be shifting towards regional and national governments and local actors. The limited extent to which national and local policy makers and practitioners value and can engage with R&E outputs and are willing and able to act on their findings, as well as the limited linkages between research and policy and practice communities, are significant impediments to the use of evidence by decision makers in the East Africa region. The lack of a common and shared research agenda for humanitarian R&E in the region, combined with little shared analysis of data/evidence collected over the long term on cyclical causes and responses to repeated humanitarian crises, limits the potential for a strategic and future-focused body of R&E work in this region.

Going forward, there is a need for a locally owned, more strategically coherent research agenda which is broader than the current focus on resilience and which links vulnerability to issues such as energy, water, transport infrastructure, digital communications, climate change adaptation and human security.3 This will more likely gain the attention of governments and prove useful in the longer term for tackling humanitarian crises.

The study’s key findings are:

  1. In practice the governance and coordination of research and evaluation in the humanitarian sector in East Africa is almost non-existent, with multiple, ad hoc, small, short-term initiatives performed by multiple actors. These do not seem to be contributing to a widely recognised body of learning or innovation which is owned or led within the region. Innovations such as the satellite-supported livestock insurance and cash transfers developed in this region five years ago came from donors, research institutions and business.

  2. The dominant factors determining how the R&E environment operates in the humanitarian sector in East Africa are the needs to i) describe and ii) provide some evidence for outcomes and impact, within relatively short timeframes, of specific interventions, as required by head offices of donor governments, UN or INGOs – in order to secure further funding, which in turn perpetuates continued humanitarian action or research activities.
    These requirements lead to intensive internal monitoring and evaluation (M&E) activities conducted alongside programmes which account for 72% of reported evidence output from humanitarian actors, much of which is not publicly available, and to which 0.26% (totalling US$5 million) of humanitarian financial flows is allocated.

  3. Alongside these are a few longer term R&E activities, usually funded by centrally held, specific research budgets of donors. This amounted to US$62 million for all research activities in Kenya and Uganda in 2013 (of which US$28 million was for malaria research).
    United States and European research producers, based outside the region, tend to dominate the longer-term humanitarian R&E, generally producing better- quality outputs, albeit in isolation from the humanitarian implementing community in East Africa.

  4. There is limited coordination between donors on R&E activities in the region, in some cases leading to duplication and questionable value added. In addition, donor procedures and funding cycle practices present a number of constraints to improving the quality and transparency of R&E processes.

  5. There is no widespread application of ethical or technical protocols/research standards in use, neither are there common standards for research and evaluation, reliable and quality longitudinal datasets, or common indicators for resilience. This prevents comparisons across R&E outputs and aggregation of findings, limits enquiry and thus limits learning to improve humanitarian action.

  6. Three-quarters of all respondents (and 61% of regional respondents) said that they thought the quality of humanitarian R&E in the region was poor. Only about one-eighth of all (research for development (R4D)) studies on East Africa were peer-reviewed, compared with one quarter of all East Asian studies. Much is self-published, based on small samples and short timeframes, with limited methodological diversity or rigour. Choice of methodology was only explained in half of studies and limitations were pointed out in one-third of studies sampled. Less than half of sampled studies included some kind of social inclusion, vulnerability or gender analysis. This study found no highly critical evaluations.

  7. There is very limited involvement of local actors in R&E activities beyond enumeration functions and less than 10% of research grants seem to go direct to local institutions. This is a source of considerable frustration for NGOs and local researchers. Poorly performing local research institutions including universities and weak analytical skills among researchers were seen by donors and research institutions as key barriers to joint R&E activities. While local actors acknowledged the need to build analytical skills, they valued acquired practical experience over training programmes or qualifications as a way of improving their skills.
    They also wanted research syntheses, including systematic reviews, research tools and standards.

  8. More is written than read in the region. The majority of respondents (51% online, 68% of key informant interviews (KIIs)) based in East Africa described themselves as both producing and consuming evidence. Host government respondents tended to describe themselves as consumers only. One-third of regional KII respondents and 63% of regional online responses reported that demand for evidence came equally from two sources: from within their own organisations and from donors. This was most likely centred on situation updates and evidence of effectiveness. Responding to this, two-thirds of respondents consequently wanted data and data-gathering tools, and examples of successful programme approaches and impact. One-third expressed varied, deeper and broader interests in understanding the root causes of humanitarian crises and in issues such as conflict, corruption, climate change and urbanisation.

  9. Lack of time, information being too scattered and lack of summaries were the main reasons for not reading R&E studies. The second biggest reason was lack of trust in the quality and credibility of the research. Barriers to research uptake exist at two levels; 1) around the ability of decision makers to both value and understand research outputs (and the consequent responsibility imposed on producers to better target and package their outputs);

  1. the limited engagement and trust between humanitarian researchers and government policy makers to date.
  1. Aside from GIS and satellite data gathering/mapping and a few mobile text response mechanisms, the study found remarkably little evidence of digital data in communications driving change in humanitarian response.6 The study found relatively few political economy studies or longitudinal/retrospective analyses of crises and patterns of humanitarian response and few cost-effectiveness studies.