Chhavi Kotwani, Search for Common Ground
In peacebuilding, we often face complex settings where the context continues to evolve, the outcomes of our interventions are not defined at the outset, or the project activities are not clear. How do we account for this uncertainty, ensure we are responsive to change, and also measure the contribution of our programming? In such instances, evaluators, managers, and donors can use outcome harvesting as an evaluation approach to retrospectively identify emergent impact by collecting examples of what has changed in “behavior writ large” (actions, relationships, policies, practices) and then work backwards to determine whether, and how, the intervention has contributed to these changes (1). The process begins with harvesters formulating questions that guide the harvest based on the needs of its users, and drafting descriptions of the ‘outcomes’ or changes based on project documentation. They then engage key informants to review these drafted outcomes and formulate additional ones, which they verify with the help of independent third-party individuals. The finalized outcomes are analyzed to inform answers to the guiding questions of the harvest, and discussions are facilitated to promote the use of the resulting findings.
OUTCOME HARVESTING AS A CONTINGENCY PLAN
At Search for Common Ground (Search), outcome harvesting has been an especially useful tool, often as a fallback plan and supplement to traditional linear results-based models such as the logical framework. In Uganda, we adopted outcome harvesting for the Supporting Access to Justice, Fostering Equity and Peace project focused on strengthening peacebuilding and conflict mitigation by improving access to justice in land matters and promoting peace and reconciliation. The logical framework no longer reflected the reality of the context or project activities, both of which had evolved over time and the flexibility afforded by outcome harvesting ensured that the evaluation was responsive to the changing nature of the project. In Kenya, where project Inuka!’s goal was to prevent radicalization and violent extremism by increasing trust and collaboration between the community, government actors, and security forces, the results outlined in the logical framework focused only on the activities conducted by Search and did not adequately capture the role played by grantees and partners. Outcome harvesting was better adapted to local needs and capacities than a traditional final evaluation and empowered local partners to speak about success in their own words and from a local perspective. Lastly, Search employed outcome harvesting for the Youth Community of Practice in Peacebuilding (YCoP) project aimed at empowering Iraqi youth to work in concert to promote sustainable peace in their communities. YCoP spanned only six months and had a limited budget earmarked for the evaluation. In this case, outcome harvesting proved to be the appropriate solution as it was easy to use, did not require extensive resources, and could be adapted to a shorter time frame.
OUTCOME HARVESTING AS A STRATEGY FROM THE OUTSET
While outcome harvesting is valuable as a contingency plan, there is also a strong case to be made for it as a central component of the monitoring and evaluation plan from the outset. The merits of utilizing it as an evaluation tool have been evident from its use to gauge the impact of Search’s advocacy efforts in relation to the farmer-herder conflict in the Sudano-Sahel. Outcome harvesting sought to trace the extent to which Search’s Global Affairs and Partnerships team’s policy advocacy contributed to (i) the level of attention and funding directed at the conflict by the US government, (ii) how key government and civil society actors understood and framed the farmer-herder conflict dynamics, and (iii) influenced the strategic focus of the policy response to the conflict. Engaging in this process helped the team articulate the changes they had observed in a clear manner and link them back to the bigger goals they were working to accomplish. The frequent conversations amongst the team members about the outcome descriptions and analysis also facilitated a shared understanding of their work in Nigeria. They were able to think about their advocacy efforts beyond the time horizon of just a year, how to best invest their time and resources, and how to reshape their goals and expectations to be more realistic and aligned with the pace of policy change.
Search also introduced an adapted approach to outcome harvesting to create a space for reflection in its ongoing collaborative project with Humanity United in South Sudan. As part of this initiative, peacebuilding scholars and experienced practitioners accompany young South Sudanese leaders as the latter collectively innovate and act to transform conflicts in their community. Since the project did not have a predetermined set of objectives or goals from the outset, outcome harvesting has been an ideal tool to capture the emergent changes over the duration of the project. An important component of the monitoring, evaluation, and learning strategy have been the monthly conversations with the peacebuilding experts and practitioners about the changes that they are observing and hearing about through their work with the local peacebuilders. In the weeks leading up to the call, participants record their outcomes in a form that prompts analysis at the individual level about the significance of the changes and the intervention’s contribution to them. The monthly conversations then create an opportunity for implementing partners and donors to come together to pause, discuss their observations at length, and discern the implications for program strategy moving forward. The findings are substantiated using additional tools like surveys and are reflected back to the young peacebuilders on the ground. The design of the process and its scope of enquiry has been shaped by the changing needs of the project and its participants.
STRENGTHS OF OUTCOME HARVESTING
The attributes of outcome harvesting that have contributed to its success as a tool for evaluation and reflection have been its intuitive appeal, flexibility, and its participatory and inclusive approach. In complex environments where objectives and paths to achieve them are largely unpredictable, and predefined theories of change must be modified to respond to changes in the context, it is sometimes more useful to first collect evidence about significant changes that have been observed, and then trace them back to the intervention (2). It is a logical and accessible approach suitable for engaging a variety of stakeholders, and what sets it apart is its deliberate attempt to search for unintended results which might otherwise be neglected. The process allows for flexibility regarding how frequently data is collected, and for multiple iterations of formulating and substantiating outcomes, making it responsive to and reflective of the evolving nature of a project. Outcome harvesting is also strengthened by the participation of all stakeholders who are best positioned to see the changes affected by the project. Ideally they are able to shape the process by helping to formulate the guiding questions of the harvest and by deciding which changes constitute significant outcomes.
CHALLENGES OF OUTCOME HARVESTING
Certain limitations and challenges that are associated with outcome harvesting can also hinder its adoption and implementation. While intuitive, the approach does require skill and time, especially to familiarize participants with a method they are often not well-versed with. Considerable training is required to be able to identify and formulate high-quality outcome descriptions that focus on behaviour change and are adequately detailed, while also ensuring the scope of the analysis continues to respond to the guiding questions. It can also be challenging, especially in complex environments, to establish a clear link between the intervention and the changes observed. Moreover, while the process has done well to create a space for reflection in many cases, more attention needs to be directed towards incorporating the reflection findings back into the project strategy and implementation.
Another important barrier to the widespread adoption of outcome harvesting as a central element of monitoring, evaluation, and learning (MEL) approaches has been the low level of buy-in amongst donors. Donor focus has largely been on logical frameworks associated with results-based approaches, and there is a reluctance to consider alternatives such as outcome harvesting. While they have been open to the approach as a supplement to the logical framework, there have been reservations regarding the rigour and validity of the findings of outcome harvesting. There is potential for change given a current shift among some donors towards MEL approaches that are participatory, iterative, and responsive to programming contexts. Unfortunately, this often does not apply to multilateral and bilateral donors, which account for a significant share of development assistance.
Despite these challenges, outcome harvesting has proven to be valuable in the examples above, not only as a supplementary tool or a contingency plan, but as the central component of the monitoring, reflection and evaluation strategy. An upcoming blog post titled ‘Outcome Harvesting: Best Practices for Learning and Reflection’ looks to consolidate some of the lessons and best practices from Search’s use of outcome harvesting across different projects and contexts.
Read Chhavi Kotwani’s next blog on ‘Outcome Harvesting: Best Practices for Learning and Reflection’.
References:
- Ricardo Wilson-Grau and Heather Britt, Outcome Harvesting (Cairo: Ford Foundation, 2012),
Ricardo Wilson-Grau and Heather Britt, Outcome Harvesting (Cairo: Ford Foundation, 2012),