Evaluator for a final external evaluation of a multidisciplinary project

Job
from Handicap International - Humanity & Inclusion
Closing date: 01 Nov 2019

1. General information

1.1. About Humanity & Inclusion (HI)

*Our vision:** Outraged at the injustice faced by people with disabilities and vulnerable populations, we aspire to a world of solidarity and inclusion, enriched by our differences, where everyone can live in dignity.

Our mission: HI is an independent and impartial aid organization working in situations of poverty and exclusion, conflict and disaster. We work alongside people with disabilities and vulnerable populations, taking action and bearing witness in order to respond to their essential needs, improve their living conditions and promote respect for their dignity and fundamental rights.

On its 35th anniversary, the Handicap International network changed its name to Humanity & Inclusion.

Our Values: Humanity; Inclusion; Commitment; Integrity

1.2. About Humanity & Inclusion (HI) in the country/region

HI has been operational in Middle-East and responding to various humanitarian crises consistently. HI's coordination office is based in Amman, and coordinates activities in various geographical hubs.

Through its long-established presence and reputation, HI has built strong coordination networks and collaborative working relations with other actors responding to humanitarian crisis. Indeed, HI has been co-leading the PRDWG under the Health Cluster, steering a cross/inter-hub and operational coordination mechanisms with 14 organizations.

HI is part of the current mine action coordination participating to protection coordination meetings and Clearance coordination meetings, and has been a key-actor in the field of Risk Education in the region, having directly reached over 317,000 beneficiaries in 2018. For the clearance component, HI is already well-recognized and in links with all existing Humanitarian Mine Action actors and authorities.

As a key-actor in health and HMA sectors, HI has specialized and highly qualified human resources present both in field and in each operational hub. Furthermore HI has been developing over the years a portfolio of technical resources (training courses, technical guidelines, etc.) adapted to the specific challenges of humanitarian interventions in middle-east and regularly adjusted to take into consideration lessons learnt and needs, which is highly valuable for the implementation of the activities.

2. Evaluation Context

2.1. Brief Presentation of Project

Project title: Emergency and post-emergency response to the basic and specific needs of conflict-affected populations.

Implementation dates: 01-Aug-2018 to 31-Jan-2020 (18 months)

Location/Intervention zones: Two distinct geographical zones (zone 1&2) having two separate cross-border administrative setups

Operational partners: Project is implemented using two modalities: Direct implementation, and through implementing partners with remote management. List of partners will be furnished at later stage.

Target groups: Host Communities; Internally Displaced Persons

Project budget: 8,532,205 €

ECHO direct funding: 6,800,000 €

Project objectives: To contribute to save lives, increase protection and maintain human dignity in conflict-affected zones.

Expected results and indicators:

  • % of people with functional limitations completing their physical rehabilitation follow-up who report an improvement of their functional independence

  • % of people with psychosocial disabilities and persons in psychological distress completing their MHPSS follow- up who report a reduction of sign of distress.

  • % of population declaring having safe behaviors towards explosive hazard

  • % of effective internal and external referrals made to specialized and non-specialized services.

Main activities implemented:

HI plans to target 98,530 beneficiaries, approximately

  • 10,200 people with injuries and/or disabilities and their caregivers will be provided with multi-disciplinary rehabilitation services.

  • 300 persons (in the Center) with psychosocial disabilities and trauma will receive MHPSS standalone services, as a pilot activity.

  • 80,000 crisis-affected individuals will benefit from Risk Education activities, and 680 will receive safety trainings.

  • At the time of writing, while activities are at the inception phase, it has been estimated that HI will operate in 3 to 5 districts, covering 15 villages encompassing 50,000 persons. Please note that the beneficiaries of clearance activities are included already in the counting of Risk Education activities, to avoid double-counting.

  • 3 health implementing partners will benefit from personalized capacity development plans and trainings in an effort to build their independence and ensure their ability to continue responding to the needs of those most vulnerable.:

  • In total, organizations participating in PRDWG, related clusters and working groups will benefit from technical support on inclusion and rehabilitation.

NOTE: Detailed Logical Framework and narrative project documents will be shared upon final selection.

2.2. Reasons for the Evaluation

This end-of-project evaluation was requested by the donor ECHO. The main purpose of this independent end-of-project evaluation is to assess the quality of design, inception and implementation of the project with achievement of its objectives through related activities. This exercise will also inform, assess and advise on the implementation modalities (direct implementation, partnerships and collaborations with health structures). This study will also shed light on the impact of interventions, through partners and to what extent their capacities have been built during this project. It shall also highlight the cross cutting concepts of inclusion, gender, accountability, participation of local communities. Furthermore, this exercise will provide a set of best practices, lessons learned and recommendations for future interventions for continuous improvement of HI programming as per HI Project Quality Framework.

The users of this evaluation will be:

Users:

HI Field: Head of Mission, Head of Programs, MEAL Coordinator and other program and technical staff. At field level: Field Coordinators and project managers

HI HQ: Technical Advisors (MEAL and thematic areas), Operations Officer

External Stakeholders: Donors, and other technical and operational staff at Field, Country or HQ Level

3. Evaluation objectives (3 pages maximum)

3.1. Overall Objective of the evaluation

Assess the project, based on HI Quality Framework which includes but not limited to impact, efficiency, value for money, participation, effectiveness, relevance and sustainability of the implemented project

3.2. Specific Objectives & Scope of Evaluation

The specific objectives of this evaluation are:

  1. Establish the relevance of the project design and implementation

  2. Determine to what extend the project has achieved efficiency in design and implementation.

  3. What are the longer-term impacts and mid-term and short-term effects and how sustainable are these changes?

  4. Provide recommendations to project stakeholders based on identified best practices and improvement areas (lessons learned)

The scope of the external evaluation is to analyze the impact (using mix-method approach) of the implemented project in two geographical zones/hubs. The evaluation should clearly report on:

• Is the implementation of the proposed project consistent with the initial assessment, design and plan?

• Is the project implementation aligned with general and specific objectives stated in the project proposal?

• Does the project respect the work plan proposed to the donor?

• Has the designed project produced the expected impacts through the implemented activities?

• Have the project inputs been converted into project outputs and outcomes, as planned? (Quality and Quantity)

• Does the project respect the main donor guidelines? (correspondence of Objectives-Outcomes – Outputs – Source of Evidence)

• Does the project takes into account and meet the specific needs for persons with disabilities, children, women and elderly?

• According to the achieved Output-Outcomes, should the Organization change its approach or strategy for the upcoming similar project? Any recommendations must be in line with the evolving context of area of operations.

3.3. Evaluation Criteria (HI Quality Framework)

HI subscribes to the Quality Framework define in HI’s Planning, Monitoring and Evaluation Policy, which is based on Development Assistance Committee (Base-HI QUALITY FRAMEWORK) criteria for evaluation:

a. Impact,

b. Effectiveness

c. Efficiency

d. Sustainability,

e. Relevance / Appropriateness,

HI also promotes systematic analysis of the monitoring system and cross cutting issues (gender, inclusion, environment, protection etc.).

All HI external/independent evaluations are expected to use HI Quality Framework throughout the whole evaluation process. In particular, the evaluation must complete the following table and include it as part of the final report.

The evaluator will be expected to use a table (to be provided by HI at a latter stage) to rank the performance of the overall intervention using the HI QUALITY FRAMEWORK criteria. The table should be included either in the executive summary and/or the main body of the report.

The criteria used to rank the performance will be the following ones:

Changes (Impact)

Sustainability

Effectiveness

Efficiency

Value for Money

Relevance

Participation

3.4. Key Evaluation Questions

The following set of questions shall be answered by this evaluation exercise.

A. CHANGES:

· What changes did the project bring out in targeted communities and partners?

· Were there any longer terms changes in targeted communities and partners through various sectors of intervention? Is approach adequate enough for longer term impacts? What are the strengths and weaknesses of various approaches?

· What were the intended and/or planned changes?

· What were the unintended/unplanned changes?

B. EFFECTIVENESS:

· To what extent were the projects objectives achieved? Were the targets realistic? To what extent the targets were achieved?

· Did the outputs lead to the intended outcomes? What is the strength of the relations between outcomes with its corresponding outputs?

C. EFFICIENCY:

· Were stocks and required services available on time and in the right quantities and quality?

· Were activities implemented on schedule and within budget?

· Were the processes of inputs to outputs conversion optimal?

· Were outputs delivered economically?

D. SUSTAINABILITY:

· Are the benefits likely to be maintained for an extended period after the project ends?

· What were the aspects of the project that contributes to sustainability?

E. RELEVANCE:

  1. Were the project objectives consistent with beneficiaries’ needs and context; and with Humanity and Inclusion policies and strategies?

The evaluation report is expected to provide sets of:

a. Best Practices

b. Lessons Learned

c. Recommendations

4. Evaluation methodology

In general the evaluation exercise will follow the following mandatory steps:

  1. Initial Briefing with HI management at Coordination and in the field.

  2. Development of data collection tools in close collaboration with HI management.

  3. Agreement on schedule with specific dates with HI management at coordination and in the field.

  4. Submission and approval of Inception report including data collection tools.

  5. Data Collection in the field

  6. Submission of preliminary findings and workshop with HI management at coordination office in Amman and including HQ. The preliminary findings will be shared with all relevant stakeholders, as advised by the management.

  7. Submission of Draft report

  8. Incorporating stakeholders’ feedback in draft report.

  9. Final draft submission.

Overall quality of the process will be ensured by:

· All the results of the evaluation, as well as an assessment of their limitations, should be made available to the people concerned with the evaluation (All relevant stakeholders as approved by HI Management)

· The results are disseminated through the appropriate channels and in a format that is suited to the recipient (funding body, general public, beneficiaries, experts or other).

· The results of the evaluation are produced and disseminated within the appropriate time-frame according to the use intended for them.

· When disseminating the evaluation report, the commissioning party is bound not to change the nature of the evaluation’s findings.

4.1. Collection Method

The evaluator shall adopt a mixed-method approach where following data collection tools maybe applied.

  1. Secondary data analysis and reviews with existing data management approaches and M&E Systems in place. Currently, the mission has an established M&E department at coordination level and is under process of expansion and strengthening of existing systems.

  2. House-hold Interviews (semi-structured including both open ended and close-ended questions)

  3. Key Informant Interviews

  4. Focus Groups discussion

  5. Phone Interviews (structured and semi-structured)

  6. Observations in the field

POINTS OF ATTENTION

· Please note that all the data collection tools shall cover all HI quality framework points under study

· The field data collection exercise will take place in two distinct geographical areas

a. Zone 1

b. Zone 2

· The proposed methodology should have two approaches:

a. Direct Implementation

b. Implementation through partners with remote management

· Consultants are expected to collect an appropriate range of data using Mix-method approach. This includes (but not limited to):

a. Direct information: Interviews/group discussions with beneficiaries and related stakeholders – observation visit to project sites and to the facilities provided to the beneficiaries

b. Indirect information: Secondary information analysis: including analysis of project monitoring data or of any other relevant statistical data.

4.2. Actors involved in the Evaluation

Management: This evaluation will be coordinated by Coordination office of HI in Amman. The lead focal point for this exercise will be MEAL Coordinator with the support of a steering committee comprising of MEAL Manager, program, technical and support unit staff.

External Stakeholders: Consultant and their team will work with targeted populations, community leaders/representatives, donor, local authorities, other NGO actors in the field, cluster representatives and other related stakeholders.

5. Principles and values

Selected consultant(s) and the team will comply with the following institutional policies and values of Humanity & Inclusion.

  • Adapting participatory approach

  • Respect security policies, protocols and procedures

  • Child Protection Policy

  • Protection of Sexual Exploitation and Abuse Policy

  • Code of Conduct Policy

  • HI Values

Humanity: Our work is underpinned by the value of humanity. We include everyone, without exception and champion each individual’s right to dignity. Our work is guided by respect, benevolence and humility.

Inclusion: We advocate inclusion and participation for everyone, upholding diversity, fairness and individual choices. We value difference.

Commitment: We are resolute, enthusiastic and bold in our commitment to developing tailored, pragmatic and innovative solutions. We take action and rally those around us to fight injustice.

Integrity: We work in an independent, professional, selfless and transparent manner.

6. Expected deliverables and proposed schedule

6.1. Deliverables

The evaluation exercise will have the following expected milestones and deliverables.

1. Inception report specifying the methodology

2. Preliminary findings Workshop with HI Management

3. Draft report and a possible feedback document to provide feedback

4. Final report, with PowerPoint presentation and summary (mandatory)

6.2. Evaluation Schedule

  1. Evaluation briefing with Mission (coordination team (Skype or face-to-face): 1 day

  2. Desk review, preparation of field work in close coordination with evaluation focal point of HI: 3 days

  3. Inception Report (including data collection tools): 1 day

  4. Travel and security brief: 2 days

  5. Geographic Zone 1 (Direct Implementation modality): Field work, which includes collection of primary data, meetings with stakeholders and analysis of available secondary data: 7 days

  6. Geographic Zone 2 (Remote Management modality): Field work, which includes collection of primary data, meetings with stakeholders and analysis of available secondary data: 7 days

  7. Geographic Zone 3 (Implementing partners modality): Field work, which includes collection of primary data, meetings with stakeholders and analysis of available secondary data: 7 days

  8. Debrief Workshop in-country & Presentation of Preliminary findings: 2 days

  9. Return Travel: 1 day

  10. Draft Report: 4 days

  11. Feedback from HI Management and Final Report with Data sets 4 days

Total: 39 working days

7. Resources

7.1. Expertise required from the consultant(s)

  1. Relevant degree / equivalent experience related to the evaluation to be undertaken
  2. Knowledge in health, mental health (rehabilitation) and humanitarian mine action programming with relevant experience for Multi-sectorial and integrated approach
  3. Significant field experience in the evaluation of humanitarian / development projects in similar themes
  4. Significant experience in coordination, design, implementation, monitoring and evaluation of programs
  5. Experience in OECD-HI QUALITY FRAMEWORK Criteria based evaluations.
  6. Sound Statistical and analytical skills.
  7. Good communications skills and experience of workshop facilitation
  8. Ability to write clear and useful reports (may be required to produce examples of previous work)
  9. Fluent in English (Arabic will be an asset)
  10. Understanding of donor requirements
  11. Ability to manage the available time and resources and to work to tight deadlines

7.2. Resources available to the evaluation team (data, document, etc.)

  • Evaluation Toolkit

  • Project Proposal and Log-frame

  • Project Budget

  • Assessments

  • Reports

  • Dataset of beneficiaries (anonymous)

  • Any other related documentation to evaluate the project

  • Set of Institutional policies of HI

  • ESRI based data management technologies (Information Management System for Mine Action) (IMSMA Core)

7.3. Required documentation and resources from consultant(s)

  • Legal and valid registration.

  • Evidence that government taxes have been paid

  • The consultant(s) are responsible for personal/life/travel and health insurance during the evaluation for themselves and their team(s).

  • Proof that the Service Provider has paid social security contributions.

  • Certificate from your tax office or equivalent.

  • The consultant(s) will also provide any necessary materials (including their own laptops) required for the evaluation.

8. Submission of bids

Ø Further information and or any queries that you may have can be directed to email to tender.info@sr.hi.org

Ø Please send your official proposals to email tender.info@sr.hi.org not later than Friday November 1st, 2019 by 16:00 GMT.

Ø For any communication / request of information / submission of proposal, please clearly indicate “ECHO external final evaluation” in the email subject line.

Ø Please note that only short-listed candidate(s) will be contacted / interviewed.

8.1. Technical Proposal

The technical proposal should contain:

  1. Profile / CV of the Consultant(s) and proposed team.

  2. Previous work samples/examples of final evaluation studies with references.

  3. Proposed methodology with three approaches (mandatory to adopt and propose three approaches)

a. Direct Implementation.

b. Remote management.

c. Implementation through partners.

Furthermore, the proposed methodology shall have coverage of all HI Quality Framework points, participatory and mixed-method approach, with clear timelines for all the phases of this study. (Please refer to 6.2: Evaluation Schedule)

8.2. Financial Proposal

The financial proposal should outline:

  1. Total Cost

  2. Cost per day of each contributor;

  3. Additional costs (additional services and documents);

  4. Transport costs (international and local),

  5. logistics costs,

  6. translation costs;

  7. With agreement proposed schedule of payments.

9. SCHEDULE OF PAYMENTS

Upon submitting the proposals the consultants’ agree upon the below given schedule of payments

· 30% upon contract signature

· 40% upon approval of first draft of evaluation report

· 30% upon approval of final draft of the evaluation report with presentation and summary.

All payments will be made upon reception of invoice and signature of the selected consultants, by cheque in USD/EUROS, under the name of the contracting parties. The offer and payments are subject to in-country fiscal regulations applicable and fulfillment of deliverables.

10. Annexes

Following Annexes will be shared with the selected consultant.

1. Evaluation Protocol and Standard procedure

2. Inception Report Template

3. PME Policy

4. Report Template

5. Related documentation of the project (log frame, proposal, budget, reports, assessments etc.)

6. Assessing quality of the Evaluation – Template

7. Child Protection Policy

8. Protection, Sexual, Exploitation and Abuse Policy

9. Code of Conduct Policy

How to apply:

Ø Further information and or any queries that you may have can be directed to email to tender.info@sr.hi.org

Ø Please send your official proposals to email tender.info@sr.hi.org not later than Friday November 1st, 2019 by 16:00 GMT.

Ø For any communication / request of information / submission of proposal, please clearly indicate “ECHO external final evaluation” in the email subject line.

Ø Please note that only short-listed candidate(s) will be contacted / interviewed.