PURPOSE AND OBJECTIVES OF THE EVALUATION
2.1. Rationale for the Evaluation**
This evaluation is conducted as an exercise of accountability towards the donor and the beneficiaries at its final implementation stage. It aims to provide useful insights for programme learning and accountability especially in light of the transitioning process from full scale emergency operations to rehabilitation/reconstruction and resettlement
2.2. Objectives of the Evaluation**
The overarching purpose of the evaluation is to assess the overall performance of the project and to determine if it has achieved its intended outputs and outcomes in the different sectors clearly explaining why or why not through an integrated analysis of the entire result chain (inputs, activities, outputs outcomes and Likelihood of Impact) and contextual factors. The evaluation will use a multi sector approach to determine the efficacy of project based on the DAC criteria (effectiveness, efficiency, relevance, sustainability and impact)as shown below.
2.3. Users of the Evaluation**
Direct users: Action Against Hunger field teams, Technical and Senior Management Teams, Action Against Hunger Technical Advisors/ Director in the HQ (NY), Ministry of Agriculture, NEMA, SEMA, S&FMOH, RUWASA.
Indirect users:** ELA team Action Against Hunger -UK, Action Against Hunger International Network, OFDA and other donors, federal, regional and local governments, ministries, UN agencies and Global Clusters, NGOs and NGO Consortiums as well as humanitarian learning platforms (such as ALNAP).
2.4. Use of the Evaluation**
The evaluation will be used for Learning, improvement and accountability. Lessons learnt (from all sector sof the project), identified good practices and recommendations are expected to be taken on board in designing any other intervention. The evaluation findings and results will inform knowledge management base of Action Against Hunger and will ensure best practices are adopted and applied within all the sectors. Feedback from the evaluation will be used to improve quality of our programs, modification and revision of design based on the lessons learnt. This gains will be utilized across AAH International network.
3. EVALUATION SCOPE**
3.1. Elements covered by the evaluation**
The evaluation will focus on the entire project funded by OFDA, including all its technical areas. Nevertheless, as some of the project’s objectives are less often implemented within Action Against Hunger (e.g. shelter and disease awareness linked to WASH) the evaluation will adopt a multi sectoral approach and establish linkages and interconnectedness within the sectors.The evaluation will also establish
Lessons learnt during the implementation of the project and adopt best practices which can be used to improve the quality and design of future interventions.
The geographical scope of the evaluation will cover the project sites in both Yobe and Borno state. However, simple random sampling of some LGAs and communities/wards will be done in to maximize on the time allocated for the evaluation.
3.2. Cross-cutting issues**
Throughout the evaluation process, gender concerns should be addressed in line with the Action Against Hunger Gender Policy. All data should be disaggregated by sex and age and different needs of women, men, boys and girls should be considered throughout the evaluation process.
4. EVALUATION CRITERIA AND QUESTIONS**
As per Action Against Hunger Evaluation Policy and Guidelines 2015, Action Against Hunger adheres to the Organisation for Economic Co- operation and Development (OECD) Development Assistance Committee (DAC) criteria evaluation approach for evaluating its projects.
In particular, Action Against Hunger uses the following adapted criteria: Design, Relevance/Appropriateness, Coherence, Coverage, Efficiency, Effectiveness, Sustainability and Likelihood of Impact. Action Against Hunger also promotes a systematic analysis of the monitoring system in place within the aforementioned criteria.
Evaluation questions have been developed to help the evaluator assess the project against these criteria (refer to Annex III). The evaluator may adapt the evaluation criteria and questions, but any fundamental changes should be agreed between the evaluation manager and the evaluator and reflected in the inception report.
All independent evaluations carried out in Action Against Hunger are expected to use DAC criteria in data analysis and reporting. In particular, the evaluator must complete the DAC criteria rating table (refer to Annex IV) and include it as part of the final evaluation report.
5. EVALUATION DESIGN AND METHODOLOGY
This is a one-group design evaluation, examining a single project, without including any comparison with people that did not participate in the project. In the event that the endline survey report is available, the evaluation is also expected to discuss changes achieved “before and after**” the project has been in operation.The set of indicators defined in the logical framework will be used to measure the project performance.Information on Incidence and prevalence of high-morbidity rates,Case fatality rates for diarrhea and Rates of admission, default, death, cure, relapse, non-response, transfer and length of stay will be collected using SMART methodology
Outlined below is the suggested methodological approach for the evaluator to collect quantitative and qualitative data and the chronological steps of the evaluation process. The evaluator will to the extent possible develop data gathering instruments and methods which allow collecting sex and age disaggregated data. The instruments need to make provision for the triangulation of data where possible. A mix of qualitative and quantitative tools will be developed for the purpose of this evaluation. The tools to be used should include:
v Household questionnaire
v Focus group discussion
v Key informant interviews
v Direct observations
5.1. Evaluation Briefing**
Prior to the evaluation taking place, the evaluator is expected to participate an evaluation technical briefing with the Evaluation Manager Action Against Hunger -UK, the M&E focal person for the Nigerian Mission and M&E Advisor in the NY headquarters. Briefings by telephone must be agreed in advance.
5.2**. Desk review**
The evaluator will undertake a desk review of project materials, including the project documents and proposals, progress reports, logframe,Monitoring and evaluation plan among other public outputs of the project (such as publications, communication materials, videos, recording etc.), results of any internal planning process and relevant materials from secondary sources (Annex V).
The evaluator should clearly state the sampling approach in terms of sites and beneficiaries. As no access or security issues are expected to hinder representativeness of the entire population (amount of people served), simple random sampling is suggested based on beneficiaries’ project lists, community distribution sites, or health facility sites and mobile clinics,water points . As mentioned in the evaluation scope section, the aim is for the evaluation to be statistically representative of all sites targeted by the program. In case a different sampling approach is proposed or time/access constraints come up, for the sampling of sites, priority should be given to both sites in Borno and Yobe state based on the numbers of activities implemented and beneficiaries served. The criteria for the selection should involve community leaders, volunteers and direct beneficiaries. The sampling approach should be adjusted and further detailed by the evaluator in the inception report.
5.4. Inception Report
At the end of the desk review period and before the field mission, the evaluator will prepare a brief inception report based on the format provided. The report will be written in English and will include the following sections:
Key elements of the Terms of Reference (TORs) to demonstrate that the evaluator will adhere to the TORs;
• The methodological approach to the evaluation include an evaluation matrix in annex to specify how the evaluator will collect data to answer the evaluation questions, pointing out the limitations to the methodology if any and the choice of sites per field visit;
• The data collection tools;A detailed evaluation work plan; and
• Statement of adherence to Action Against Hunger Evaluation Policy and outline the evaluation report format.
The inception report will be discussed and approved by the Evaluation Manager in AAH-UK and the Nigerian Mission M&E unit and shared with other relevant stakeholders.
5.5 Field Mission
Primary data collection techniques
As part of the evaluation, the evaluator will interview key project stakeholders (expatriate/national project staff, local/national representatives, local authorities, humanitarian agencies, or donor representatives) as per the list in Annex VI. The evaluator will use the most suitable format for these interviews as detailed in the inception report. The evaluator is also expected to collect information directly from beneficiaries. Towards enriching triangulation, , The evaluator will also conduct Focus Group Discussions with relevant stakeholder (such as beneficiaries, non-beneficiaries, WASH committees, Health facility staff, Community volunteers, Cash vendors among other groups who will be identified during planning stage), key informants interviews (traditional leaders and community volunteers, etc.) and household surveys.
The evaluator will visit the project sites and the facilities provided to the beneficiaries (if any) according to select methods described in the inception report.
Secondary data collection techniques: Desk review
The evaluator will further review complementary documents and collect project monitoring data or of any other relevant statistical data available. Data from KAP Surveys, PDMs,Cash transfer and surveillances will be analyzed.
Debriefing and stakeholders workshop
The evaluator shall facilitate a learning workshop both in the field and in the national office to present preliminary findings of the evaluation to the project and key stakeholders (including Local, National actors and implementing partners actors); to gather feedback on the findings and build consensus on recommendations; to develop action-oriented workshop statements on lessons learned and proposed improvements for the future.
5.6 Evaluation Report
The evaluation report shall follow the following format and be written in English:
v Cover Page;
v Summary Table** to follow template provided
v Table of Contents**
v List of acronyms
v Executive Summary must be a standalone summary, describing the intervention, main findings of the evaluation, and conclusions and recommendations. This will be no more than 2 pages in length
v Background Information
v Methodology** describe the methodology used, provide evidence of triangulation of data and presents limitations to the methodology
v Findings includes overall assessment of the project against the evaluation criteria, responds to the evaluation questions, all findings are backed up by evidence, cross-cutting issues are mainstreamed and; unintended and unexpected outcomes are also discussed
v Conclusions** are formulated by synthesizing the main findings into statements of merit and worth, judgments are fair, impartial, and consistent with the findings
v Lessons Learnt and Good Practices present lessons that can be applied elsewhere to improve project performance, outcome, or impact and; identify good practices: successful practices from those lessons which are worthy of replication; further develop on one specific good practice to be showcased in the template provided in Annex VII
v Recommendations should be as realistic, operational and pragmatic as possible; that is, they should take careful account of the circumstances currently prevailing in the context of the action, and of the resources available to implement it locally. They should follow logically from conclusions, lessons learned,Monitoring and accountability framework and good practices. The report must specify who needs to take what action and when. Recommendations need to be presented by order of priority
v Annexes should be listed and numbered and must include the following: Good practice template (annex VII), Evaluation Criteria Rating Table (annex IV), list of documents for the desk review (annex V), list of persons interviewed (annex VI), data collection instrument, evaluation TORs
The whole report shall not be longer than 30 pages, 50 pages including annexes. The draft report should be submitted no later than 10 calendar days after departure from the field. The final report will be submitted no later than the end date of the consultancy contract. The reports and annexes will be accepted in the English Language only.
5.7 Debriefing with Evaluation Manager Action Against Hunger -UK**
The evaluator should provide a debriefing to the Evaluation Manager in Action Against Hunger -UK to discuss any issues related to the quality of the evaluation report.
5.8 Debriefing with Action Against Hunger Nigeria Mission and NY HQ
The evaluator should provide a debriefing with the relevant Action Against Hunger HQ on her/his draft evaluation report, and on the main findings, conclusions and recommendations of the evaluation. Relevant comments should be incorporated in the final report.
6. KEY DELIVERABLES**
The following are the evaluation outputs the evaluator will deliver to the Evaluation Manager in Action
Against Hunger -UK:
Presentation of preliminary findings
Draft Evaluation Report
Final Evaluation Report
All outputs must be submitted in English and in Word Document format.
The quality of the inception report and the evaluation report will be assessed by the Evaluation manager in Action Against Hunger -UK. The evaluator is expected to follow the format, structure and length as defined under section 5.4 and 5.6 above.
 The criterion has been rephrased to “Likelihood of Impact” as a thorough impact assessment is linked to the estimation of attribution, which can only be measured through experimental or quasi experimental evaluation designs. The evaluation design for carrying out a performance evaluation would not be suitable to determine the effects attributed to the project.
 The endline will probably be available by the time the evaluator will be in the field.
 At least 24 hrs before the de-briefing with HQ. This presentation should built on the preliminary findings presented at the learning workshop, including the needed adjustments based on feedback received from stakeholders in the field ing:.05aM:
7.2 Profile of the evaluator
The evaluation will be carried out by an international evaluation consultant with the following profile:
v A program management specialist with technical skills in Health,Nutrition,food security and livelihoods
v Over 10 years field experience in evaluation of humanitarian / development projects in conflict and post conflict environment.
v Degree in Health, Engineering, Food security, Public health or Nutrition, with experience in conducting evaluations on the same sectors.
v Significant experience in coordination, design, implementation, monitoring and evaluation of programs;
v Good communications skills and experience in workshop facilitation and presentations.
v Ability to write clear and useful reports (may be required to produce examples of previous work);
v Fluent in English;
v Previous experience in conducting (USAID/OFDA) evaluations
v Ability to manage the available time and resources and to work to tight deadlines;
v Prior experience in Nigeria or knowledge of the NE Nigerian context preferred
v Knowledge OECD) Development Assistance Committee (DAC) criteria evaluation approach for evaluating its projects
v Experience in working in emergency set ups with minimal resources.
HOW TO APPLY:
Interested parties must submit their applications by email referenced under title ‘Consultancy for Final Independent External Evaluation of Action Against Hunger project in Nigeria’ at firstname.lastname@example.org
Deadline for submission: 19th Octobre at 17:00 (GMT+3).
Action Against Hunger will consider applications from individual consultants only, who will be the sole responsible to deliver the outputs of the assignment.
Applications should include:
• Up-to-date curriculum vitae,
• One supporting document of max 3 pages outlining core competencies in line with the assignment, the proposed methodology of work including timeframe and location and the financial expectations.
Only shortlisted applications will be contacted. Shortlisted candidates may be asked to provide references, evidence of past experience, and additional administrative documents.
The final selection of applicants, or cancelation of the assignment, is at the sole discretion of Action Against Hunger.
For more information, prospective consultants may contact Monday to Friday between 9am and 5pm (GMT+3) at the following email address or by phone:
Email: email@example.com or tel: +254 (0)728 65 95 43