Country: Kenya
Closing date: 02 Jun 2017
Terms of Reference: End of Project Evaluation
Final Evaluation of Kwale Girls Project (KGP)
- Background
1.1 Background to Build Africa
Build Africa is an international NGO with offices in the UK and supporting organisations in Kenya and Uganda. It aims to improve the quality of education and support livelihoods by developing the capacity of communities. Build Africa works with communities stretching across several regions in Kenya enabling school communities to improve the quality of education they provide for their children. Find Build Africa on http://www.build-africa.org for more information.
2. Project Description
KGP Goal and Outcomes
Build Africa’s work in Kwale, Kenya has been to improve the quality of primary education in 72 schools in Msambweni, Lunga Lunga and Kinango Sub Counties focusing on the educational needs of disadvantaged girls, ensuring good quality learning experiences and addressing barriers that stop girls attending, participating and attaining in primary school.
The Kwale Girls’ Project (KGP) supported by Comic Relief and Big Lottery employs a holistic approach that engages girls, boys, teachers and parents in training and advocacy designed to increase confidence, build knowledge and motivate them to lead change.
The total targeted beneficiaries across 72 schools and communities were as follows; 26,020 primary school aged girls, 724 teachers and school staff members, 37,481 parents, boys and community members all within 72 school communities.
2.1. Intended project goal
Overall Goal/Outcome:
Improved education quality resulting in increased educational attainment of girls.
2.2. Intended objectives and outcomes
Specific Outcomes for KGP include;
- Adoption of child-friendly and gender sensitive teaching methods, contributing to improved quality of teaching and increased educational performance for girls.
- Communities value and celebrate the importance of education for girls, creating an environment where they are supported, resulting in improved retention.
- Communities acknowledge and have increased capacity to address violence and abuse against girls, leading to girls feeling protected from violence both in and out of school, resulting in them accessing education within a safe environment.
- Girls have increased knowledge and understanding of the opportunities available to them, improving their attitudes, self-confidence and aspirations while becoming positive role models for other girls in the community.
3. Evaluation purpose, use and evaluation questions
The purpose of the evaluation is use evidence to develop findings on the extent to which the Kwale Girls Project met the intended outcomes, as evaluated in part against baseline data and annual data collected against the baseline at the end of project years 1 and 2. The analysis of the data and the presentation of the findings must also generate learning to inform future design and programming for future education projects of Build Africa.
The end evaluator will therefore secure data to answer the key evaluation questions and from its analysis, generate lessons for the improvement of future similar such efforts.
The key evaluation questions are:
a. To what extent have interventions contributed to the achievement of increased educational outcomes for girls?
b. To what extent has the project been successful in changing parents and communities attitudes towards girls’ education?
- To what extent has the project managed to enhance girls’ knowledge, attitudes, and practices when it comes to life skills such as self-awareness and risk mitigation?
- To what extent has the project been successful in increasing the capacity of communities to violence and abuse against girls?
- To what extent has the project improved teaching practices with regard to gender sensitive pedagogy?
- To what extent have the approaches used in the project been relevant and appropriate in improving education quality?
- To what extent are the project outcomes sustainable beyond the life of the project? The findings, learning and recommendations from the evaluation will be consolidated into a report to be shared with the two main donors (Comic Relief and Big Lottery) and other key stakeholders at the local, national, and international level. It will also be available for internal use.
4. Build Africa Evaluation Quality Standards
Build Africa works to meet evaluation quality standards in line with the OECD Development Assistance Committee (DAC) criteria for evaluating development assistance, combined with the BOND Evidence Principles. These tools provide guidance to the standards that Build Africa and partners should uphold in the entire evaluation process e data collection.
The BOND Evidence Principles specifically focus on the quality of evidence within an evaluation. The standards are as follows:
o Voice and Inclusion: We present beneficiaries’ views on the effects of the intervention, and identify who has been affected and how;
o Appropriateness: We use methods that are justifiable given the nature of the intervention and the purpose of the assessment;
o Triangulation: We make conclusions about the intervention’s effects by using a mix of methods, data sources, and perspectives;
o Contribution: We can show how change happened and explain how we contributed to it; o Transparency: We are open about the data sources and methods used, the results achieved, and the strengths and limitations of the evidence.
5. Evaluation approach and methodology
The evaluation has adopted a design that seeks to assess the effectiveness of the project in reaching the outcomes, and especially to explore the influence of the project in encouraging community participation in the achievement of the outcomes.
The evaluation will employ a mixed-methods approach that consists of establishing the improvements from the baseline, review of project document, literature and records, capturing quantitative data through a questionnaire and qualitative data through the capture of most significant change stories and Rapid Rural Appraisal style Focus Group Discussions. This approach will contribute towards a triangulation of key data sources and indicators within the log frame. The consultant must ensure appropriate beneficiary participation in evaluating the project.
This participatory component of the approach will support the evaluation to develop findings that are evidence-based in the experiences, perceptions and lives of participants.
Study Methodology.
A) Desk Review & consultation within Build Africa
o Review of all project documents, literature relating to project design and implementation including Theory of Change, Log Frame.
o Review of project records, Quarterly and Annual Reports and monitoring data.
o Discussion within Build Africa around the evaluation design.
B) Development of data tools
Based on the evaluation design (approach, methodology and key evaluation questions), the necessary data collection tools should be developed. These will include:
o Tool 1: Semi-structured questionnaire for pupils (girls & boys), teachers, parents
o Tool 2: Most Significant Change (MSC) guide for Key informants.
o Tool 3: Rapid Rural Appraisal (RRA)/Focus Group Discussion (FGDs) for girls, boys, male parents & female parents.
C) Field data collection
Collect the data required to answer the key evaluation questions against the logframe indicators as per year 1 and 2 data collections.
The data collection will include:
o Semi-structured questionnaire: The research team will conduct semi-structured questionnaires with beneficiaries who are pupils, teachers or parents of supported schools. Most significant change tool: The research team will conduct most significant change sessions with individual respondents.
o Rapid Rural Appraisal/Focus Group Discussion: The research team will conduct RRA/FGDs each with participants. These discussions will include interactive and participatory methods of collecting data.
o To ensure accuracy, data will be collected digitally through mobile phones. This will ensure details such as GPRS are also collected besides eliminating errors and ensuring accuracy.
o The selected research team will suggest sample methods and sizes for each outcome within their application and subsequent review of logframe if they are selected to carry out the final data collection and evaluation.
o The total targeted beneficiaries were as follows; 26,020 primary school aged girls, 724 teachers and school staff members, 37,481 parents, boys and community members all within 72 school communities.
D) Data analysis & final report
o Data entry, cleansing and validation conducted by the research team.
o Data analysis conducted by the consultant. The analysis of the questionnaire data will be quite straight forward as many of the questions will be closed/multiple choice, allowing for quick analysis. The qualitative data will be analysed using a content analysis technique to identify findings from the Most Significant Change stories and the RRA/FGDs.
o Draft report to be done by the evaluator for review by BAUK and BAK team o Final report to be developed by evaluator and presented to BAK
6. Tasks and deliverable s
The required consultancy deliverables are:
o One detailed inception report in English, including: a detailed workplan, detailed methodology for both data collection and analysis, sample size calculations (where appropriate), data collection tools, and outline of the structure of the final evaluation report.
o A draft report for comment by KGP staff and Build Africa U.K.
o An evaluation report in English including accurately analysed data, evidence-based findings against the evaluation questions, lessons learned and recommendations.
o All typed data sets in English from both quantitative and qualitative data collection
o A summarised power point presentation for sharing with partners and other stakeholders
The consultant will also be responsible to:
o Attend an inception meeting (remotely if required) with Build Africa Kenya/UK staff to be introduced to the project evaluation, and requirements of the data collection exercise.
o Hire a team of data collectors capable of achieving the high quality standards required by the assignment.
o Review existing and design the data collection tools with key Build Africa Kenya personnel.
o Explore the potential for mobile data collection methods and liaise with Build Africa on the feasibility.
o Pilot/test the tools, and practice administering the tools within the data collection team.
o Supervise and lead the data collection team in the field to collect high quality data – this will amount to the completion agreed questionnaires, most significant change stories and focus group discussions (in line with the proposed sample).
o Supervise the data entry into the formats provided to an accurate and high standard – data entry must be conducted by a data entrant that has not participated in the data collection.
o Analyse all data and write the draft and final reports
o Provide the typed raw data to Build Africa Kenya contact staff and final reports.
Build Africa will be responsible to:
o Share relevant documentation and making it available to the consultant
o Hold an inception meeting (remotely if required) with the consultant to provide background information and to finalise timelines and expectations.
o Provide the log frames for sample size determination.
o Respond to any queries or concerns regarding the data collection tools and/or exercise
o BAK to mobilise relevant project communities and partners in advance and to coordinate the field work to ensure that data collection team are able to capture high quality data from relevant sample of informants
o Review the final dataset and report against agreed timelines
o Approve final report and the short version of the power point presentation o Make payments as against agreed deliverables and timelines set out in the contract
7. Budget
Please submit a full budget including professional fees for all team members including data collectors and all related logistical costs (travel, food and accommodation) for the full exercise.
It is expected that the duration of the evaluation including all preparation, completing an inception report with data tools, all data collection and report writing will not exceed 30 days. The evaluator(s) will be selected based on their relevant experience and expertise, rigor of their design, availability and value for money.
8. Skills and qualifications of consultancy team
The evaluation should consist of candidate(s) that meet the following criteria:
o Master’s degree in a relevant Social Science or related discipline coupled with advanced skills in research methodologies (essential);
o Significant experience of working in senior management roles in education in similar contexts (preferred);
o Demonstrable experience of designing and leading final evaluations of development projects especially primary education (essential);
o Experience in evaluations of projects similar to KGP with detailed knowledge of the Kenya primary education system;
o Excellent knowledge of, and experience of administering, appropriate literacy and numeracy assessments for pupils in the early years of primary education (essential);
o Experience of working with young children, facilitating participatory approaches to data collection (essential);
o Demonstrable experience of leading and supervising high quality data collection exercises (essential);
o Significant skills and experience in quantitative data collection and analysis (essential);
o Experience of collecting, analysing and synthesising qualitative data using a rigorous approach (essential);
o Experience of overseeing and training data collectors, including providing a quality assurance function (essential);
o Able to reliably access the Internet, as well as relevant national policy/planning documents and surveys (essential);
o Excellent communication skills including report writing (essential);
o Capacity to use mobile data collection methods (desirable);
o Experience of working in Kenya or East Africa (essential);
o Fluent in Kiswahili and English in both writing and speaking (essential); o Willingness to familiarize self with, and sign up to abiding by, Build Africa’s Child Protection Policy (essential).
9. Child Safeguarding
It is essential that the successful applicant familiarize self with, and sign up to abiding by, Build Africa’s Child Safeguarding Policy.
We reserve the right to conduct background checks for all shortlisted applicants in line with our child safeguarding policy. This may include collecting references from past employers or commissioning organisations.
How to apply:
- Application process and timeline
This final evaluation is expected to start by mid-June latest with a final report complete by July 17th 2017.
To apply, please complete a copy of the “KGP evaluation: application form” document and submit with a copy of your CV (and the CV of any other proposed consultants for this assignment, if applying as a team) and a detailed budget in Excel
Please do not submit past work or any additional documents with your application or your application will be automatically disqualified.
Interested Consultants should forward an email request for the Application Form via email to lina@build-africakenya.org. Closing date: 2nd June 2017 09.00am UK time / 12 noon East Africa time
All applications to be submitted to hr@build-africakenya.org with a copy to bak@build-africakenya.org