Improving Skills Training for Employment Program - Executive summary

The Improving Skills Training for Employment Program (ISTEP) is a five-year 1 (April 2014 to September 2019) $13 million program in Tanzania funded by ¶¶ÒùÊÓƵ. Colleges and Institutes Canada (CICan) is the executing agency responsible for program implementation, working in conjunction with national stakeholders including the Ministry of Education, Science and Technology (MoEST); the Vocational Education and Training Authority (VETA); and the National Council for Technical Education (NACTE). CICan provides a $1.3-million in-kind contribution to the program, with local partners, Tanzanian technical and vocational education and training (TVET) institutions, MoEST, VETA, NACTE and Tanzanian employer associations providing multiple in-kind contributions.

The ISTEP is designed to build the capacity of Tanzanian technical and vocational training institutions to deliver effective skills for employment programs. Using a collaborative and consultative approach, the program aims to bring Canadian and Tanzanian stakeholders together in partnership to build individual, institutional and system-level capacities among Tanzanian actors. Through CICan, 14 Canadian community colleges are partnering with 11 Tanzanian technical and vocational institutions to develop demand-driven, competency-based programs that will prepare students for the world of work as employees with extractive or tourism operators or as entrepreneurs who can create their own businesses and potentially employ others. Two other Canadian colleges, Seneca College and Douglas College, provide consultancy services to the program.

In 2017, ¶¶ÒùÊÓƵ contracted Salasan Consulting Inc. to conduct a formative evaluation of the ISTEP. The evaluation was carried out by Ron Titus and Ruta Mutakyahwa during October and November 2017. The formative evaluation of the ISTEP was designed to take stock of the results achieved and consider measures to strengthen project implementation. With the evaluation taking place during year four of its implementation, it offers an opportunity to inform both ¶¶ÒùÊÓƵ and CICan of what is working and why, and what needs improvement to strengthen the future implementation of this initiative.

The purpose of the evaluation is to:

The specific objectives of the evaluation were to:

The evaluation approach adhered to a results-based evaluation methodology, aligned with the Organisation for Economic Co-operation and Development - Development Assistance Committee standards and practices for evaluation. Mixed methods supported data triangulation and analysis. Sources were discussed with stakeholders and ¶¶ÒùÊÓƵ to support the Evaluation Team to independently assess and determine the reliability of data in its weighting of findings. Data collection methods (e.g. interview guides) were linked to specific criteria and questions in the evaluation design matrix to ensure that all questions were adequately addressed and that the criteria and each question had multiple and diverse data sources.

Seven Canadian community colleges were visited or were engaged through telephone interviews by the Team Leader, and their four partner Tanzania colleges were visited by the Evaluation Team during field visits. Purposive sampling was selected and approved through discussions with ¶¶ÒùÊÓƵ and CICan, as it was deemed the most appropriate sampling method given the limited time frame and the variance in the number of programs and stages of programs. Because of the sampling strategy used, confidence intervals are not applicable, the sample is subject to bias and findings are not generalizable to institutions that were not visited. A case study approach, however, supported data triangulation and deep analysis of the sites visited.

Guided by the evaluation design matrix and the questions within, data was collected and analyzed, and the findings are presented according to each of the main evaluation categories. As part of the mixed methods analysis of weaving together distinctive qualitative and quantitative data, the Evaluation Team used general inductive analysis as the basis for analyzing the data gathered. Emerging and preliminary findings were shared with ¶¶ÒùÊÓƵ and other key stakeholders identified and approved by the department, to verify and identify any data gaps or additional data sources to be consulted.

Gender considerations were mainstreamed across the data collection strategy, ensuring gender equity in data sources and specific data collection strategies to assess stakeholders’ distinctive capacities in terms of delivery and development of gender-sensitive materials, curricula and policy. Gender equality analysis supported the assessment of sex-based differences in results, as well as the analysis of capacities of various stakeholders, with respect to developing and implementing gender-sensitive curricula, materials and policies in the TVET sector. Governance and environmental sustainability were also mainstreamed across the evaluation approach.

Key findings*

Effectiveness:

  1. The ISTEP has increased collaboration across sector actors (government, industry, training institutions); and within TVET institutions, it has increased abilities to use labour market information, key performance indicators and the Evaluation and Standardization Management Information System ; improved technical and pedagogical capacities; improved leadership and management; strengthened abilities to deliver competency-based training; and increased abilities to attract and retain female and male students.
  2. The ISTEP’s achievements to date are definitively contributing to better coordination and promotion of TVET in Tanzania; enhanced management, design and delivery of demand-driven, gender and environmentally sensitive skills for employment programs for the extractive and tourism sectors in targeted TVET institutions; and the increased availability of technical and vocational graduates for the extractive and tourism sectors.
  3. There was high stakeholder satisfaction with the ISTEP’s achievements and contributions to date, with stakeholders identifying a range of benefits they are receiving from the program.
  4. The program was found to have numerous successes and face several key challenges. A key success of the program’s implementation that was consistently identified across data sources and methods is its model. Two key challenges of program implementation were its lack of a sufficient equipment budget and its short duration.
  5. There were a number of positive unintended outcomes identified during the evaluation process, including outcomes at the individual, institutional and community levels.

Efficiency:

  1. The ISTEP is being achieved on time and budget with no documented inefficiencies in its management or implementation.

Relevance:

  1. The ISTEP’s results were found to be relevant to and addressing the needs and priorities of both primary stakeholders and the country of Tanzania, particularly with respect to its new policy on industrialization.

Sustainability:

  1. Although the sustainability and exit plan has not been updated since the beginning of the project, there is strong local ownership and commitment, and the type of investments made by the program (e.g. building institutional capacities and investments in curriculum) are key sustainability mechanisms suggesting the longevity of program results.

Crosscutting themes:

Gender equality

  1. The ISTEP is achieving its gender equality objectives and is promoting effective gender mainstreaming activities throughout program implementation as designed.

Environmental sustainability

  1. Despite the absence of an environmental strategy, environmental sustainability has been mainstreamed within the ISTEP as can be seen from the program progress reports and witnessed by the Evaluation Team during visits to the TVET institutions.

Governance

  1. While the ISTEP is clearly addressing ¶¶ÒùÊÓƵ’s governance objectives in its design and implementation, CICan does not appear to understand the department’s approach and requirements for governance as a crosscutting theme and has not intentionally mainstreamed or reported on the governance issues to date.

Design considerations

  1. The ISTEP was designed based upon a sound understanding of the local context and lessons learned from previous projects that CICan implemented both in Tanzania and in other developing countries.

Conclusions*

  1. Despite delays in course approvals, the ISTEP is generally meeting its immediate outcomes and well on the way to meeting its intermediate outcomes. The exception is with those Tanzania institutions where resources (human, buildings, equipment, finances) are not adequate to deliver the ISTEP courses developed (Immediate Outcome 1310), such as at Pwani Regional Vocational Training and Services Centre (RVTSC), Shinyanga Vocational Training Centre[SA2]  and Moshi RVTSC.
  2. The ISTEP is being implemented efficiently, with resources or inputs being converted to outputs in an economical manner.
  3. The ISTEP’s results are relevant to primary stakeholders’ needs and priorities. In addition, the ISTEP’s results are relevant to Tanzania’s needs and priorities, particularly the country’s new policy on industrialization.
  4. It is highly likely that the results or benefits of the ISTEP will continue after ¶¶ÒùÊÓƵ’s involvement ends. That is, the immediate outcomes were found to be sustainable.
  5. The ISTEP is contributing to reducing gender-based inequalities in access and participation in technical and vocational education and training.
  6. The ISTEP is achieving results in environmental sustainability.
  7. The ISTEP has achieved some governance results, but intentionality and greater understanding within CICan will enable CICan to[SA3]  better relay these results.
  8. The ISTEP is a well-designed and highly relevant model TVET program for Tanzania and for all stakeholders involved. Its theory of change is found to be sound and appropriate to local needs.

Recommendations*

For ¶¶ÒùÊÓƵ

  1. ¶¶ÒùÊÓƵ should lead in the setting up of a donor coordination group for the TVET sector and a separate TVET working group to include representatives of all TVET stakeholders in Tanzania.
  2. ¶¶ÒùÊÓƵ should ask CICan to record, in their annual progress reports, the in-kind contributions the Tanzanian partners are providing to the ISTEP.

For Colleges and Institutes Canada

  1. CICan should consider hiring a procurement specialist to centralize project procurement processes for the partnerships.
  2. CICan should explore, with VETA and NACTE, how the ISTEP can assist the regulatory bodies to transfer the ISTEP learning[SA4]  to private TVET providers.
  3. CICan should conduct a governance analysis for the ISTEP, following ¶¶ÒùÊÓƵ guidelines; prepare a governance plan for the remainder of the project; implement the plan; and report on the plan’s implementation in the ISTEP progress reports.
  4. CICan should work with its partners to prepare an updated sustainability and exit plan for the ISTEP.

For the Ministry of Education, Science and Technology

  1. All curricula developed through the ISTEP should be shared with all interested institutions of the program.
  2. The NACTE should consider harmonizing its curriculum and courses across institutions.

Lessons*

  1. Success of a project is enhanced when there is a strong partner relationship built on trust, commitment, transparency, shared responsibility and vision of the leadership.
  2. Five years is not a long enough time frame to secure the long-term sustainability of results for a TVET program such as the ISTEP.
  3. TVET projects and programs need to consider their budgeting to ensure that there is sufficient budget for equipment or support of donations of equipment, which may entail reducing the number of institutions supported within a given budget—depending on the specific outcomes of a TVET initiative. Alternatively, other methods of addressing the equipment needs for new courses need to be defined and agreed to between implementing partners and stakeholders prior to the approval of the Project Implementation Plan.
  4. Establishing good working relationships with regulatory bodies and involving them in project activities are keys to facilitating the approval of new curriculum, courses and programs.
  5. International experience is very important for teachers to see what is possible and what is happening outside their home country.
  6. There is a need for TVET institutions to get outside their walls and into the communities they serve, to promote TVET as a viable alternative education and livelihood.

*The key findings, conclusions, recommendations and lessons listed above are those of the consultant and do not necessarily reflect the views of Colleges and Institutes Canada, ¶¶ÒùÊÓƵ or the Government of Canada. ¶¶ÒùÊÓƵ does not guarantee the accuracy of the information provided in this report.

The Department took note of the consultant’s findings, conclusions and recommendations and has shared them with relevant stakeholders for consideration.

Footnotes

1
An amendment process for the CICan-¶¶ÒùÊÓƵ Contribution Agreement to make a one-year no-cost program extension official is currently underway.
Date modified: