Summary report: Formative Evaluation of the Canadian Francophonie Scholarship Program (CFSP) Phase VI - Development Component

Project number: D-00452 / P-000647
Evaluation Type: Formative
Commissioned by: Partnerships for Development Innovation, Education Section, ¶¶ÒùÊÓƵ
Consultant: Baastel Consulting Group, Inc.
Dated: January 31, 2020

Evaluation rationale and purpose

This formative evaluation is an independent exercise aimed at producing reliable, relevant and timely data in order to: (i) report the results of Phase VI (2015–2019) of the Canadian Francophonie Scholarship Program (CFSP); (ii) relay the results of the current phase to the education section (KSI) of the Partnerships for Development Innovation (KFM) so as to obtain approval for another phase of the CFSP; (iii) recommend program improvements based on Canada's Feminist International Assistance Policy as they pertain to gender equality and the empowerment of women and girls.

The main recipients include: (i) ¶¶ÒùÊÓƵ; (ii) the Executing Agency (EA), a consortium comprised of the Canadian Bureau for International Education (CBIE) and World University Service of Canada (WUSC); (iii) the 37 member countries of the Francophonie that receive Official Development Assistance (ODA) from Canada. The evaluation covers all components of the intervention during the 2015–2019 period.

Specific evaluation objectives

The exercise aims to: (i) evaluate the effectiveness and efficiency of the program; (ii) evaluate the relevance and sustainability of program results; (iii) formulate assertions, conclusions, recommendations and lessons to inform the implementation of future iterations, and introduce new measures to improve the program.

The evaluation also has a secondary focus on gender equality and on determining whether the program's basic assumption remains valid to guide another phase of the CFSP, and to determine the positioning and added value of the program in: supporting institutional capacity-building, and promoting gender equality and the empowerment of women and girls in partner countries.

Development context

Announced by the Mulroney government in 1986 at the first Summit of the Francophonie, the CFSP was quickly established the following year. From 1987 to April 2013, when the current phase of the CFSP was defined, the program awarded 2,034 scholarships. Over 76% of these went to applicants from Africa. Each year, an average of 95 scholars were selected from nominations submitted by recipient countries of the CFSP. The scholars enrolled in a variety of schools with French-language curricula – a total of 22 universities and a dozen colleges in six Canadian provinces.

After the approval of Phase V in 2009, the CFSP significantly revised its ultimate outcome from the training of high-calibre scholars to building the institutional capacity of partners in recipient countries. This revision was based on the belief that an effective way to build institutional capacity is to create a cadre of change agents by training individuals who hold high-level positions related to the countries’ priority areas.

Other changes made to the CFSP in 2009 include: (i) the establishment of scholarship quotas per country; (ii) the development of a gender equality strategy; (iii) the establishment of Local Advisory Committees (LAC); (iv) the selection of employed candidates; (v) the improvement of post-training follow-up to promote a greater sense of belonging among CFSP alumni.

The CFSP is currently nearing the end of its sixth phase. The assumption underlying the 2009 revision was only partially reviewed during the previous evaluation. The current phase of the CFSP was approved before Canada's Feminist International Assistance Policy came into effect in 2017; the next phase of the CFSP must now be incorporated into this Policy.

Development intervention

With an initial duration of five years (2015–2019), but extended by one year, Phase VI of CFSP has a $50.5 million operating budget, supplied entirely by ¶¶ÒùÊÓƵ. Of this budget, $45.4 million is reserved for scholarships and $3.9 million for EA file-management and selection services. Administered by ¶¶ÒùÊÓƵ’s Partnerships for Development Innovation Branch, the CFSP operated for a little over 30 years. Between 1987 and 2019, this foreign policy and development instrument awarded nearly 3,000 scholarships to citizens of Francophonie-member developing countries, selected based on their ability to promote the sustainable development of their country.

The CFSP’s long-term goal is to promote the development of Francophone recipient countries by giving priority to the training of women and men trainers, improving the skills of college and university staff in the teaching and research field, and increasing skills and enhancing aptitudes by training specialists and managers in the public and private sectors. Since September 2014, a quota of 256 scholarships per year has been divided among 26 countries in sub-Saharan Africa, four in North Africa and the Middle East, four in Asia and three in the Caribbean.

Intervention rationale

The CFSP is founded on four principles: sustainable development, candidate excellence, variable scholarship duration, and the return to and reintegration of scholars in their home country. The program’s ultimate outcome is to build the institutional capacity of partners in Francophonie-member countries. Its two intermediate outcomes are: (i) the effective reintegration of scholars (women and men) at partner institutions who can transfer their knowledge and skills; (ii) the increased presence of women at the institutions of recipient countries.

The project organization is modeled on the capacity-building cycle. It includes three key periods that benefit from different supports: (i) the selection of candidates, institutions and scholars through a participatory process involving national stakeholders; (ii) supervised training in Canada provided by a set of Canadian Training Institutions (CTIs); (iii) the return and reintegration of scholars, largely contingent on national partners.

The program has a gender equality strategy to promote women's access to the program, to increase their skills and to empower them, as well as a strategy to minimize environmental impacts.

Stakeholders and direct recipients

CBIE-WUSC, an EA selected through a competitive process, manages the project. The training is provided by a set of CTIs offering French-language study programs.

Key partners and recipients play major roles in the CFSP’s implementation: (i) CFSP’s supervising ministries identify the priorities and the LAC oversees the pre-selection of candidates; (ii) home institutions nominate candidates, support their reintegration projects and commit to reintegrating them post-studies; (iii) scholars apply their training to build institutional capacity.

Evaluation approach and methodology

Evaluation approach

The evaluation adopted a systemic perspective based on the collection and analysis of qualitative and quantitative data, and on principles and methods that meet OECD standards. It was conducted under ¶¶ÒùÊÓƵ’s authority.

Methodology

The evaluation criteria are as follows.

These criteria informed a series of questions. An evaluation matrix matched each criterion and question with measurement indicators, as well as the type and source of information sought.

Data collection and analysis techniques

Key information came from a logically built sample encompassing nine CTIs and 13 recipient countries in the Caribbean, Asia and Africa. Data from semi-structured interviews were complemented by a comprehensive review of the literature, the CFSP database, and external annual surveys.

The analytical framework was based on the structure defined by the Logic Model (LM), the Performance Measurement Framework (PMF) and the evaluation matrix architecture. The data collected were linked to identify convergences, challenges and good practices.

Limitations of the evaluation

The evaluation exercise faced three main constraints:

Key findings1

Efficiency

Efficiency

Since 2015, to improve the CFSP’s performance, the EA has developed management tools, including a theory of change, a gender equality strategy and action plan, customized training programs, and a database.

Relevance

In the countries visited, the CFSP is the only scholarship program that explicitly seeks to build institutional capacity by training change agents to actively promote national development. The CFSP offers the possibility of professional and social advancement to deserving managers lacking the resources needed to become specialized in their field.

Since the 2013 merger of the Canadian International Development Agency (CIDA) and the Department of Foreign Affairs and International Trade (DFAIT), the CFSP remains a foreign policy instrument but is now Canada’s main or perhaps only bilateral contribution to the development of developing nations. Because of its strategic direction, the CFSP increases the visibility of Canada’s Feminist International Assistance Policy, announced in 2017. The CFSP’s gender equality criterion has proven to be relevant, and it has enabled the access, success and reintegration of women in decision-making positions.

Through the CFSP, Canada plays a key role within the Francophonie by directly promoting one of its four missions: "support education, training, higher education, and research." Finally, the CFSP highlights the expertise of CTI in partner countries and the international academic community.

Sustainability

Since its creation in 1987, the CFSP has trained over 2,500 women and men from 37 Francophonie countries in over 50 fields of expertise, at over 20 CTIs. It is one of the few cooperation programs with such durability. The CFSP has clear long-term impacts. The fact that the CFSP helps women and men hold strategic institutional positions and is based on each home country’s national training priorities supports these countries over the long term.

Gender equality

By making gender equality a focus of scholar recruitment, the CFSP supports women who wish to participate in the decision-making processes that affect the development of national institutions. The EA’s implementation of affirmative actions supports the achievement of gender equality in the selection of candidates, the success of women and girls, and the recognition of their skills upon their return to their home institutions. With the academic, personal and social skills they acquired, many of the women were able to have their expertise recognized and are in the process of obtaining or are already working in decision-making positions at their home institutions.

Governance

The CFSP is one of the few programs to involve recipient countries in selecting candidates through the establishment of a participatory decision-making mechanism – the LAC.

Several factors hinder the CFSP’s governance: (i) the lack of status of LACs, which are created solely to meet CFSP needs and have no guarantee of stability; (ii) lack of formal national agreements with the CFSP; (iii) varying level of involvement of the Canadian representatives in the target countries.

The environment

The main factors contributing to the scholars’ ability to integrate environmental considerations into their work are: (i) the presence of environment-related development plans, strategies, and budgets in the public institutions; (ii) the organization of training and awareness sessions for the institution's professionals on topics such as environmental assessments, optimal use of resources and renewable energy.

Key conclusions

Efficiency

The CFSP’s immediate and intermediate outcomes have been met or exceeded, as operationally defined by PMF indicators. The ultimate outcome – the institutional capacity-building of partners in recipient countries – remains difficult to measure because: (i) the effects of reintegrating scholars in their home institution takes time; (ii) the majority of Phase VI scholars are still studying in Canada; (iii) there is no formal mechanism to monitor scholars after their training.

An appropriate response to the priority institutional capacity-building needs of recipient countries requires: (i) assessing the status of the institution's needs; (ii) clear and explicit national strategies on the part of the countries regarding the type and level of skills sought through the use of CFSP scholarships, and precise targeting of recipient institutions; (iii) the deployment of measures to not only reintegrate scholars when they return home but to retain them, at least for a period of time that would more or less allow them to implement their knowledge and acquired skills; (iv) the creation of a critical mass of trained agents in the institutions, which would vary depending on the size of the institution and the level of influence of the positions held by these agents.

A more focused positioning would enable the CFSP to better contribute to the achievement of development objectives in the recipient countries’ priority sectors.

Efficiency

In most countries visited by the consultant, the LAC selection process has become increasingly rigorous, democratic and transparent. The final selection made by the EA serves to validate the selection made by the LAC, encourage the participation of women and ensure the compliance of applications and their eligibility by the CTI. This process needs to be improved in other aspects, including the scholarship selection criteria and the immigration procedure.

In countries where there is no official Canadian representation or in countries where there is no "development section" in the embassy, promotion, visibility and Canada’s role in the CFSP, with few exceptions, is ineffective. Of the 37 countries eligible for scholarships, 15, or 40%, did not fill their quotas, indicating the limitations of mechanisms to promote the CFSP, which is partly the responsibility of Canadian representatives.

The CFSP’s main weakness lies in the accountability of stakeholders. Roles and responsibilities are neither clear nor well executed by stakeholders. The program would be more effective if stakeholders were involved in the definition of – and accepted – their roles and responsibilities.

The presence of a significant number of scholars training in Canada at the end of the program poses a management problem that requires immediate attention. Nearly two-thirds of CFSP scholars, the vast majority of them women, will still be enrolled when the Phase VI ends extension in 2020.

Relevance

From Canada’s perspective, the CFSP directly promotes the implementation of Canada's Feminist International Assistance Policy and is a major instrument for highlighting Canada's commitment on the world stage through the Francophonie and for increasing the influence of Canadian higher-education institutions in the international scientific community.

The CFSP meets a need expressed by recipient countries to improve the performance of their institutions and to improve the skills of agents who can promote the development of their institutions.

There is a significant gap between the CFSP’s formal relevance (as part of major policies of Canada and of partner countries) and its programming relevance. Aligning these two dimensions would require revising its basic parameters, including the sectors it covers (private, public, civil society), the principle of awarding scholarships by quotas, training levels, priority support sectors and more generally, the positioning of the program in the recipient countries with respect to existing scholarship programs. This coherence would support the definition of a framework that would provide the CFSP and its stakeholders with strategic power and structure.

Sustainability

The CFSP’s legacy is first and foremost achieved by individuals – women and men who share the experience of a culture, a vision of work, and building knowledge they acquire as part of their stay in Canada. These are the individuals who drive the institutions of their home countries and who become agents of change as their careers progress.

The responsibility for maintaining ties between alumni and promoting a process to renew and continually enhance knowledge and skills acquired through training extends beyond the EA’s mandate. Over the long term, it is up to the institutions, with agreements at various levels, which show the willingness of states to cooperate, institutions to establish partnerships, and individuals to work together at their home institutions.

Canadian and national educational institutions have begun to develop partnerships in Asia and Africa; synergies are being created with Francophonie agencies, and institutions and national public services are engaged in structural change. However, much remains to be done and the Canadian government has a central role to play to: promote exchanges between partners; maintain contacts between scholars; and provide long-term support to maintain this “intangible heritage” of culture and knowledge.

Gender equality

The CFSP actively supports recipient countries in the implementation of their gender equality policies. It promotes women's empowerment and autonomy, and increases their chances of occupying decision-making positions in their home country’s institutions and universities. The CFSP was able to promote equality between women and men at all stages, from promoting the program among recipient countries through LACs to positive efforts that favour the selection of women and their reintegration, as well as their assignment to decision-making positions, and the support of women by the EA during their stay in Canada.

The increased percentage of female candidates stems from efforts undertaken since 2015 to reach gender equality despite the small pool of eligible women at target institutions. The women's success rate and the recognition of their skills by their peers and by CTI supervisors serve as leverage to advance their skills back at their home institutions. However, increasing the number of women in decision-making positions remains a challenge for the CFSP given the persistent inequalities between women and men in most recipient countries, and socio-cultural resistance to occupying such positions, given that men are more educated and hold more leadership positions at their home institutions.

The environment

Although the number of CFSP scholarships related to the environment is relatively small, these scholarships promote the development of significant expertise. Consolidation of expertise in this area is required to create a true policy-dialogue instrument through which Canada could identify capacity-building priorities with partner countries, promote CTI expertise, and provide targeted support to partner countries that will be affected by climate change.

Environmental and sustainable development issues are among the priority action areas of Canada's Feminist International Assistance Policy. As is the case for gender equality issues, the CFSP could become a privileged instrument for promoting sustainable development values by including positive action measures in scholarship criteria.

Lessons

L-1
The transition in approach from an individual to an institutional capacity-building excellence scholarship is a medium to long-term commitment.
L-2
Institutional capacity-building implies a clear and explicit national strategy on the type and level of skills sought using scholarships and the precise targeting of recipient institutions.
L-3
An institutional capacity-building scholarship program is more effective when it combines long-term training programs that promote the development of sustainable inter-institutional links with short-term training programs that meet the recipient institutions’ more immediate strategic needs.
L-4
Institutional capacity-building scholarship programs are more effective when accompanied by complementary measures such as: supporting the development, implementation, and monitoring of the capacity-building plans of the scholar’s institution; and finding synergies with programs that benefit the target institution.
L-5
Increasing the number of women in decision-making positions in the target countries and institutions depends upon the recipient countries’ willingness to implement national gender equality policies and the commitment of institutions to develop and implement positive actions that support gender equality.
L-6
Signing a formal agreement between the Canadian representative in the recipient country and the national authority promotes the participatory and partnership approach of the scholarship program and increases the accountability of the parties.
L-7
A strong Canadian presence in the recipient country facilitates the efficiency and effectiveness of a process that goes from selecting a candidate who fulfills the capacity-building needs of a strategic institution for the development of the recipient country, to the implementation of a change intervention supported by skills acquired through the training of one scholar or a critical mass of scholars
L-8
The transition to a capacity-building approach involves upgrading administrative, financial and technical management tools, including a theory of change, an adapted logic model, a clear definition of stakeholder roles and responsibilities in Canada and in the recipient country, and a support budget monitoring the reintegration of scholars at their institutions.
L-9
The cost-effectiveness of running the launch of a campaign and a screening process in the target countries must be evaluated and adjusted based on the number of scholarships or scholarship years allocated to each country.
L-10
Creating a critical mass of scholars within the same institution who can transfer their knowledge and develop policies, processes and methods requires that the country has a consistent number of scholarship years, and that the identification of recipient scholars is based on the institution’s capacity-building needs.
L-11
The gender equality goal, related to access to and outcomes of the scholarship program, requires: the establishment of incentives to counter family and work-related obstacles; measures to support scholars returning to their home institutions; and gender-specific risk-identification and -mitigation measures.
L-12
The effective reintegration of scholars occurs when the training related directly to an approved project is supported by the home institution, when the institution continues to pay scholar salaries during the training period, and when the scholars collect data in their country with the support of their institution.

Good alternative program practices

Key recommendations

Support the program (under ¶¶ÒùÊÓƵ’s responsibility)

R1.1
Make provisions to allow Phase VI scholars still studying in Canada to complete their degree.
R1.2
Pursue the EA’s policy regarding the promotion of gender equality and extend it to the environment and sustainable development sectors.

Support the Francophonie (under ¶¶ÒùÊÓƵ’s responsibility)

R2.1
Promote the CFSP as Canada’s preferred tool to meet the Francophonie’s mission to support education, training, higher education, and research, namely by targeting a limited number of training areas and adopting a strategy for professional exchanges and closer cultural ties between Francophonie scholars during their training and upon returning to their home countries.
R2.2
Proactively get the CFSP involved in creating synergies between Canadian and national educational institutions and developing strategic alliances with Francophonie institutions such as the AUF, its regional representative and its digital campuses.

Increase the accountability of stakeholders (shared responsibility between ¶¶ÒùÊÓƵ and partners)

R3.1
Formalize the roles and responsibilities of stakeholders in a document covering all stages, from identifying priorities, selecting institutions, and supporting scholars during their training to the implemention of their intervention projects once they return to their home institutions.
R3.2
Clarify ¶¶ÒùÊÓƵ’s expectations and the respective roles and responsibilities of the EA and ¶¶ÒùÊÓƵ regarding the CFSP’s representation in the given countries and communications with national authorities.
R3.3
Help partner countries identify priorities for building better information systems to monitor the implementation of scholars’ reintegration projects, as well as the implementation of the approaches and methods acquired through customized training programs.

Improve the positioning of the program (shared responsibility between ¶¶ÒùÊÓƵ and partners)

R4.1Reduce the number of priorities to create a critical mass of change agents in the institutions and to facilitate monitoring the results achieved.R4.2Encourage leading CFSP countries to create strategies that use available scholarships in keeping with CFSP priorities, while avoiding duplication, and promoting complementarities between the scholarship programs and aligning the PCBF with Government of Canada funding timelines.R4.3Revise the scholarship quota for each country so that all partner countries can access doctoral-level training.

Program-management tools (¶¶ÒùÊÓƵ-EA shared responsibility)

R5.1Thoroughly review the program's logic model and performance measurement framework, and adjust the level of management resources to better align with expected outcomes and program positioning.R5.2Simplify the stakeholder guides and encourage the translation of basic information on the annual campaign into the national language.R5.3Develop tools to guide the LACs and home institutions in how to monitor the reintegration of scholars.R5.4Maintain the database on scholars, include information on reintegration and strengthen its use as an effective decision-making and accountability tool.R5.5Improve analytical and reporting tools to better report on cumulative results against the baseline and targets.

Management responses

The Department took note of the consultant’s findings, conclusions and recommendations and has shared them with relevant stakeholders for consideration.

If you would like a copy of the evaluation report, please contact info@international.gc.ca.

Date modified: