¶¶ÒùÊÓƵ

Evaluation of the Canadian International Innovation Program (CIIP) 2015-16 to 2018-19

Final Report

Diplomacy, Trade and Corporate Evaluation Division (PRE)

¶¶ÒùÊÓƵ 

February 2020

List of Acronyms, Abbreviations and Symbols

AAFC

Agriculture and Agri-Food Canada

BFM

International Business Development, Investment and Innovation Branch

BID

Director General of Investment and Innovation Bureau

BII

Investment and Innovation Bureau

CAD

Canadian Dollars

CBS

Canada Based Staff

CFPs

Calls for Proposals

CIIP

Canadian International Innovation Program

CIIRDF

Canada-Israel Industrial Research and Development Foundation

CPI

Consumer Price Index

CXI

CanExport Innovation Program

FDI

Foreign Direct Investment

FTA

Free Trade Agreement

FTE

Full-time Employees

FY

Fiscal Year

GGI

Going Global International

GoC

Government of Canada

HQ

¶¶ÒùÊÓƵ Head Quarters

IBD

International Business Development

IIA

Israel Innovation Authority

ITA

Industrial Technology Advisor

ISED

Innovation, Science and Economic Development Canada

JC

Joint Committees

NRCan

Natural Resources Canada

NRC-IRAP

National Research Council of Canada Industrial Research Assistance Program

OECD

Organization for Economic Co-operation and Development

OGDs

Other Government Departments

OGM

Asia Pacific Branch

PDA

Partnership Development Activities

PRE

Diplomacy, Trade and Corporate Evaluation Division

R&D

Research and Development

SMEs

Small and Medium sized Enterprises

SSHRC

Social Sciences and Humanities Research Council

S&T

Science and Technology

ST&I

Science, Technology and Innovation

TB

Treasury Board

TBS

Treasury Board of Canada Secretariat

TC

Trade Commissioners

TCS

Trade Commissioner Services

Executive Summary

This evaluation looked at the Canadian International Innovation Program (CIIP) between FY2015-16 and FY2018-19. The evaluation was conducted in-house by the Diplomacy, Trade and Corporate Evaluation Division (PRE) between February and December 2019, and relies on interview data, survey data, project data and a document review.

The evaluation team found that Partnership Development Activities (PDAs) are successful at increasing awareness of Canadian small and medium enterprises (SMEs) about opportunities abroad. In the majority of cases, participation in a PDA led to discussions between Canadian SMEs and foreign counterparts on collaborative research and development (R&D) projects. PDAs were also seen by Canadian stakeholders as contributing to broader results for Canada such as international business development success and to strengthening the reputation of the country’s innovation ecosystem.

While it is still too early to see commercialization results from projects that were funded during the evaluation reference period, the R&D funding component of the Canadian International Innovation Program (CIIP) has been successful at generating interest from foreign companies into the Canadian innovation ecosystem, and vice-versa. Although commercialization results will not be available for a few years, the CIIP has contributed to Science, Technology and Innovation (ST&I) outcomes, and supported Canadian commercial interests and priorities in countries of focus.

Canada has ST&I agreements with a wide variety of countries, including those that have the highest innovation potential according to international rankings. While the CIIP is not focused on the countries with the highest innovation potential, the current geographic focus of the program remains relevant to Canada’s commercial interests and trade priorities. 

Although a direct comparison of the CIIP delivery models, between the National Research Council – Industrial Research Assistance Program (NRC-IRAP) and the Canada-Israel Industrial Research and Development Foundation (CIIRDF) is difficult due to differences in nature, scale and resources, NRC-IRAP was found to have unique advantages over CIIRDF.

The evaluation found that while the current CIIP Steering committee is generally efficient and effective at supporting the delivery of the program, broader questions remain about the coordination of international innovation activities between relevant Government of Canada (GoC) stakeholders. While CIIP activities presented strong alignment with broader GoC priorities on innovation, coordination with other government departments (i.e. ISED) and agencies is sub-optimal, and there appears to be increasing overlap between certain GoC initiatives and some CIIP activities.

Current grants and contributions norms and practices, based on Departmental legal advice, prevent ¶¶ÒùÊÓƵ from sitting on the CIIRDF board of directors as a full voting member. This has reduced Canada's capacity to effectively advance its interests and address ongoing operational challenges that affect CIIP delivery by CIIRDF. 

The current funding levels of the CIIP, which remain unchanged since 2005, are reducing the program’s relevance for foreign counterparts, and not meeting the needs of Canadian companies. Similar programs by like-minded countries are larger than the CIIP, and have fewer restrictions on eligibility.

Recommendations

1. It is recommended that the Investment, Innovation and Education Bureau (BID) expand membership of the CIIP Steering Committee to ensure effective coordination of the program with other GoC international innovation activities.
2. It is recommended that Investment, Innovation and Education Bureau (BID) conduct a periodic review of the geographic focus of the CIIP, that considers both Canadian commercial interests and innovation potential in partner countries.
3. It is recommended that Investment, Innovation and Education Bureau (BID), in collaboration with missions abroad, continue organizing regular PDAs in countries of focus, and consider expanding the PDA model to other countries with which Canada has an ST&I agreement.
4. It is recommended that ¶¶ÒùÊÓƵ engage the Israel Innovation Authority to find mutually acceptable solutions to ongoing CIIRDF issues related to administrative costs, financial management, and performance measurement and reporting.

Program Background

Canadian International Innovation Program

The Canadian International Innovation Program (CIIP) is a funding program designed to promote industry-oriented innovation partnerships with countries with which Canada has a Science, Technology and Innovation (ST&I) Agreement. It funds projects that have the potential for commercialization, and it stimulates Science and Technology networking and matchmaking activities to further new partnerships and accelerate the commercialization of R&D activities. The program brings economic benefits to Canadian companies and ultimately to Canada.

The CIIP funds two types of initiatives:

Joint Research and Development Projects

The main activity of the CIIP is to fund joint research and development (R&D) projects between Canadian and foreign companies from Brazil, China, India, Israel and South Korea.

The CIIP has annual Calls for Proposals (CFPs) for each country which focus on areas of common interest for both countries.

As a rule of thumb, each country is allocated between Can$ 900K and 1M per annum for CFPs. Successful applicants can receive up to Can$ 600K to pursue a collaborative R&D project. 

For Brazil, China, India and South Korea the CFPs and projects are managed by the National Research Council of Canada’s Industrial Research Assistance Program (NRC-IRAP) on behalf of ¶¶ÒùÊÓƵ.

The CFPs and projects for Israel are managed by the Canada-Israel Industrial Research and Development Foundation (CIIRDF).

Partnership Development Activities (PDAs)

The CIIP also supports R&D Partnership Development Activities (PDAs). PDAs are intended to facilitate the identification of specific projects between Canadian companies and companies in partner countries that could lead to industrial R&D cooperation.

PDAs are managed by Trade Commissioners at posts in CIIP target countries in collaboration with NRC-IRAP and local partners, except in Israel, where no recent PDAs have been organized.

While successful PDA participants can receive up to Can$ 15,000 to participate in a PDA, the average PDA contribution provided to a company is Can$ 2,600.

Timeline of the CIIP Program

Geographic Focus of the CIIP Program

While the CIIP is focused on 5 countries, the Terms and Conditions of the Program allow for other countries with which Canada has an ST&I agreement to benefit from funding.

Program Roles, Responsibilities and Resources

¶¶ÒùÊÓƵ

While the Department has the overall responsibility for the program, the CIIP R&D CFPs and project management are delivered by two external organizations. The program is housed under the Investment and Innovation Bureau (BID) of the International Business Development Branch (BFM). Canada’s Trade Commissioner Service (TCS) and ST&I officers at missions abroad, are responsible for planning and organizing Partnership Development Activities and helping Canadian companies connecting with qualified international contacts for Research and Development partnerships.

Human Resources of CIIP at ¶¶ÒùÊÓƵ

CIIP is directly managed by 1.5 FTEs in the Science, Technology and Innovation Division (BII), and supported by the broader BII division.

The CIIP is also supported by ST&I staff at missions abroad.

Program Resources

In the 2015 TB submission, the program was allocated Can$ 5 million per year between fiscal years 2015/2016 and 2020/2021, with the same amount ongoing past 2020/2021. Over 90% of this funding is in the form of contributions (Vote 10), while the remainder is for operating expenses (Vote 1).

Canada-Israel Industrial Research and Development Foundation

With Israel, the contributions are managed by a third-party organisation, the Canada-Israel Industrial Research and Development Foundation as identified in the ST&I agreement signed by Canada and Israel in 2006. Because of the contribution agreement in place and in accordance with the Transfer Payment Policy, CIIRDF has considerable independence in its choice of projects to fund and operates with minimal guidance from ¶¶ÒùÊÓƵ. CIIRDF has 3.5 FTEs working on the CIIP program, 2.5 in Canada and 1 in Israel.

CIIRDF has 3.5 FTEs.

NRC-IRAP

NRC-IRAP is responsible for the administration of the contribution funding of R&D projects between Canadian and International companies for Brazil, China, India and South Korea. NRC-IRAP also helps set the strategic direction of the program through its participation in the CIIP Steering committee.  With its technical expertise in science, technology and innovation, NRC-IRAP is responsible for the selection, assessment of eligible recipients, and the monitoring of funding. Finally, NRC-IRAP coordinates with its foreign counterparts to design and coordinate the application and selection process. NRC-IRAP has 4.75 Full-Time Equivalent (FTEs) dedicated to the CIIP.

NRC-IRAP has 4.75 FTEs.

Methodology

The methodology for this evaluation is based on the utilization-focused approach, which aims to make the evaluation useful for key stakeholders and to support decision-making. During the planning phase, the evaluation team engaged with program stakeholders of the Investment and Innovation Bureau (BID) at ¶¶ÒùÊÓƵ, NRC-IRAP and CIIRDF to ensure that the scope and questions of the evaluation would provide useful findings and support decision-making processes related to the future of the program.

The evaluation applied a mixed-method approach using qualitative and quantitative data collected through various techniques and data sources to respond to the evaluation questions. Data was collected between March and July 2019. Preliminary findings were presented to BID in August 2019, and the final evaluation was drafted between September and December 2019. The following methods/data sources were used to answer the evaluation questions:

Data Collection Methods

Key Stakeholder Interviews

Semi-structured interviews were conducted in-person and by phone with key stakeholders at ¶¶ÒùÊÓƵ HQ (HQ) and missions abroad (MISSION), relevant other federal and provincial Canadian government departments (OGD), and external stakeholders both Canada- and locally-based (EXT) and Private companies (COM). A total of 85 interviews were conducted as part of this evaluation. Six standard interview protocols were developed to ensure the relevance of questions for different groups of stakeholders.

Site visits

Site visits to partner countries were conducted for in-person interviews with mission staff responsible for the delivery of the program and external stakeholders. The evaluation team also conducted site visits to local companies having benefited from CIIP funds to pursue R&D partnerships with Canadian companies.

The evaluation team visited all five countries that are currently eligible for the CIIP program.

Online Survey of Canadian Companies

An online electronic survey was administered to Canadian companies having participated in CIIP activities, including Partnership Development Activities (PDAs) and participation in Calls for Proposals (CFPs).

The survey was sent out to 94 Canadian companies in May 2019 and was open for four weeks. When the survey closed in June 2019, the final participation rate was 37% (35/94).

Document and Literature review

The evaluation team conducted a comprehensive review of documentation provided by the Program team, including financial documents, statistics, correspondences, reports, meeting records, and other relevant files. Other sources were reviewed, including Canadian federal and provincial as well as the international collaborative innovation policies, strategies and programs of like-minded countries. Articles from peer-reviewed sources, were examined to analyze best practices in international collaborative innovation policy.

Data review

A variety of data sources were consulted and reviewed to assess the performance of CIIP in the areas of: achievement of results, client satisfaction, efficiency and effectiveness of governance mechanisms, geographic focus, coherence with Departmental and GoC priorities and relevance to partner countries. The departmental reporting tool TRIO was used to examine the performance of the TCS in delivering trade and innovation results at missions abroad. Data from Global Innovation Indices such as the Global Innovation Index, the Global Competitiveness and the Bloomberg Innovation Index were used to compare the rankings of CIIP partner countries compared to other countries with which Canada has Science Technology and Innovation agreements.

Finding #1.1: Partnership Development Activities are an effective tool that allow Canadian Companies to increase their knowledge of targeted markets and to gain access to Global R&D Networks.

During the evaluation reference period, the CIIP contributed to the organization of 21 PDAs. These activities have made strong contributions to the immediate outcomes of the CIIP (Appendix 2: Logic Model), both in terms of strengthening knowledge of target markets and raising awareness about R&D opportunities in these markets for Canadian Companies.

• 76% of Canadian participants surveyed indicated that participation in a PDA led to discussions with a foreign company on collaborative R&D projects.
• 52% of Canadian participants surveyed have indicated that the PDA led to a partnership with a foreign company.
• The majority of PDA participants are small to medium sized companies (5-99 employees) with limited resources to undertake exploratory business travel. PDA funding fills an important need for these companies, and is an incentive for them to explore international R&D opportunities.

Over 80% of Canadian SMEs surveyed said PDAs increased their knowledge of the foreign market (n=17)

Over 90% said PDAs increased their knowledge of collaborative R&D opportunities in the foreign market (n=17)

Finding #1.2: PDAs contribute to broader commercial and non-commercial results.

According to Trade Commissioners (TCs) at missions abroad and Canadian companies, PDAs also contribute to commercial results in International Business Development (IBD), Foreign Direct Investment (FDI) and to other benefits that are not fully captured by official performance measurement systems.  The majority of sector TCs interviewed at missions abroad saw a strong incentive to participate in the organization of PDAs because it allowed them to expand their client base by gaining more ST&I-oriented clients, and secondly it helped them to generate new business opportunities, contributing to their Key Performance Indicators (KPIs).

PDAs were seen by a majority of stakeholders as a strong opportunity to showcase the Canadian innovation ecosystem. While Canada has a strong and dynamic innovation sector, it is relatively small and not well-known internationally. Stakeholders stated that PDAs were useful to strengthen Canada’s reputation as an innovation nation.

• 76% of Canadian companies surveyed indicated that participation in a PDA led to results beyond joint R&D partnerships, such as new leads, new sales, and expanded business networks.

These collateral benefits of PDAs were only partially reflected in TRIO.

* Successes in Trio are KPIs for the TCS that include a lead pursued (Opportunity Pursued) by a Canadian company, a commercial transaction (Economic Outcome Facilitated), or an investment by a foreign company in Canada (FDI win).  

Finding #2.1: The R&D funding component of the CIIP has generated interest from foreign companies into the Canadian Innovation ecosystem, and vice-versa.

NRC-IRAP took over the R&D funding component of the CIIP for Brazil, China, and India starting in FY2015/16. South Korea was subsequently added as a country of focus in 2017. NRC-IRAP successfully negotiated and signed memorandums of understanding with Brazil, China and India by April 2016, and by June 2017 had an agreement in place with South Korea. For Israel, CIIRDF continued to be responsible for managing the CFPs on behalf of the GoC and the Israel Innovation Authority (IIA). During the evaluation reference period, a total of 13 Calls for Proposals (CFPs) were completed, generating 343 expressions of interest and 163 full proposals submitted. In total, 46 projects were approved for funding.

Most stakeholders agreed that NRC-IRAP had adapted well to its new role, and put in place adequate mechanisms to manage the CFPs. In addition, interviewees stated that NRC-IRAP had improved its management of CFPs over time and demonstrated capacity for self-improvement.

• 88% of Canadian companies surveyed by the evaluation team, stated that it was unlikely/very unlikely that they would have pursued collaborative International R&D activities without CIIP Funding.

* Projects having received funding as of March 31st, 2019.

While demand for the CIIP has increased, the quality of proposals varies by country. Demand for CIIP funding, as measured by the # of applications, has increased during the evaluation reference period. However, the quality of applications varies by country. While the CIIP receives more high-quality applications than it can fund in some countries (e.g. South Korea, China, Israel), in other countries it has been difficult to identify strong proposals.

For example, the most recent (FY2019) CFP with South Korea resulted in 10 project proposals being approved for funding, however only 2 could be funded under the Can$ 900K cap for this economy. In the end, two additional proposals were funded from the CFP, thanks to re-allocation of funding. Nonetheless, these types of situations are likely leading to missed opportunities for the CIIP.

Finding #2.2: It is still too early to see commercialization results from projects that were funded during the evaluation reference period. 

During the evaluation reference period, 39 projects received funding commitments for a total of Can$  17.4 million. While the first CIIP projects selected by NRC-IRAP received funding in FY2016/17, the majority of projects only started in later years. So far, only 1 project from NRC-IRAP  and 2 from CIIRDF have been completed. For these types of projects, commercialization results (i.e. new products/services, increased sales, new employees) can take a few years after project completion, and so for the majority of CIIP-funded projects commercialization results aren’t expected until FY2022-2023.

CIIP Planned vs. actuals Contributions (Vote 10) expenditures:  2015/2016 to 2018/2019

While the CIIP baseline budget is $4,620,000 per year, some funds from 2015/2016 were re-profiled to later fiscal years to give NRC-IRAP time to operationalize the program.  The program experienced delays in implementation in the first two years of implementation (2015/2016, 2016/17), and some funding was further re-profiled to later fiscal years.

Expenditures have progressively risen to meet expected levels, and given current commitment levels in future fiscal years, it is expected that the program will meet planned disbursement levels.

Status of CIIP funded projects and expected timeline for commercialization results

Finding #2.3: While commercialization results will not be available for a few years, the CIIP contributes to other benefits for participating Canadian SMEs.

For reasons of comparability, this evaluation is focused on the most recent edition of the CIIP (since 2015/16). While the current edition of the CIIP has yet to produce tangible commercial results (i.e. new products/services, increased sales, new employees), the program has incentivized Canadian SMEs to engage in collaborative international R&D activities and contributed to other benefits. For example, Canadian companies stated that receiving funding from the CIIP had helped them gain access to new markets, build new partnerships, improved their reputation and attract new investors. An evaluation of a similar European program (i.e. EuroStars 2008-2013) found that beyond strictly commercial results, EuroStars program participants reported the following benefits: increased innovative outputs (measured by patent filings), improved management skills, new partnerships between SMEs and increased recognition/reputation of their companies.

Not all collaborative R&D projects will achieve commercial success

Based on project data from CIIRDF, 79% of projects funded between 1996 and 2015 never reported any commercial results. The evaluation of the EuroStars program between 2008-2013 found that 10% of approved projects are withdrawn prior to inception, and a further 13% never achieve commercialization. There are a number of reasons that contribute to delays and/or impediments to commercialization of joint R&D initiatives, including: supply chain disruption; incompatibility with co-dependent technologies; partnership failure; and company business challenges. In addition, the longer the period between project initiation and product delivery, the greater the chances that market externalities will negatively impact commercialization.

Overall, the evaluation of the EuroStars program found that while 52% of projects achieved commercialization within 2 years following project end, 35% achieved commercialization later than 2 years after project end, and 13% of projects never achieved commercialization.

EuroStars Program 2008-2013: Time to Commercialisation

Some success stories of Canadian SMEs through participation in the CIIP

Tangent Design Engineering

Tangent Design Engineering is a Calgary, Alberta-based company that develops technology in the fields of: medical devices, oil and gas, industrial equipment, agriculture, consumer goods and has developed specialized knowledge and skills in gas detection, advanced process modelling and simulations, robotics, optical design, wireless wide-area low power networking, mobile equipment design and regulatory compliance (medical, industrial, explosion proof).

In 2016, Tangent participated in an India CIIP PDA focused on “smart grids” resulting in a partnership with the India based company, Allied Engineering Works. In 2017, their CIIP project proposal to design and manufacture an artificial-intelligence based electronic metering and monitoring system for the India power distribution sector was approved and awarded the full amount of funding, Can$  600,000. CIIP funding supported 8 new positions including 5 engineers for Tangent Design Engineering.  CIIP PDAs allow firms like Tangent Design Engineering to increase their knowledge of the local market by connecting with local firms, discovering opportunities for partnership and collaboration.

Without CIIP, Tangent would not have pursued an international partnership in India, their [Tangent’s) participation in the PDA, allowed them to gain a better understanding of the market.”

Redlen Technologies

Redlen Technologies is a Saanichton, British Columbia-based company that manufactures high resolution semiconductor radiation detectors and imaging equipment used in the fields of: Nuclear Cardiology, CT Scanning, Baggage Scanning and Dirty Bomb Detection.

CIIRDF brought Redlen Technologies together with Spectrum Dynamics, an Israel based medical device company that has developed high definition imaging technology. CIIP funding allowed for the integration of Redlen’s radiation detectors inside a Spectrum heart scan imaging device creating the ability to focus on specific anatomical points in the body.

In 2012, Spectrum Dynamics reported that this product generated Can$ 7 million in sales in its first 18 months on the market, and it has continued to generate millions more in recent years.  Due to this project, and other business by Redlen, the number of technical staff employed by the company increased from 38 in 2013, to 61 in 2016.

“There are other products, as a result of knowledge gained from this project, that are still in the developments

Finding #3: CIIP contributes to strengthened science, technology and innovation outcomes and supports Canadian commercial interests in target countries.

As seen below, CIIP target countries have slightly better science, technology and innovation outcomes than non-CIIP countries according to the TCS Client Satisfaction Survey. This means that in countries where the CIIP funds activities, companies are experiencing better results on ST&I thanks to the services they have received from the TCS.

The majority of mission-based stakeholders stated that the CIIP also contributed to broader Canadian commercial interests and strengthened bilateral relations in countries of focus. For example, the CIIP was seen as supporting ongoing Free Trade Agreement (FTA) negotiations with Brazil (i.e. Mercosur), and joint projects funded under the CIIP were a staple talking point in bilateral meetings with CIIP countries of focus. In China, continued ST&I collaboration through the CIIP, was seen by both sides, as a rare positive element of the strained bilateral relationship with Canada. 

TCS Client Satisfaction Survey: Achievement of outcomes for STI Business line:

(% Agree, Average for 2016-2019)

Finding #4: While the CIIP is not focused on the countries with the highest innovation potential, the current geographic focus of the program remains relevant to Canada’s commercial interests and trade priorities.

The current geographic focus of the CIIP remains relevant to Canadian commercial interests. When the program was launched in 2005, the geographic focus of the CIIP was informed by multiple commercial and diplomatic considerations including market access and the expected emergence of the Brazil, Russia, India, and China (BRIC) economies.

Canada has ST&I agreements with some of the most innovative countries, but the CIIP is not targeting them. Based on the analysis of major innovation indexes, CIIP countries are generally outranked by other countries in Europe and Asia. Canada has bilateral ST&I agreements with many of these countries, including several European states, Japan and Chile. The current geographic focus of the program continues to prioritize broader commercial considerations (e.g. business development, market access, trade diversification) rather than focus solely on the quality and potential of the local innovation ecosystem.

A periodic review of the geographic focus would be useful, but a broader scope could further spread resources thin. While a majority of stakeholders acknowledged that the geographic scope of the program was influenced by broader commercial considerations rather than innovation potential, they also agreed that it would be valuable to re-assess the focus of the CIIP periodically. At the same time, most stakeholders expressed concern about the potential broadening of the CIIP’s geographic focus, without additional resources, and worried that this would further reduce the relevance of the program by spreading resources thin. The majority of stakeholders agreed that a broader geographic focus would require additional funding.

Finding #5: Canadian companies that participated in CIIP activities expressed high levels of satisfaction with the program

Overall, Canadian companies had a positive client experience from participation in the CIIP. Based on interviews and through the survey, Canadian companies were highly satisfied with the processes and the activities of the CIIP Program for all countries of focus. According to post-PDA assessments, 98% of Canadian companies that participated in a PDA stated that this had allowed them to meet their objectives.

A small number of Canadian companies recommended improvements to the application and selection process, including a faster selection process, more flexibility related to eligibility of sectors, partners and geographic focus, and broader funding opportunities for Canadian SMEs to undertake international innovation activities.

Finding #6.1: While a direct comparison of delivery models between NRC-IRAP and CIIRDF is difficult due to differences in nature, scale and resources; NRC-IRAP has unique organizational advantages.

Finding #6.2: The delivery mechanism (i.e. NRC-IRAP vs. CIIRDF) has little impact on the length of CIIP CFPs.

In the absence of medium and long-term performance data (results), this evaluation looked at the efficiency of delivery models (i.e. CIIRDF & NRC-IRAP). One such measure is the length of CFPs, measured by the number of days between the launch of a CFP, and the notification of successful applicants.  While it would be preferable to measure the end of the CFP based on the date that successful applicants received a payment, this data was unavailable.

The available data between 2016 and 2019 shows that, on average, a CIIP CFP lasted 195 days from launch to notification of project approval across all five CIIP countries. Korea consistently has the shortest time interval, Israel and India hover around the average and Brazil has the highest.

Overall, the evaluation team found that the delivery mechanism does not have a strong incidence of the length of CFPs. Rather, the length of a CFP is determined by other factors, such as good collaboration with the partner organization, delays in proposal assessment, and other time constraints.

Finding # 7: While the current CIIP Steering committee supports the effective and efficient delivery of the program, questions remain about the right membership.

The CIIP steering committee has experimented with different models over the past few years. Membership has fluctuated over time to include representation from Global Affairs’ geographic branches and other federal Departments. Based on the review of steering committee minutes, the evaluation team found that the current model is fulfilling its roles and responsibilities as outlined in the terms of reference.

Generally, interviewees reported that the current composition and size of the CIIP steering committee was efficient and effective at supporting program delivery. Still, a minority of stakeholders wondered whether the CIIP steering committee should include representation from other stakeholders, such as ¶¶ÒùÊÓƵ geographic programs and Innovation, Science and Economic Development (ISED) Canada. 

Based on the review of CIIP Steering Committee minutes, the evaluation analyzed the frequency of discussed topics. The top four most frequently discussed topics of the CIIP Steering Committee meetings are related to its general operations: funding, partnership development activities, the budget and country program updates. Discussions about CIIRDF and Israel were more frequent at the steering committee compared to other countries. Issues related to involving CIIRDF are the 5th most frequently discussed topic of the CIIP Steering Committee, significantly outranking other CIIP partner countries and topics.

Finding # 8: Current grants and contributions regulations, and Departmental legal advice prevent Global Affairs from sitting on the CIIRDF board of directors as a full voting member. This has reduced Canada's capacity to effectively advance its interests and address operational issues.

Canada currently sits on CIIRDF board meetings as an observer, without voting privileges.

Typically, Canadian members of the CIIRDF board of directors have been either representatives from the academic world or from business associations. Having a Government of Canada representative on the CIIRDF board of directors has been a long-standing request from Israel. The desire for a Canadian government representative to sit on CIIRDF’s board of directors conflicts with the ¶¶ÒùÊÓƵ’s practices and norms on grants and contributions to arms-length entities. Based on interviews and document review, the evaluation team found that there have been no changes to this internal practice, which is based on Departmental legal advice. Short of having government of Canada representation on the CIIRDF board of directors, stakeholders interviewed stated the importance of selecting the right board members to represent Canada. They should represent both research interests and business interests.

¶¶ÒùÊÓƵ's limited role on the CIIRDF board of directors has effectively limited its ability to influence decisions by the board. 

This has led to a perception amongst some stakeholders that the projects funded by CIIRDF are not benefiting Canada and Israel equally. Furthermore, key stakeholders and document review revealed that CIIRDF faces some administrative challenges, including:

• Administrative expenses: CIIRDF has struggled to respect the 11.6% cap on administrative expenses set out in the contribution agreement.
• Financial management: Challenges with the management of contributions.
• Performance measurement and reporting:

  Discrepancies between results reported by CIIRDF, and results reported by ultimate project recipients and a perception of embellishment of results by certain stakeholders.

Some stakeholders believe that ¶¶ÒùÊÓƵ has not been able to fully resolve these issues, due to its limited role on the CIIRDF board of directors.

Finding # 9: The CIIP fills a unique programming niche at ¶¶ÒùÊÓƵ and existing program management mechanisms are adequate to support effective and efficient delivery of the program.

CIIP does not overlap with other trade funding programs at ¶¶ÒùÊÓƵ. Most trade funding programs at ¶¶ÒùÊÓƵ are designed to support international business development (IBD) or Investment (FDI) with the exception of the CanExport Innovation Program (CXI), formerly known as Going Global International (GGI). The CXI program does fund STI-oriented initiatives similar to PDAs, but it’s geographic scope (e.g. All countries) is broader, and the types of eligible applicants are different. However, while the evaluation team found no evidence of overlap between CIIP and CXI, the distinction between these programs was not always clear to stakeholders outside BID.

STI officers and sector TC’s at mission have incentives to work together on organizing PDAs. Based on interviews with stakeholders at mission, there was confusion about the difference in roles and responsibilities between Sector TCs and ST&I officers at mission. However, this overlap was not always seen as negative. In fact, stakeholders saw synergistic opportunities in working jointly on PDAs. Sector TCs have a strong incentive to actively participate in the organization of PDAs as they can lead to IBD results as well as allow them to diversify their client base. On the other hand, ST&I Officers benefit from the sector knowledge and network of their TC colleagues, which leads to a higher quality PDA.

STI Officers at mission feel well supported by the HQ CIIP team, but some are concerned about the impact of turnover of key program personnel at HQ

Despite the small size of the CIIP team at HQ, ST&I officers at missions abroad felt well supported by the CIIP team at HQ. However, some stakeholders expressed concerns about the risks of turnover of key CIIP personnel at HQ, and the negative impact this would have on program operations. ST&I Stakeholders at mission found that the ST&I Wiki was a useful tool to share information, tools and resources about CIIP and ST&I more broadly. Available data points to an increase in traffic on the ST&I Wiki page, since it was created in FY2016/2017. In 2018, there were over 60 visits to the ST&I Wiki page by 36 ¶¶ÒùÊÓƵ employees from CIIP countries.

Finding # 10: Although the CIIP is aligned with broader GoC priorities on Innovation, coordination of program delivery with other government departments and agencies is uneven, and there appears to be increasing overlap between certain GoC initiatives and some CIIP activities.

Innovation is a key priority for the GoC, and several Departments and Agencies are active in the Innovation space. Several Government of Canada (GoC) Departments and Agencies have a stake in innovation, including ISED, NRC, NRCAN, Health Canada, and others. Innovation has been an important priority for successive governments and is seen as a key to Canada’s future prosperity. Accordingly, investments in innovation by the GoC have been important in recent years, notably in budget 2017 and 2019. With the development of Canada’s new innovation and skills plan in 2017, ISED was identified as the lead department for innovation, and has received an important share of the new investments to support innovation.

¶¶ÒùÊÓƵ has developed a strong working relationship with NRC-IRAP, but effective coordination with ISED is lacking. Although CIIP activities are well aligned with Canada’s innovation priorities*, the role of international partnerships to support the domestic innovation agenda remains unclear. With recent GoC investments in Innovation there is increasing mandate creep, by certain Departments and Agencies, into the international space. For example, the evaluation team has learned that some of the new international innovation activities by ISED will include staff at missions abroad (i.e. BEJING) and commercial missions abroad (i.e. 2019 ISED Clean Tech mission to India). If coordination between ¶¶ÒùÊÓƵ and NRC-IRAP has greatly improved as a result of changes to program management, ISED and ¶¶ÒùÊÓƵ don’t have an effective coordination mechanism on International Innovation Partnerships and activities. Overlap between certain ISED activities and the CIIP is not necessarily negative. Some stakeholders stated that there were untapped opportunities for synergy that would result from greater coordination between ISED and ¶¶ÒùÊÓƵ.

• 13/15 of PDAs and all CFPs presented strong thematic alignment with Canada's Innovation priorities* (Canada’s Innovation priorities include the 5 Superclusters + the Economic Strategy Tables).

Shifting mandates of Selected GoC Departments and Agencies

Case Study: Whole-of-Government approach:

Australia’s Global Innovation Strategy

While at geographic opposites of the globe, Australia and Canada present several similarities. Amongst other things, both countries are relatively small in terms of population and share similar economic structures with abundant natural resources and a strong primary economic sector. Both countries import the majority of their machinery, transportation and high-tech equipment. Both countries are also significant contributors to global research and development and have a solid track record in developing innovation and new technologies. Yet, both countries face some of the same issues in meeting the challenges, opportunities and global trends related to the fourth industrial revolution.

To respond to this situation, the Australian Government launched it’s Global Innovation Strategy (GIS) in October 2016. The strategy is a whole-of-government approach, led by the Department of Industry, Innovation and Science, and the Department of Foreign Affairs and Trade, to increase Australia’s innovation and science connections internationally.

The Strategy builds on new and existing initiatives and represents an integrated and coordinated approach to Australia’s engagement with international partners in innovation and science. Specifically, the Strategy is designed to:

• encourage other countries to partner with Australia
• draw together existing resources and programmes
• access opportunities through international collaboration and start-up support
• shift engagement to scalable, more effective multi-partner collaborative projects.

The GIS represents a A$36 million (Can$  32.4 M) investment over four years.

A June 2019 formative evaluation of the Global Innovation Strategy found that early performance metrics indicate positive outcomes for program components.

Finding # 11: Some coordination between ¶¶ÒùÊÓƵ, the OGDs and other Canadian stakeholders on ST&I happens as part of preparation for the ST&I Joint Committee meetings.

Current consultation and coordination mechanisms are not standardized. The bilateral ST&I Joint Committee (JC) meetings between Canada and its foreign ST&I partners are an opportunity to discuss potential areas and opportunities for greater ST&I collaboration. These meetings are generally held every 2 years and provide an opportunity for Canadian ST&I stakeholders, including GoC Departments and Agencies, provincial governments, academic institutions, business networks and the private sector to discuss Canadian ST&I priorities, opportunities and initiatives.

While the evaluation team found that there were usually some coordination activities in preparation for the ST&I Joint Committee meeting, representation by various groups of stakeholders varied widely.

The CIIP was not always on the agenda of the JC meetings for countries of focus.

The CIIP program is one of the only federal funding mechanisms for joint industrial R&D.

Missions abroad would like to receive more information from NRC-IRAP about CIIP CFPs

Despite good collaboration between NRC-IRAP and ¶¶ÒùÊÓƵ HQ, ¶¶ÒùÊÓƵ employees at missions abroad expressed a wish to receive information from NRC-IRAP on CFPs, including lists if applicants and a summary of joint proposals.

Example of Coordination by Canadian Innovation Stakeholders

Finding #12: As current funding levels of the CIIP are unchanged since 2005, the program is losing relevance in the context of growing competition, and not meeting the needs of Canadian companies

While Canada is a valued ST&I partner, the CIIP is losing relevance due to it’s small size, and growing international competition. Canada remains a valued ST&I partner by its CIIP counterparts, who see Canada as having a high quality innovation ecosystem. In all CIIP countries, foreign counterparts ranked Canada in the top-tier of important partners for international collaboration. At the same time, foreign counterparts expressed concern about the modest funding levels of the CIIP program compared to partnerships with other countries. Many partner countries also demonstrated willingness to increase their share of funding.

Given that program funding levels have remained the same since program inception in 2005, the CIIP has experienced a 20% erosion of purchasing power due to inflation. It is estimated that Can$ 1,000,000 in 2005 would be worth only Can$ 799,699 in 2018, based on the Consumer Price Index (CPI) from the Bank of Canada.

The program struggles to meet increasing demand for international innovation funding from Canadian companies. Current funding levels are not meeting the demand of Canadian industry.  There has consistently been increasing interest in the program, but limited amounts available to fund projects. Every year, the number of projects approved for funding is higher than actual amounts of available funding, limiting the number of projects that can be funded. Further, key program stakeholders voluntarily restrain active promotion of the program due to its limited size, in order to manage expectations about program funding. In general, key program stakeholders take a more targeted recruitment approach, rather than broader promotion and marketing of the program (e.g. on Social media, Online marketing campaigns). The majority of participants are either recruited through the TCS network or through the NRC-IRAP network of ITAs.

Finding # 13: The CIIP is generally smaller than similar programs by like-minded countries, and has greater restrictions on eligibility.

There is a wide variety of international innovation partnership programs by like-minded countries with differences in design, geographic scope, funding levels and sectors of focus. None of them are exactly the same, but many are trying to achieve similar objectives as the CIIP.

The evaluation team found that, comparable programs, whether government to government or foundation-managed were larger than the CIIP. While some of the similar programs were only slightly larger (e.g. China –Australia, UK – SK), the foundation managed programs (e.g. Israel – India, Israel – SK) were much larger.

Conclusions

Conclusion #1:

(Supported by Findings # 10, 11)

In recent years, the Government of Canada has made innovation a key priority with important investments and a number of initiatives. This has led to an increase in the number of GoC stakeholders that are involved in the international innovation space, and to some extent, to a blurring of lines and mandates of participating Departments and Agencies. While some informal and ad hoc coordination happens as part of Canada’s ST&I Joint Committee meetings, there are likely opportunities for synergy between different GoC initiatives, that aren't being capitalized on due to limited coordination, between key stakeholders.

Conclusion #2:

(Supported by Findings # 2, 4, 13)

The international context has changed since CIIP’s inception in 2005. While the current geographic focus of the program remains relevant to Canada’s commercial interests, the CIIP is absent from some of the countries with the highest innovation potential and most dynamic innovation sectors. Further, in considering an expanded geographic scope of CIIP, the program should be mindful of the risk of further reducing the relevance of the CIIP for partner countries and Canadian industry if current funding levels are maintained. The CIIP has experienced growing demand from countries of focus in recent years, while funding levels have remained unchanged.

Conclusion #3:

(Supported by Findings # 1.1, 1.2, 4)

Partnership Development Activities are highly beneficial to Canadian small and medium enterprises and represent excellent value for money. While PDAs do not always lead to collaborative R&D projects, they contribute to broad commercial results for Canada, some of which are not captured by current performance measurement systems.

Conclusion #4:

(Supported by Findings # 6.1, 6.2, 7, 8)

The current governance structure and management practices of CIIRDF continue to represent a challenge for ¶¶ÒùÊÓƵ. The Department is unable to effectively advance it’s interest on the CIIRDF board of directors, and it is unable to fully resolve unsatisfactory management practices by CIIRDF, including financial management, performance measurement, reporting practices, and administrative expenses.

Recommendations:

Recommendation #1:

It is recommended that Investment, Innovation and Education Bureau (BID) expand membership of the CIIP steering to ensure effective coordination of the program with other GoC international innovation activities

Recommendation #2:

It is recommended that the Investment, Innovation and Education Bureau (BID) conduct a periodic review of the geographic focus of the CIIP, that considers both Canadian commercial interests and innovation potential in partner countries.

Recommendation # 3:

It is recommended that Investment, Innovation and Education Bureau (BID), in collaboration with NRC-IRAP and missions abroad, continue organizing regular PDAs in countries of focus, and consider expanding the PDA model to other countries with which Canada has an ST&I agreement.

Recommendation # 4:

It is recommended that ¶¶ÒùÊÓƵ engage the Israel Innovation Authority to resolve ongoing issues with CIIRDF related to administrative costs, financial management, performance measurement and reporting.

Management Response and Action Plan

Annex 1: Evaluation Questions

The questions below were developed based on a preliminary document review as well as scoping discussions with key Global Affairs and Government of Canada stakeholders.

Annex 2: Program Expected Results

The following Logic Model summarizes the program intervention, including its main activities, outputs and different levels of outcomes. It is based on the Performance Measurement Strategy developed by the Innovation and Investment Bureau developed in April 2015. During the evaluation, the Logic Model and Performance Measurement Framework were being revised by the program.

Annex 3: Considerations for future programming

In recent years, the Government of Canada has made innovation a key priority with important investments and a number of initiatives at various departments and agencies. This has led to an increase in the number of GoC stakeholders that are involved in the international innovation space, and to some extent, to a blurring of lines and mandates of participating departments and agencies. Given the number of Departments and Agencies in the Government of Canada that are stakeholders in science, technology and innovation,  it would be beneficial to take a whole-of-government approach to international engagement in order to increase coherence, and to fully leverage international initiatives in support of Canada’s domestic innovation priorities.

Annex 4: Program Context: International Innovation

Innovation is an Essential Element for Canada’s future prosperity

Canada’s national prosperity and competitiveness are inextricably linked to the capacity to participate in and benefit from innovation. Canada is good at creating companies but struggles to scale them up. The reasons include a small and fragmented local market, shortages of experienced business talent, a lack of sources of growth capital and an aversion to risk on the part of some of Canada’s established companies. Industry Research & Development (R&D) levels in Canada are relatively low, with the majority of private sector R&D investments in Canada being concentrated in a small number of very large companies.

Canada has a high quality innovation ecosystem, but  international competition is fierce

Canada has a productivity gap relative to other leading nations, partly due to the innovation gap. A decline in Canada’s ranking on the 2018 Bloomberg Innovation Index is the most recent evidence of the intensity of this competition, and of Canada’s faltering place in it. Canada’s capacity for innovation remains excellent, but the underpinnings of that capacity are eroding and the country is less successful in creating domestic wealth from innovations than many of it’s peers. Other countries recognize the importance of innovation in driving long-term growth and are investing heavily in innovation systems that support successful, competitive companies.

International collaboration is increasingly essential for successful innovation

International partnerships are an essential catalyst for science and technological innovation, as these collaborations often accelerate the pace of discovery and result in improved commercialization. Networks are particularly central for SMEs which often lack tangible resources (e.g. financial and human-capital resources) and serve as platforms for business development in foreign market relationships. SMEs become better equipped to develop innovative business solutions that are geared towards the customer preferences and requirements in foreign markets. Export-oriented SMES grow more than twice as fast as those that do not, while internationally active SMEs are three times more likely to introduce products or services that are new to their sector than those that are entirely domestic in orientation. The process of collaborative international innovation produces talented people with enhanced skills in inquiry and problem solving. Unleashing the potential of highly skilled people to generate and develop new ideas into products, processes, organizations, and systems is the key to creating lasting prosperity.

Governments have a key role to play in supporting international partnerships

One of the key roles for government can be in acting as a ‘trusted intermediary’ bridging gaps in private-sector networks. In the UK, a review of the UK Trade and Investment services demonstrates how SMEs with government funding for innovation were significantly more likely to innovate and that innovators grew faster than non-innovators linking exporting, innovation and competitiveness in both domestic and overseas markets. Given the structure of Canada’s innovation sector, and the tendency of  Canadian companies to participate less in international partnerships  than their international peers, it is especially important for the Government of Canada (GoC) to play this role.