¶¶ÒùÊÓƵ

Evaluation of Canadian International Assistance Programming in Ukraine, 2009/2010 to 2017/2018

Executive Summary

The evaluation found that Canada was an important player and significant donor in Ukraine over the evaluation period, providing consistent and long-term international assistance that had significant impact in all areas examined. It leveraged available financial resources and Canadian political support to engage the post-2014 reform and modernization process in Ukraine.

The department’s development programming addressed the Government of Ukraine’s priorities, while humanitarian and peace and stabilization programs provided quick, flexible and responsive assistance to address emerging needs. International assistance programming achieved significant outcomes in select programming areas, such as support to Ukraine’s police reform, advancing gender equality and the establishment of the juvenile justice system. Notable results in the dairy and horticultural sectors also helped support economic stabilization in the country.

Canada’s responsiveness was particularly evident during Ukraine’s Revolution of Dignity and near economic collapse of 2014 when the department significantly increased and diversified its portfolio of funded activities in the country. As the crisis became protracted, support for early recovery and the humanitarian-development-stabilization nexus in eastern Ukraine was segmented across program streams. This compartmentalization of programming proved to be a limitation for Canadian objectives to move from crisis response to longer-term reconstruction, peacebuilding and development efforts in eastern Ukraine. The piloted Integrated Country Framework, often cited as a tool with good potential to maximize departmental coherence, was never approved or used to support joint strategic planning or coordination across program streams in Ukraine.

Programming across all three international assistance streams generally targeted needs and was delivered in accordance with project plans. However, in some program streams, investments were spread across many activities without proper inter-project linkages to enable the achievement of strong results. Recurrent political instability and the fragile security situation in the country further impacted the sustainability of project results.

Background

Ukraine is a lower-middle income country with a history of political and economic instability. The evaluation period included the 2008 global financial crisis and the 2014 Ukraine political crisis, known as the Revolution of Dignity, both of which contributed to the worsening of socio-economic conditions for Ukrainians.

Russia annexed Crimea in 2014, leaving parts of the eastern regions de facto separated and under the control of pro-Russian military forces. The situation in Ukraine continues to be a protracted crisis that affects 5.2 million people, 1.5 million of whom UNHCR estimates were internally displaced in 2018. Canada, the United States, the European Union and others supported Ukraine’s sovereignty and territorial integrity. They condemned Russia’s intervention in the country and also increased their assistance in subsequent years.

Ukraine has made slow economic progress in recent years and remains one of the poorest countries in Europe. It continues to score poorly on indices specific to corruption, economic freedom, income distribution and others. Following the 2014 presidential and parliamentary elections, the Government of Ukraine launched the Reform Action Plan 2017-2020 in an effort to reform government and facilitate stable economic growth.

Bilateral relations

Canada identified Ukraine as one of 20 countries of focus for international assistance in 2009. The two countries signed the Road Map of Priorities for Canada-Ukraine Relations, which focused on supporting sustainable economic growth, improving accountability of public institutions and advancing democracy in Ukraine. Following 7 years of negotiations, the Canada-Ukraine Free Trade Agreement came into effect in 2017.

Canada’s international assistance in Ukraine was multifaceted. It included development assistance (managed by the Europe, Arctic, Middle East and Maghreb branch), peace and stabilization assistance (International Security and Political Affairs branch), and humanitarian assistance (Global Issues and Development branch).

Ukraine received a total US$5.6 B in official development assistance from 2009 to 2017. Canada provided 12% of all assistance and was the fourth-largest donor to Ukraine after the EU, the United States and Germany.

Text version

Map of Ukraine showing government-controlled areas of eastern Ukraine (dark blue), non-government controlled areas of eastern Ukraine (light blue), area annexed by Russia (yellow), and border checkpoints not controlled by Ukraine (star)

GAC bilateral disbursements to Ukraine (in Can$ M)

Source: Statistical Reports on International Assistance, 2009/10 to 2017/18

Text version

GAC bilateral disbursements to Ukraine (in Can$ M)

• 2017/18: $51;
• 2016/17: $51;
• 2015/16: $77;
• 2014/15: $106;
• 2013/14: $45;
• 2012/13: $41;
• 2011/12: $41;
• 2010/11: $34;
• 2009/10: $18

Methodology

Evaluation scope and questions

Purpose of evaluation

The evaluation:

• covered the period from fiscal year 2009/10 to 2017/18 with a specific focus on the period after the 2014 Ukrainian revolution
• sampled 34 projects valued at $181 million that were implemented by the Europe, Arctic, Middle East and Maghreb (EGM) branch, the International Security (IFM) branch and the Global Issues and Development (MFM) branch 

Responsiveness and flexibility (relevance)

1. To what extent was GAC’s international assistance aligned with Ukraine’s changing needs and priorities at the country strategy and sector levels?
• To what extent was there a flexible response across GAC in the context of the emerging priorities, consequences and challenges?
• Which factors influenced the responsiveness and flexibility of GAC’s international assistance to Ukraine?

Coherence

2. What was the degree of internal coherence and coordination among departmental actors involved in Ukraine?
3. How did Canada coordinate international assistance with the Government of Ukraine and other relevant actors? How could coherence and coordination be further improved? 

Effective solutions and results (effectiveness, gender equality, efficiency, sustainability)

4. To what extent did GAC’s international assistance achieve outcomes across various sectors and types of programming?
• To what extent were the specific results in gender equality and empowerment of women and girls achieved?
• Could the same results have been achieved in a more efficient way?
• How could efficiency be improved in future programming?
5. What is the likelihood that the results of GAC’s international assistance will be sustained or continue in priority sectors?

Methodology

Evaluation research and analysis were conducted by an external consulting group. Analyses were performed at three levels: Program level, Sector/Thematic level (which included Advancing Democracy, Sustainable Economic Growth (SEG), Peace and Stabilization, Humanitarian Assistance and Gender Equality) and Project level. Data collection missions to Ukraine were conducted in September and November 2018.

Political economy and conflict analysis 

A political economy and conflict analysis was performed to provide an understanding of the Ukrainian context and to analyze the changes that took place during the evaluation period. It was used to situate GAC’s strategies and programming against the changes within the country context prior to and following the 2014 events.

Portfolio analysis

n = 187 projects
Portfolio data were analyzed for 187 projects implemented in the evaluation period. Analysis was performed to assess the changes in the main sectors of international assistance programming. Portfolio analysis served to develop a sampling approach and criteria.

Document review

Review of:

• Departmental policy and strategic documents
• GoU key policies and strategies
• External documents on Ukrainian priorities and context (other donors, multilateral organizations, research bodies)
• Project-level documents (agreements and implementation documents; logic models and performance measurement frameworks; annual and final reports; evaluation reports) 

Key informant interviews

n = 253 interviewees

• GAC staff (n = 18)
• Implementing partners in Canada and Ukraine including local NGOs (n= 87)
• Representatives of Ukraine’s national, regional and local government (n = 73)
• Project beneficiaries (n = 46)
• International multilateral organizations and donors (n = 29)

Site visits

n = 14
Site visits were conducted within SEG, Peace and Stabilization and Humanitarian Assistance sectors in four regions of Ukraine: Dnipro, Zaporizhia, Luhansk and Donetsk regions (oblasts).
Site visits allowed the evaluation team to assess project results in the regions and to interview local government, project stakeholders and project beneficiaries.

Triangulation and validation

Data collected during evaluation were synthesized and triangulated in order to arrive at the evaluation findings, which then led to conclusions and recommendations in the final evaluation report.

Note: the external compilation and analysis of information was a limitation. ¶¶ÒùÊÓƵ mitigated this through continuous engagement with the external evaluators and provided on-site support and controls through participation in data collection exercises in Ukraine.

Responsiveness and flexibility

Canada provided responsive and agile assistance that supported the Government of Ukraine’s priorities and addressed the most urgent governance, stabilization and humanitarian assistance needs.

Canadian programming before and following the 2014 political crisis was well-aligned with Ukraine’s development priorities: sustainable economic growth and democratic transformation. The Development Program supported the stabilization of financial systems; the establishment and reinforcement of business regulations; improved competitiveness for small and medium enterprises; election support; and, improved performance of the horticulture and dairy industries.

The Stabilization and Reconstruction Task Force supported human rights monitoring and protection.

The department’s footprint expanded after the 2014 crisis to include humanitarian assistance.

The department:

• addressed urgent needs of people affected by the conflict
• supported mine action and ceasefire
• facilitated reforms of the newly elected Ukrainian government 

Factors that shaped the department’s flexible and agile response included:

• strong leadership by Canadian embassy
• Development Program’s decentralization and field presence grounded in long-term, active dialogue with the Ukrainian government and Canada’s implementing partners
• streamlined financial mechanisms of Humanitarian and Peace and Stabilization Operations enabling funds to be quickly disbursed to partners

Canada’s ability to target longer-term resilience and peacebuilding in conflict-affected regions was limited by the department’s internal structures.

Canada did not respond cohesively to protracted needs in eastern Ukraine. While Canada funded some peacebuilding and development programming in conflict-affected regions, it did not target much of the broader economic recovery and physical reconstruction, local governance, resilience and social cohesion needs of the area. Program streams often viewed these as outside of their direct mandates.

Source: Statistical Reports on International Assistance, 2009/10 to 2017/18.2009

Programming streams and sectors

¶¶ÒùÊÓƵ’s programming increased in its scope following the 2014 crisis to include:

• international humanitarian assistance managed through the Humanitarian Assistance Program
• further Development Program work within the thematic areas of electoral support and the rule of law
• moving into policy reform work within Peace and Stabilization Programming

Text version

Programming streams and sectors for 2009/10 to 2013/14: Development - Sustainable Economic Growth (Entrepreneurship, Business Enabling Environment, Rule of Law, Accountable Public Institutions), Agriculture; Peace and Stabilization (START) - Election Support, Human Rights Monitoring

Text version

Programming streams and sectors for 2014/15 to 2017/18: Development - Sustainable Economic Growth (Banking Reform, Agriculture), Advancing Democracy (Rule of Law, Election Support, Citizen Participation, Anti-Corruption, Public Administration); Peace and Stabilization (PSOPs/START) - Police Reform, Humanitarian Mine Action, Peace Process and Social Cohesion; Humanitarian Assistance

Coherence

Internal Coordination-Nexus

Communication across program streams was generally strong, but coordination was limited to select activities. The pilot Integrated Country Framework was not implemented and did not help to improve coordination.

Ukraine was a high-priority country for Canada throughout the evaluation period. The former CIDA and DFAIT worked well together on issues that pertained to Ukraine prior to the department’s amalgamation in 2014. Internal and external stakeholders cited Canadian ambassadors as active proponents of a whole-of-department and whole-of-government approach that prioritized diplomacy, trade, development and eventually humanitarian and peace and stabilization programming in the country.

Designated focal points for humanitarian, peace and stabilization and political affairs exchanged information and coordinated with counterparts at Headquarters and among themselves at the Canadian embassy. However, linkages across program streams were not always evident nor were they systematically planned. Coordination was primarily related to information sharing rather than integrated programming. Some department stakeholders voiced concerns about working in silos and not being able to capitalize on opportunities for joint action.

An integrated country framework was often cited by management and staff as a tool with the potential to align programming and reporting across program streams. The Canadian embassy piloted a draft integrated country framework twice, but neither attempt was approved nor actively used to manage in-country activities.

There was no evidence of a corresponding integrated performance measurement framework at the country level. Instead, reporting on results in the various programmatic sectors relied on partner project frameworks to understand the performance of Canada’s investments.

An internal review of the piloted Integrated Country Framework demonstrated that engagement across programming streams remained limited by:

• lack of dedicated resources
• competing priorities
• lack of high-level commitment to an integrated process
• different internal business processes such as contrasting reporting lines and timelines

External coordination

Canada took an active role in coordinating its international assistance with external actors and within the humanitarian coordination system. However, the absence of a formal development coordination structure and limited participation of Government of Ukraine officials narrowed coordination effectiveness.

Ukraine lacked a formal development donor coordination structure. Nevertheless, the Development Program at the Canadian embassy was actively involved in donor coordination. Interviewed stakeholders viewed Canada as a positive contributor to coordination discussions across all international assistance streams. Embassy leadership and staff from the Development Program, a defence attaché, and a temporary in-country peace and stabilization officer all played prominent roles in engaging and coordinating with international and Ukrainian counterparts.

The embassy led some donor coordination groups, such as those for the elections and agriculture sectors, and was actively involved in meetings with police reform stakeholders. Canada, along with other donors, advocated for the advancement of reforms in those sectors. After the 2014 crisis, the Ukrainian government became more actively engaged in coordinating donor activities; however, many of the donor coordinating structures did not have any representation from the Ukrainian government.

Canada was active in the UN humanitarian cluster coordination system through the embassy’s designated humanitarian assistance focal point who was also a full-time Development Officer. The humanitarian focal point was responsible for participation in consultation meetings (e.g. the humanitarian-development nexus) and working group attendance.

Departmental implementing partners, many of whom had long-term presence in the country, coordinated with the Ukrainian government and other national and local stakeholders at the sectoral and project levels. This coordination was particularly strong within the humanitarian mine action, peace process and social cohesion and policy reform sectors.

Ukraine reform conferences took place in the United Kingdom (2017) and Denmark (2018) where the Government of Ukraine launched its Reform Agenda and provided updates on its implementation. Canada hosted the third Ukraine Reform Conference in July 2019. The conferences became part of a long-term partnership on reform implementation between Ukraine and the international community.

As a result of the 2019 conference, a new vision was established for Ukraine. It included specific reforms that would help the country realize its goal of becoming a “European and Euro-Atlantic future where reforms and growth will include and benefit all of its citizens”. These included the rule of law, public administration, the security sector, the energy sector, land, health, education, pensions and the protection of human rights.

Effective solutions and results

Sustainable economic growth

The Development Program’s support to the sustainable economic growth (SEG) sector contributed to the strengthening of the Ukrainian economy through demonstrated achievements within the banking sector, diversifying trade channels and improving the business environment. 

Programming in the SEG sector was well-aligned with Ukraine’s changing needs and priorities. This alignment was facilitated by the responsive design of projects and driven by strong beneficiary participation and input. It was also strengthened by the Canadian embassy’s ongoing engagement of Ukrainian authorities and close collaboration between experienced Canadian implementing partners and Ukrainian project teams.

Overall, projects delivered effective, relevant and efficient assistance. The Strengthening the National Bank project, for example, was an important achievement for Canada’s Development Program that contributed to broader economic stability in Ukraine, particularly after the 2014 crisis. Canada and the International Monetary Fund worked together on this project to enhance the capacity of the National Bank of Ukraine to support greater price stability, lower inflation and to improve Ukraine’s ability to adjust to adverse external economic shocks. This supported the recovery of the public finance system and the stabilization of the banking sector. The project’s scope was later extended to developing the capacity of non-banking financial institutions. Overall, stakeholders highlighted the benefits of this project and stated that the project “helped save the financial banking sector.”

The Canada-Ukraine Free Trade Agreement (2016) provided new development programming opportunities within this sector. The Canada-Ukraine Trade and Investment Support project provided technical support to small and medium enterprises and Ukrainian government partners. Results for this project demonstrated increased growth and viability of Ukrainian enterprises. Additionally, the “She Exports” campaign in support of Women in Business provided women entrepreneurs with new trade opportunities through close cooperation with Ukraine’s Export Promotion Office.

Sustainable economic growth (agriculture)

The Development Program supported a large number of small rural households, small and medium-sized enterprises, and small landholders in the dairy and horticultural sectors. It helped increase production and productivity for dairy and horticulture farmers and contributed to the recognition of family farms by the Government of Ukraine.

The Horticultural Development project provided support to almost 7,000 small farmers and rural families, including poor and vulnerable households. A second phase was developed to build on the results of the original project. Both phases improved collaboration among farmers through market consolidation and clustering practices. This work further led to economies of scale for product marketing, equipment leasing and technical services.

The Improving the Competitiveness of the Dairy Sector project helped over 3,000 dairy farmers gain access to better processing and storage technologies and marketing skills. Close to 40% of targeted dairy farmers became members of the 20 cooperatives created by the project. Two hundred women and young people enrolled in cooperative and administration training through this project. The Supporting Dairy Business Development project aimed to build on these achievements and as of 2017 supported 12 cooperatives representing over 1,000 members. Support to family farms impacted gender roles. Men participated in milk farm activities, traditionally considered a women’s activity. Women gained access to finance, acquired new knowledge and networks. Dairy sector programming further contributed to successful advocacy for formal legal recognition of family farms under the law.

The effectiveness of some agriculture projects was less apparent. While the Grain Storage and Marketing Cooperative project contributed to the drafting of law on agricultural cooperatives and developed a curriculum on cooperatives to post-secondary institutions, its impacts were limited to a small number of farmers in proportion to the large amount of funding it received. It had little opportunity for replicability due to high capital infrastructure costs.

Horticultural Development project
A “pick-your-own” pilot helped a family cooperative with 10 members facing labour shortage challenges. The pilot was successful, and the cooperative expanded the next year. Olga,a member of the family cooperative,became the leader of the project’s Women’s Group in the Zaporizhia region. This 150-member group received training on topics including technical production, sales and marketing. In 2019, the group remained active and continues to expand.

Advancing democracy

Canada-funded interventions demonstrated strong results in rule of law and electoral support programming. However, overall programming in support of advancing democracy was spread across a diverse range of thematic areas without integration of clear sustainability or exit strategies.

The Development Program funded interventions in key governance thematic areas: rule of law, democracy and elections, civil society and citizenship participation and institutional development. It managed a limited number of projects in each area, which amounted to fragmented support to any one area. As a result, support to small, stand-alone initiatives lacked strong interlinkages with other projects or sectors such as civil society support and investigative reporting.

Long-term instability impacted progress toward planned project results. Despite the challenging context, many projects still demonstrated strong results. Rule of Law programming, however, was formulated more coherently than others with good inter-project linkages.

This development programming demonstrated positive results. The Quality and Accessible Legal Aid project helped expand the nationwide free legal aid system to enable Ukrainians, including vulnerable populations, to access legal assistance. More than 300,000 beneficiaries received assistance each year. Two justice system support projects funded in 2012 and 2015 made a contribution to the increase in organizational and technical capacities of Ukraine’s core judicial institutions: the High Qualifications Commission of Judges of Ukraine and its associated institution and the National School of Judges of Ukraine.

Important results were achieved in the improvement of judicial procedures and the introduction of citizen-centred court administration in Ukraine. Also, the Juvenile Justice Reform project focused on improving the juvenile justice system, a niche area not assisted by other donors. The project helped improve legal and policy frameworks, develop probation tools such as pre-trial reports, risk assessments and rehabilitative programming, and establish 11 youth probation centres.

Peace and stabilization – police reform

The Peace and Stabilization Program (PSOPs) contributed to reducing tensions, building confidence and increasing community-level stability. Its most prominent achievement was in supporting police reform.

Peace and stabilization programming strengthened the capacity of government institutions, civil society and local populations to recover from conflicts.

Project interventions were useful in monitoring developments on the ground and providing protection through their presence by:

• monitoring ceasefire (by the OSCE Special Monitoring Mission) and human rights
• promoting safety and security, for example, through checkpoint security improvements and humanitarian demining
• contributing to confidence-building within communities

Canada was also recognized for its strong role in promoting gender equality, in particular, in its contribution to the launch of the first Ukrainian Association of Women in Law Enforcement.

Although Canada’s peace and stabilization funding in Ukraine was smaller compared to larger donors, Canada made an impact by providing advice, technical assistance and equipment toward police reform, humanitarian mine action and social cohesion.

Along with support from the United States, the Canadian Police Mission in Ukraine and the Police Training Assistance Project played valuable roles in the establishment of the Patrol Police in 2015 and the Police Academy in 2018. These projects contributed to training, curriculum development, building community policing capacity and the establishment of the media centre of the Patrol Police and the Bike Police Unit. Both initiatives significantly evolved over time by building relationships with national counterparts and tailoring their approaches to emerging needs. All interviewees noted that this progress on police reform would not have been possible without Canada’s support. They also emphasized that support for police reform remained a work in progress. Key challenges in implementing police reform were the absence of a national police strategy and the need to provide longer-term support to maintain achieved results or support long-lasting impacts.

Peace and stabilization – mine action

Peace and stabilization programming achieved good results in its support to humanitarian mine action and social cohesion linked to the peace process. However, support was dispersed without an explicit vision for longer-term peacebuilding efforts.

PSOPs programming provided policy support, technical assistance and relevant demining equipment to Ukraine’s Ministry of Defence and the State Emergency Services.

The Building Ukraine’s Humanitarian Demining Capacity project:

• helped the Ukrainian government develop policies, practices and institutions
• provided training
• installed and operated a digital map showing hazards and cleared areas
• helped procure modern equipment

The Threat Reduction and Clearance of Mines in Conflict Affected Areas of Eastern Ukraine project:

• provided risk education and survey and clearance activities
• conducted hazard marking and clearance operations
• built the capacity of the Ukrainian National Humanitarian Mine Action Authority and national explosive ordnance disposal and clearance teams

Canada’s programming helped develop a Gender in Mine Action Program that encouraged and supported the mine action sector to mainstream gender in policies, programming and operations. Canada, along with other key donors, advocated for changes in legislation. This support enhanced the efforts of the Ukrainian government to draft mine action legislation, set up an Information Management System for Mine Action and adopt national mine action standards.

Support to the peace process and social cohesion in eastern Ukraine strengthened the understanding and oversight of conflict dynamics and increased the feelings of safety and confidence within communities. Important activities included ceasefire and human rights monitoring, enhancing checkpoints in government and non-government-controlled areas, and providing 34 monitors to the OSCE Special Monitoring and Human Rights Monitoring Missions.

The sustainability of Canada’s efforts in promoting peace and stabilization was affected by the fragmented nature of programming, which was a collection of individual projects without a cohesive long-term vision.

Targeted peace and stabilization support to the main international humanitarian mine action partners helped build the capacities of Ukraine’s Ministry of Defence and other government stakeholders engaged in the implementation of the national mine action program. By donating equipment and providing strategic advice, Canada helped build trust and engage with key Ukrainian partners.

Humanitarian assistance

Humanitarian partners addressed urgent needs along the contact line and in the government controlled areas but faced challenges in non-government controlled areas. The transition from crisis response and stabilization to longer-term reconstruction, peacebuilding and development revealed gaps in departmental processes.

Canada has been Ukraine’s third-largest donor of humanitarian assistance since 2014. The streamlined processes of the International Humanitarian Assistance (IHA) Program enabled Canada to fund trusted humanitarian partners quickly and with the flexibility to adapt to changing contexts. The IHA Program provided 75% of its funding in Ukraine to multilateral agencies and 25% to non-governmental organizations. Funding was directed at partner activities in line with the objectives of annual UN Humanitarian Response Plans for Ukraine. While Canada’s contribution to Ukraine’s humanitarian appeal decreased between 2015 and 2018, it has remained at 2% of total appeal amounts since 2016. This decline was similar to that of other donors providing humanitarian support to Ukraine.

The most visible and reported outcomes achieved by Canada’s humanitarian partners were along the contact line. These included provision of basic food and service delivery, such as access to utilities (water, electricity), education and healthcare. In the government-controlled areas, sampled projects demonstrated contributions to alleviating the stress and suffering of more than a million internally displaced people. Until 2016, partners included some livelihood and recovery opportunities in project activities, for example, through income generation, cash transfers or micro-enterprise grants. As humanitarian contributions by donors declined, these activities were largely discontinued. Early recovery and support for the humanitarian-development-stabilization nexus in eastern Ukraine was limited across the department.

Humanitarian access in the non-government controlled areas was challenging for political reasons. In these areas, multilateral partners and several international NGOs provided life-saving support via local organizations, but were restricted in the scope of their engagements by the de facto authorities.

Efficiency

Despite the changing circumstances, projects across program streams were efficient and had adequate project management systems in place. The department’s ability to measure and report on performance at the sector, program and departmental levels was limited.

Financial and narrative reports generally indicated that project resources were used efficiently, were in line with the approved plans and were ultimately focused on achieving results. Project-level results-based performance management systems were the main contributor to project-level efficiency.

Project performance frameworks were based on approved logic models. In principle, projects followed results-oriented reporting through annual and quarterly progress reports. Projects employed annual and quarterly work plans, which allowed for more flexible planning and the ability to respond to external changes. Financial and human resources were made available in a timely manner to implement activities.

There were 2 country development strategies prepared for the periods 2009-2014 and 2014-2019. In principle, these included logic models at both the Ukraine program level and at the sectoral level, for example, sustainable economic growth. The Development Program lacked an approved logic model throughout the evaluation period. Both the PSOPs and IHA Programs did not have country-specific strategies; however, PSOPs did have them at the global program level against which reporting should take place.

In 2018, an integrated country framework was developed for all international assistance programming to Ukraine. This initiative was not pursued and therefore did not result in an overarching integrated intervention approach with common reporting channels.

Full conclusions could not be drawn on the efficiency of programming due to a lack of sectoral level performance measurement frameworks for the Development Program and the absence of Ukraine-specific measurement frameworks for the IHA and PSOPs Program.

Sustainability

Program sustainability was challenging in the context of recurring political instability and the country’s fragile security situation. The department’s investments, however, showed a degree of sustainability in some thematic areas.

The likelihood of achieving sustainable outcomes was higher in areas where there was a clear commitment and willingness to reform, for example, police reform, banking and agriculture. In those areas, international assistance programming helped develop and implement reforms and interventions that were more likely to be sustained.

Department-funded interventions frequently did not consider exit or risk management strategies during the design or implementation phases. As a result, opportunities were missed to contribute to cumulative results with a greater degree of sustainability.

Political instability and the fragile security situation in the country limited the sustainability of some programming objectives, particularly within the advancing democracy sector. However, this sector remains one of the critical areas of development support in Ukraine.

Gender equality

Canada was viewed as a key advocate for gender equality in Ukraine. Strong gender equality results were achieved in the Development Program’s agriculture sector and in the PSOPs Program.

Various stakeholders perceived Canada to be a leader in promoting gender equality in Ukraine. Canada addressed gender equality at all levels:

• G7 participation and leadership
• political and diplomatic visits
• project programming

Canada’s Feminist International Assistance Policy (2017) positively affected advocacy and reinforced gender equality programming. Dedicated development gender equality-focused staff were assigned as focal points at the embassy, including a local gender equality advisor who worked closely with Headquarters gender advisors.

Gender equality was well integrated in development programming and led to good results. For example, the Improving the Competitiveness of the Dairy Sector project helped launch new forms of small and medium agribusiness and family farms that were registered to women as owners or to young couples in equal shares. The selected farms were supplied with modern equipment, which increased revenues and gave women a larger role in farm management while their husbands took care of the farm operations. Additionally, the Canada-Ukraine Trade and Investment Support Training project included sessions on gender that addressed diversity and discrimination and their impact on export activities.

Canada’s support for police reform contributed to advancing gender equality through increased women’s participation and leadership. The 2018 launch of the Ukrainian Association of Women in Law Enforcement was an important example. The police force also increased its attention toward gender-based violence and incorporated this topic into its police training curricula.

Canada provided valuable support to the vice prime minister for European and Euro-Atlantic Integration who was responsible for gender equality coordination in Ukraine. This included assistance in strengthening the national framework for Ukraine’s implementation of the Convention on the Elimination of All Forms of Discrimination Against Women (CEDAW) ratified by Ukraine in 1981.

Conclusions

Canada’s international assistance programming in Ukraine was well aligned with the country’s needs and national priorities. The department’s responsiveness and flexibility in humanitarian and stabilization programming were especially critical immediately after the 2014 Revolution of Dignity and the outbreak of conflict in eastern Ukraine. During this time, the department significantly increased and diversified its portfolio in the country. Partner-led programming addressed urgent political and economic stabilization, humanitarian assistance and governance needs.

The department’s flexible and agile response was supported by:

• some quick internal processes
• good communication between Headquarters and the Canadian embassy in Kyiv
• relationships with long-term experienced partners

As the crisis continued, gaps in the department’s processes and a lack of coherence across programming streams revealed missed opportunities in bridging crisis response with longer-term reconstruction, peacebuilding and development efforts in eastern Ukraine. Communication across program streams was good throughout the evaluation period. Some examples of internal coherence existed, but the separate programming, reporting and approval mechanisms of each program stream restricted the department’s ability to attain more strategic internal coherence. Since each program stream addressed a large number of priorities, the support within and across certain sectors was fragmented. Without the presence of an approved integrated, long-term, country-specific strategy that meaningfully included all relevant program streams, Canadian international assistance was unable to link successes between streams and timelines. Without a common performance framework, Canada’s overall progress toward overall outcomes within the country were difficult to reinforce.

Despite the lack of coherence in planning and communicating results, Canada’s Ukraine international assistance programming led to stand-out results in police reform, advancing gender equality, and juvenile justice. Strong political leadership supported a positive image of Canada’s international assistance in Ukraine.

Recommendations and management responses

Department

Recommendation

Clarify expectations and responsibilities of departmental actors with respect to humanitarian-development-peace nexus programming.

1. The Deputy Minister of Foreign Affairs and the Deputy Minister of International Development should establish a departmental process to clarify expectations and responsibilities of different departmental actors with respect to nexus programming. This process should determine roles and responsibilities of departmental programs in different country contexts where all or some departmental actors are operational. It should identify options to enable better financing across the nexus and close existing gaps in funding prevention and long-term recovery efforts.
   
   The process should be consistent with the actions outlined in the OECD DAC Recommendation on the Humanitarian-Development-Peace Nexus and address department-wide structural and financial constraints that limit nexus programming, as shown in Ukraine. In situations where multiple departmental streams deliver programs in a given country, the process should clarify who leads and develops an integrated, whole-of-department vision and strategic plan. For Ukraine, this should build on the work of the draft Integrated Country Frameworks, as well as the work of the International Security and Political Affairs branch to develop a Canadian Integrated Conflict Analysis Process.

Management response

Accepted

The Global Issues and Development branch, the International Security and Political Affairs branch, and the International Assistance Operations bureau will collaborate to identify options for a way forward to enhance the department’s approach to nexus programming. This will include, for example, tools, roles and responsibilities and guidance in line with the 2019 OECD-DAC Recommendation on the Humanitarian-Development-Peace Nexus. A recommended option will be presented for discussion at Executive Committee.

Europe, Arctic, Middle East and Maghreb branch

Recommendation

Focus development programming and address medium- and longer-term reconstruction and recovery needs in eastern Ukraine.

2. The Assistant Deputy Minister for the Europe, Arctic, Middle East and Maghreb branch should narrow the number of programming sectors for the Ukraine Development Program, in particular to those where there are good prospects for sustainable outcomes and genuine commitment for reform. Where appropriate, the branch should implement interventions that more fully consider the medium- and longer-term peacebuilding, reconstruction and recovery needs in eastern Ukraine and ensure that development, humanitarian and peace and stabilization interventions are complementary.

Management response

Accepted

1. The program will ensure that options for reducing its sub-sectors are considered during the next investment planning cycle (spring 2020) and will narrow the number of sub-sectors further during the next strategic planning exercise (2022). 
2. The Development Program will also convene a joint meeting with the Peace and Stabilization Operations Program and the International Humanitarian Assistance Division by March 31, 2020, to discuss opportunities and set specific actions that align with recommendations from the Nexus Contact Group led by the Peace and Stabilization Operations Program, the International Humanitarian Assistance Bureau, and the International Assistance Operations Bureau (see recommendation 1) related to strengthening mutual complementarity. This will include discussing appropriate opportunities for the branch to implement interventions that address the medium- and longer-term reconstruction and recovery needs in eastern Ukraine.

Recommendation

Facilitate sustainability planning

1. In support of sustainability, the Assistant Deputy Minister for the Europe, Arctic, Middle East and Maghreb branch should develop exit strategies in thematic areas where assistance will be phased out. These should include requirements for partners that must be considered during project design.

Management response

Accepted

Project team leaders are instructed by the program director to coordinate and manage projects toward sub-sector and program-level results. Going forward, exit strategies will be required for sub-sectors prior to phase out. Effective October 2019, project implementation plans for new projects will include sustainability plans with exit strategies, as appropriate. The quality and implementation of sustainability plans will be monitored through regular operational reports submitted by partners.

Considerations

Opportunities to align Canadian programming goals

As Ukraine is a complex and fragile situation, its needs are multi-faceted and changing. The department could consider solutions that orient programming decisions toward the problems associated with fragility and resilience before aligning with program stream priorities. Needs analyses for all programming in protracted crises could be tied to common understandings of the conflict drivers in the country by all relevant program streams.

The department could reintroduce an approved planning framework that integrates appropriate program streams into common outcomes and performance measurement frameworks. Canada has endorsed the OECD-DAC Recommendations on the Humanitarian-Development-Peace Nexus, which focuses on promoting more coherent action among humanitarian, development and stabilization programs in fragile and conflict contexts. The recommendations include a series of concrete actions available for each donor to implement.

Development program could consider further linking sustainable economic growth program outcomes with Canada’s overall trade interests. The Canada-Ukraine Free Trade Agreement could provide an avenue to further the reach of development outcomes that foster increased gender equality, good business practices and increased market competitiveness in Ukraine.

Exit strategies

For all international assistance programming operating in fragile contexts, program streams should integrate exit strategies into project design. This includes promotion of national and local ownership of projects and a phased handover of responsibilities.

Annexes

Annex I. Acronyms

CEDAW

Convention on the Elimination of all Forms of Discrimination Against Women

CIDA

Canadian International Development Agency

DFAIT

Department of Foreign Affairs and International Trade (Canada)

DFATD

Department of Foreign Affairs, Trade and Development (Canada)

GAC

¶¶ÒùÊÓƵ

GoU

Government of Ukraine

IHA

International Humanitarian Assistance

OSCE

Organization for Security and Cooperation in Europe

PSOPs

Peace and Stabilization Operations

SEG

Sustainable Economic Growth

Annex II. Validated Program Logic Model

This consolidated program logic model has been reconstructed on the basis of Program Logic Model 2009-2013, Canada’s Bilateral Development Strategy for Ukraine 2014-2019, the available draft logic model for the period 2010-2018, the ¶¶ÒùÊÓƵ Peace and Stabilization Operations Program, and the department’s International Humanitarian Assistance programming documents. The consolidated Ukraine Program Logic Model was validated during the fieldwork mission.

Ultimate outcome

Increased economic opportunities for women and men in a strengthened and stable democracy

Intermediate outcomes

Sustainable economic growth: Increased growth of small and medium enterprises and farms, particularly those led by women
Advancing democracy: enhanced integrity and transparency of democratic institutions and practices
Stable state: improved prevention, stabilization and recovery from conflicts and crises
Human dignity: reduced suffering, increased and maintained human dignity and lives saved in populations experiencing humanitarian crises

Immediate outcomes

Increasing competitiveness: small and medium enterprises (SMEs) and farms, particularly those led by women, introduced new management practices and functions

Outputs

• Technical assistance provided to SMEs, including farms, business associations and other governance actors on business and market development
• Technical assistance provided on targeted agricultural commodities (grain, dairy, horticulture)
• Effective strategic planning and forecasting tools developed
• Technical support provided to enhance participation of individuals and organizations in productive economic activities

Activities

• Technical and financial support to enterprises, including farms, in high growth sectors to increase their competitiveness
• Strategic planning and forecasting
• Economic policy advice aligned with European standards
• Business support organization capacity building
• Cross-cutting activities: Provision of policy advice and advocacy to and negotiation with the Government of Ukraine, other national stakeholders and civil society

Economies of scale: improved cooperation among small and medium enterprises to achieve economies of scale

Outputs

• Technical assistance provided to business associations, cooperatives, aggregators, lead farmers on targeted agricultural commodities (grain, dairy, horticulture)
• Value-chains supported and developed
• Technical assistance provided to business and market development, including relevant gender equality and environment issues

Activities

• Public-private partnerships development
• Effective strategic planning and forecasting tools
• Enhancement of economic opportunities for women and girls
• University curriculum development in targeted sectors, including horticulture and dairy technology, cooperative administration
• Cross-cutting activities: provision of policy advice and advocacy to and negotiation with the Government of Ukraine, other national stakeholders and civil society

Institutional strengthening: Enhanced capacities of public institutions to deliver on their mandates (improved enabling environment)

Outputs

• Technical assistance provided to public administration, local governments and democratic institutions such as the Central Election Commission on reform priorities, decentralization and delivery of services at the local level, legislation and regulations, electoral processes, including relevant gender equality and environment issues
• Supported strategic planning and local/ regional development, agriculture, including relevant gender equality and environment issues
• Enhanced capacities of national and regional authorities to effectively participate in planning and policy development and implementation at all levels

Activities

• Technical assistance to increase professional capacities of journalists and ensure independent reporting by the media
• Technical support to enhance access to information
• Intermunicipal and interregional cooperation
• Support for decentralization and facilitation of regional financing
• Cross-cutting activities: provision of policy advice and advocacy to and negotiation with the Government of Ukraine, other national stakeholders and civil society

Democracy: improved participation of citizens, particularly women, in public life and decision making

Outputs

• Technical assistance provided to strengthen civil society to deliver services and participate in governance reform processes
• Support provided to establish mechanisms and enhance capacities of the citizens to participate in governance and decision-making processes 
• Capacity development provided to media on investigative reporting and the necessity to have strategies required for independence

Activities

• Activities to increase civic education and awareness and strengthen participation in local decision-making processes
• Support to community mobilization and strengthening participation mechanisms and local democracy
• Support to independent electoral system throughout the electoral cycle
• Cross-cutting activities: provision of policy advice and advocacy to and negotiation with the Government of Ukraine, other national stakeholders and civil society

Rule of Law: increased equitable access to justice for women and men, particularly those from marginalized groups

Outputs

• Technical assistance provided to Ukrainian court system
• Capability development provided to court administration to deliver effective and efficient services
• Capacity development delivered for judges, lawyers, advocates and civil society on secondary legal aid, judicial processes and related legislation, including gender equality and environment issues
• Juvenile justice system established and strengthened

Activities

• Capacity development and technical support to policy making and law drafting
• Legal awareness and education on the rights of citizens
• Supported design and implementation of measures for anti-trafficking
• Legal aid support established and strengthened
• Assistance with implementation of judicial reform, including development of capacities and strengthening of judicial independence, court administration
• Support for establishment of a juvenile justice system
• Cross-cutting activities: provision of policy advice and advocacy to and negotiation with the Government of Ukraine, other national stakeholders and civil society

Peace and Stabilization: enhanced capacities and processes of institutions, civil society and local populations to prevent, stabilize and recover from conflicts and crises

Outputs

• Technical assistance and equipment provided to the key stakeholders to jointly ensure peace and stability
• Technical assistance provided for preparation of policies, frameworks, strategies and guidelines on peace and stabilization
• Physical environment rehabilitated, for example, infrastructure and demining
• Advocacy supported
• Research, surveys, reports or media articles developed or supported

Activities

Support to:

• security sector reform, such as police reform, including provision of equipment
• ceasefire monitoring
• human rights monitoring
• conflict mediation and resolution and social cohesion
• mine awareness, surveying and clearance
• improving line of contact checkpoint security
• policy planning and development
• Cross-cutting activities: provision of policy advice and advocacy to and negotiation with the Government of Ukraine, other national stakeholders and civil society 

Humanitarian assistance: increased access to and use of humanitarian assistance and protection by crisis-affected populations

Outputs

• Support provided to trusted humanitarian partners to identify and address the needs of the most vulnerable men and women

Activities

• Assistance with delivery of humanitarian assistance
• Advice and assistance on reconstruction in the affected areas
• Cross-cutting activities: provision of policy advice and advocacy to and negotiation with the Government of Ukraine, other national stakeholders and civil society