¶¶ÒùÊÓƵ

Rationale and Purpose of the Evaluation

This formative evaluation covers the five-year (2016-2021) Canadian Foodgrains Bank (CFGB) grant agreement with ¶¶ÒùÊÓƵ (GAC) for $125 million. It assesses CFGB’s performance and value added in addressing humanitarian needs and alignment with relevant policies over the period, in order to assist GAC in its decisions on the renewal of CFGB’s core funding. A grant agreement provided funding for CFGB food assistance, nutrition, public engagement and public policy programs, as well as a contribution to program management and overhead costs.[1] Under the terms of the program, CFGB contributed matching funds on a one to four matching basis. Assistance - largely aimed at addressing food security and nutrition needs of affected populations - was provided through 155 discrete initiatives in 31 countries. Actions were implemented by 15 CFGB member agencies and their partners, with CFGB headquarters (HQ) providing overall coordination, technical support and liaison with GAC.

Intervention Logic

During the five-year period from 2016 to 2021 the global food security and nutrition situation deteriorated after a 15-year period of improvement. The United Nations (UN) estimated that by 2021, nearly ten percent of the world’s population was exposed to severe levels of food insecurity because of health and economic shocks, conflict, and climate and environmental events.

Seeking to address this, the programme’s ultimate intended outcome is “reduced hunger, improved nutrition, and increased food security for hunger affected individuals and households in developing countries.” This is supported by five intermediate outcomes related to food consumption, nutrition, livelihoods, national policies, and raised awareness, and three cross-cutting themes of gender, environment sensitivity and organizational learning.

Scope and Methodology

This evaluation focussed on two of the five intermediate outcomes:

1. Stabilising and increasing immediate consumption of food by hunger affected individuals, and
2. Improving dietary quality and nutrition practices and reduction of disease among hunger-affected individuals and households, as well as their associated immediate outcomes.

The specific objectives of this evaluation were to assess CFGB’s humanitarian performance in addressing humanitarian needs through ten evaluation questions under seven criteria: Coverage, Accountability and Participation, Effectiveness, Efficiency, Coherence, Complementarity, and Connectedness. It also assessed how and to what extent the processes introduced in the grant under evaluation contributed to performance, and to what extent CFGB added value.

After an evaluability assessment, inception process and report to clarify the scope of the evaluation and refine the evaluation questions, evidence was collected through an approach utilising a detailed review of reports relating to 30 projects (accounting for 20% of programme funds), case studies of three countries (including focus group discussions and face to face and remote interviews in Lebanon, South Sudan and Zimbabwe) and an E-survey and remote interviews with staff of CFGB, member agencies and partners. The evaluation reviewed a total of 30 projects financed with GAC funds during the period under evaluation. Together, these projects account for $27.8M of funding (approximately 20 percent of total program funds over the period under evaluation). The evaluation team comprised four international and three national experts (one in each of the three case study countries).

Evaluation Findings and Recommendations by Evaluation Question

EQ 1: To what extent did CFGB and its partners act with impartiality and prioritize the vulnerable and those in most need?

CFGB strongly adhered to the principles of impartiality and prioritisation of the most vulnerable through a robust framework of systems, procedures, guidance and capacity building support, combined with agile and context appropriate beneficiary targeting. A key driver of this is CFGB’s broad network of local implementing partners that can use their local knowledge to both identify and reach the most in need, although in a small number of cases government bodies may have been perceived to have involvement in beneficiary selection. Nutrition needs were addressed to a lesser extent, partly because of limitations on the way resources could be used and to a lesser extent because of provision of relatively less guidance and capacity building on emergency nutrition programming.

• Recommendation 1: Consider how to ensure absolute clarity around provenance of assistance and neutrality in the countries where close working relations with government can be misinterpreted.
• Recommendation 2: CFGB complete revisions of the Supplementary Feeding Guidelines, and develop or provide additional nutrition guidelines based on international standards and protocols.
• Recommendation 3: GAC should consider allowing a percentage funding to be allocated on the basis of evidence collected by CFGB partners with a strong presence on the ground.

EQ 2: To what extent was the food and nutrition assistance provided by CFGB able to meet the immediate food security and nutrition needs of recipients (including the most vulnerable) in the short to medium term?

Overall, the CFGB Food Assistance (FA) portfolio was globally well targeted and locally relevant to basic food needs, but only a small proportion of GAC funding was allocated to critical nutrition needs – in part because of the siloed nature of its funding mechanisms. Furthermore, the short-term nature of many actions meant that responses did not always fully address food or nutrition gaps and a significant percentage had to be extended beyond their original temporal scope. Attempts were made to improve relevance by implementing programs with complementary objectives, but monitoring and evaluation resource constraints and the Covid-19 pandemic meant that there were shortcomings in learning in some areas.

• Recommendation 4: GAC to consider committing additional resources to increase nutrition actions to reach the most vulnerable.
• Recommendation 5: GAC and CFGB should work together to identify niche area(s) where they can collaborate on addressing global gaps around nutrition in the humanitarian context.
• Recommendation 6: GAC should consider modifications to its funding policies and architecture to permit FA and nutrition responses to be ‘bolted on’ as contingencies / crisis modifiers to development and resilience programs should the need arise. Likewise, it should look at how the process of transitioning affected populations from FA to longer term resilience programming can be eased if that is clearly in their longer-term interests.

EQ3: To what extent were the most vulnerable crisis-affected people consulted and able to influence decisions related to the assistance?

CFGB and member organizations were largely effective in ensuring the participation of vulnerable peoples, due in large part to their method of working through local implementing partners and funding requirements for situation assessments to inform project design. Feedback and complaints mechanisms and process improved significantly over the program, and effective Accountability to Affected Populations processes are now an integral component of all projects.

EQ4: To what extent did projects of CFGB members (funded by GAC) meet their stated outcomes, in a timely manner and at an acceptable level of quality and with the most appropriate modality? 

Projects met their food assistance outcome targets – e.g. Food Consumption Scores (FCS) – in most cases, and their nutrition outcomes in many cases where targets were identified (in the small proportion of programs where nutrition aspects were included - generally longer term actions). A range of factors impacted on achievement of outcomes, including limited resources, timeliness of responses, short project duration, partners’ capacity, environmental, political and economic context, and cultural / behavioural issues. Individual program modality and delivery mechanism decisions were largely appropriate to context and strongly informed by beneficiary preference. In-kind transfers were used widely – partly because of stated beneficiary preference, but also because of a somewhat ‘neutral’ stance towards cash and voucher assistance (CVA) by some members / partners.

• Recommendation 7: CFGB further develop their guidance on Situation Assessment and Project Design for both Food Assistance and Emergency Nutrition.
• Recommendation 8: Capitalize on and leverage CFGB community-level reach for improved outcomes around nutrition by building in nutrition components in Food Assistance programming
• Recommendation 9: Consider allowing funds to be allocated to food security crises identified via ‘informal’ warnings generated by partners using approved methods.
• Recommendation 10: Invest resources in gathering evidence on a population’s full FA needs and the pros and cons of half ration decisions where appropriate.
• Recommendation 11: Explore the feasibility of GAC allowing ‘top up’ funding to address specific emergencies that are longer than initially expected or revisiting the system under which extensions to existing projects are approved.
  In some cases, lean seasons last longer than initially anticipated, and affected populations require longer term support. CFGB is able to respond to these situations by extending projects, but this results in resources being removed from elsewhere.
• Recommendation 12: GAC should support CFGB to build capacities to ensure partners’ readiness for CVA. This will involve an analysis of the partners’ capacity to deliver CVA in the context of systems that exist locally.

EQ5: To what extent were CFGB outputs delivered with the most rational and economic use of inputs, and with the most appropriate modality?

CFGB, member and partners showed rational and economic use of inputs for nutrition programs, but combining FA and nutrition projects rather than running separate stand-alone actions, would increase efficiencies in achieving food security and nutrition outcomes. Satisfactory levels of efficiency were achieved with regard to modality at an individual or program level, but there is room for improvement at a portfolio and strategic level: this includes improving readiness to deliver cash and voucher approaches and the documentation and reporting processes related to CVA modalities.

• Recommendation 13: GAC to assess the utility and accuracy of cost per beneficiary data being commonly collected and how it is used to inform the selection of modalities. This would inform learning on advantages on effectiveness and efficiency of adding therapeutic foods and supplementary foods to FA programs.
• Recommendation 14: GAC and CFGB to consider more proactively targeting and determining who and where the countries and locations are that are needing year on year emergency food assistance and design programming (e.g. social safety nets) to address these longer term needs.
• Recommendation 15: GAC and CFGB to increase preparedness, anticipatory action and consider moving to more localised implementation and lighter, more efficient programming approaches including an assessment of their readiness for climate change. 
• Recommendation 16: To update GAC and CFGB policies and strategies with a more pro-CVA stance in line with other key donors including updating guidance, terminology and reporting templates around CVA.
• Recommendation 17: GAC to support CFGB to analyse relative efficiencies of differing modalities and delivery mechanisms and gather learning for dissemination and capacity building purposes.

EQ6: How/in what ways have the capacities of member agencies and their implementing partners and beneficiaries / targeted populations and their representatives been recognized and improved/built upon?

CFGB clearly value and seek to build upon the varying levels of capacity of their members and partners, and, over the course of the program, significant improvements in capacity in some core cross-cutting areas was achieved. Some improvements could be made in technical support, guidance and capacity building on nutrition, CVA and to a lesser extent FA.

• Recommendation 18: GAC to consider making nutrition a cross-cutting theme, to address the increasing global burden of malnutrition and also to improve efficiencies and effectiveness of FA.
• Recommendation 19: CFGB and members improve their abilities in capacity assessment in order to better focus CVA capacity building efforts on weaker agencies/partners. GAC to support CFGB to ensure a CVA technical advisor is appointed.
• Recommendation 20: CFGB to develop a capacity strengthening plan for increasing the capacity of members and partners around nutrition (around measurement/assessment, implementation, monitoring and evaluation). This could potentially involve the engagement of regional Technical Advisors who are able to provide on-the-ground training and/or do quality assurance of nutrition surveillance.

EQ7: Have members and implementing partners coordinated well with other actors?

CFGB membership presents significant value add through a highly collegiate way of working, and over the grant period CFGB emphasised the need for and supported improvements in coordination, particularly with external entities like government and cluster systems. These efforts were effective but some areas for strengthening remain, including involving smaller partners into national networks, and sharing lessons between agencies at a CFGB level.

• Recommendation 21: CFGB to continue with capacity building efforts and requirements around coordination (and other matters that also contribute to staff capacities), and encourage / support smaller members & partners to register with coordination bodies as a minimum.
• Recommendation 22: Consider a collective approach whereby all CFGB members and implementing partners are recognized under one membership of a Cluster. This way, even if staff capacity within a particular agency is low, they are able to participate by proxy and receive updates
• Recommendation 23: That Technical Advisors/point persons in CFGB HQ participate in the Global Nutrition Cluster, to help understand where nutrition sits globally, identify niche areas for CFGB to contribute to addressing the rising rates of global malnutrition, and how best to coordinate with others in the field.
• Recommendation 24: CFGB HQ staff participation at global levels can provide inputs into global level (feed in situation from the ground) and particularly support the 'localization' agenda – e.g., that the National Clusters include local members who know the situation on the ground.

EQ8: How and to what extent have the new project criteria and processes introduced in the 2016-March 2021 grant contributed to or hindered the observed performance towards meeting humanitarian needs?

Several of the “new” project criteria and processes – particularly those for tracking food assistance outcomes - helped with overall program performance towards meeting humanitarian needs, while others (e.g. some related to gender) gave little return on new insights for the level of effort required. Furthermore, a proliferation of new guidelines and processes on Integrated Food Security Phase Classification (IPC) standards during this grant period meant that there was less time available for improving capacities, interventions, guidance and outcomes around nutrition or CVA.

• Recommendation 25: In cases where IPC data is unavailable, CFGB and GAC mutually agree on acceptable alternative sources of data, indicators and critical levels needed for GAC funding approvals.
• Recommendation 26: That rather than a ‘green lighting process’, GAC clearly define their principles and criteria, identify key indicators required, IPC levels, and reporting requirements.
• Recommendation 27: Reconsider the minimum level of information needed to make sure that the project is effective, identifying who and how it will be used. Decrease the number of indicators and support collection of indicators in real time so that data can inform programming (i.e. fewer focused indicators), in addition to providing data for donors.
• Recommendation 28: Set more modest levels of data required (both in situational analysis and reporting) relative to the varying lengths and/or level of funding and/or availability of existing data of projects. Clearly define limited key indicators that the program will be assessed on.
• Recommendation 29: Include key nutrition indicators for food assistance programs, in addition to food security indicators.

EQ9: How compliant and to what extent have funded actions improved their vulnerability, gender and diversity aspects in the light of CFGB and GAC gender and vulnerability policies and practices in place at the time?

The Evaluation Team found significant improvements in CFGB strategy, policy, guidance, tools and practice around gender and preventing sexual exploitation and abuse, and lesser improvements around vulnerability and diversity. Limited progress was made towards meeting gendered nutrition outcomes defined in CFGB’s logic model and outlined in GAC’s Feminist International Assistance Policy (FIAP). Some actions to address shortcomings are already underway as a consequence of CFGB’s “Best Practices in Gender Responsive Programming Gender Delegation” Report produced in June 2019. However, GAC should consider increasing funding within short term food assistance programs and neighbouring longer-term programmes to enable the excessive work burdens of women and girls to be addressed. Additionally GAC funding to permit better assessment of nutritional status would permit the nutritional needs of all beneficiaries to be more impacted but should ensure a focus on the specific and additional nutritional needs of different types of women, girl and other key vulnerability groups (age and disability).

• Recommendation 30: The evaluation team is supportive of the recommendations for FA programs developed by CFGB.
• Recommendation 31: GAC to consider increasing funding under the FIAP and within short term FA programs and neighbouring longer-term programmes for actions to address the excessive work burdens of women and girls, and to ensure that adequate time and resources are attributed to the type of gender transformative change that the FIAP requires (and commits CFGB to), in a way that does not rush or increase risks to women and girls and organisational reputation within highly patriarchal contexts.
• Recommendation 32: GAC to update the FIAP to acknowledge the importance of ensuring adequate time and support for transformational change to occur within programming, and a sensitive approach in specific patriarchal settings.
• Recommendation 33: GAC, in close collaboration with agencies, to request, monitor and analyse more detailed disaggregated gendered and vulnerability and diversity information where it exists, and that other vulnerability disaggregation’s be more widely gathered, included in situation assessments, and reported to GAC, and that CFGB and GAC analyse the data and share findings and learnings.
• Recommendation 34: GAC should consider funding increases to permit more programming devoted to meeting the assessment of nutritional status and then meeting of the nutritional needs of all beneficiaries, but with a specific focus on the specific and additional nutritional needs of different types of women and girls.

EQ10: To what extent were actions coherent with relevant CFGB & GAC environmental guidance and policies in place during the period under evaluation?

While GAC increased requirements for environmental assessment, only a few occurred, and some of these lacked practical considerations and delayed the best interests of communities. However, some of the increased environmental efforts made by CFGB showed successes and warrant further support for continued capacity building, although the use of ‘environmental assessments’ is not always relevant, and a flexible approach to their application should be applied.

• Recommendation 35: GAC support to CFGB to increase practical efforts on reducing the environmental impacts of humanitarian FA programming, including:
  • Increasing efforts around changes in environmental practices related to general operation (e.g., environmental impacts of offices and travel)
  • Promotion of information technology and remote methods for M&E, capacity building and information and lesson sharing.
  • More holistic, low-tech fuel-efficient cooking awareness raising and support - Including spreading the knowledge on the need to move towards the use of food types that require shorter cooking times.

• Recommendation 36: More careful consideration of the need for environmental assessments.

Management Responses

Department’s response: The Department took note of the Contractor’s findings, conclusions and recommendations and has shared them with relevant stakeholders for consideration.

Co-operation Partner response: The partner took note of the Contractor’s findings, conclusions and recommendations and has shared them within the organization(s) for consideration.

If you would like a copy of the evaluation report, please contact info@international.gc.ca.