Highlights

• The agricultural research for development (AR4D) domain is becoming increasingly complex, and theory of change (ToC) approaches can provide critical guidance through the transformation maze concerning engagement, partnership, and research.

• Most of the major benefits that accrue to the use of ToCs relate to internal learning within project teams.

• Finding the balance between applying a ToC that is both useful and time- and resource-smart is challenging and may need iteration to get right.

• Quantitative impact assessment methods must be blended with qualitative methods in ToC-based AR4D so that evaluation becomes about both process and numbers, and new methods are needed for blended evaluation.

• The evidence base concerning the efficiency, efficacy, and failings of ToC-based AR4D urgently requires further development and synthesis.

15.1 Introduction

Despite the substantial improvements in human well-being that have occurred over the last thirty years, there is broad agreement that our food systems are not on track to reach the Sustainable Development Goals by 2030. Several different reports published since 2020 address the sustainability, economic, and policy aspects of food systems. These reports converge on one core message: we need to transform our food systems.

Agriculture is generally recognised as a key entry point for effective poverty-reduction strategies, and the adoption of improved practices, technologies, and policies has had strong, positive impacts (Alston, Reference Alston, Pardey and Rao2020; Christiaensen et al., Reference Christiaensen, Demery and Kuhl2006). Even so, food insecurity and rural poverty are persistent challenges (FAO et al., 2021). The reasons are many and complex but can be encapsulated in the observation that the rate of change in many socio-economic and Earth system trends appears to be accelerating to the point where the past is no longer a good indicator of the future (Steffen et al., Reference Steffen, Broadgate and Deutsch2015). Agricultural research for development (AR4D) faces big challenges in prioritising, targeting, and implementing activities of a type and scale that can make the best use of the many billions of dollars required to ensure food and nutrition security for all in the face of economic and zoonotic shocks and of a warming and increasingly variable climate (GCA, 2019).

The CGIAR research program on Climate Change, Agriculture and Food Security (CCAFS) was a relatively early adopter of theory of change (ToC) and impact pathway thinking as a way to orientate research, engagement, and capacity-development activities, in the quest for a food-secure future and ‘best use’ of financial resources. Robust evaluation of the effectiveness of a ToC approach is still some way off; CCAFS’s experience of it has, however, generated lessons that could enhance its effectiveness at scale. In the next section, we provide some background on the theory of change and briefly discuss its implementation in CCAFS, including how the approach was modified through time, driven by the need for pragmatism and nimbleness in reacting to change. We draw out some of the lessons learnt regarding monitoring, evaluation, and institutional and behavioural change. Examining some implications for partnerships, engagement, research, and institutional structures, we conclude with a discussion of the future implementation of ToCs in AR4D food-system transformation programmes.

15.2 Theory of Change Background

The process of AR4D is a set of research activities that produce the outputs used to contribute to behavioural change, or outcomes, via changes in the knowledge, attitudes, skills, and practices of development practitioners, extension services, farmers, and policymakers. These behavioural changes lead to impacts such as increased food security and reduced poverty. The processes that link inputs, outputs, outcomes, and impact are usually much more complicated and iterative than this (Thornton et al., Reference Thornton, Schuetz and Förch2017). How this process has been framed has changed through time, driven mostly by development and funding agencies seeking to heighten accountability to their constituents, attribute impacts, aggregate results, and establish incentives and processes to stimulate the use of performance information in management decision-making (Binnendijk, Reference Binnendijk2000).

With roots in program theory of the 1960s, ToCs have expanded to encompass a range of evaluation approaches introduced to explain why some development interventions created impact while others did not. The call for ToC-informed intervention design was triggered by the needs of evaluation practitioners who sought to understand how outcomes arise (Maru et al., Reference Maru, Sparrow and Butler2018). Another contribution to the evolution of ToCs was the rise of participatory approaches that catalyse positive development outcomes via social learning (Kristjanson et al., Reference Kristjanson, Harvey and Van Epp2014). There is no single definition of a ToC and no set methodology; rather, the approach allows flexibility according to the needs of the user or implementer (Vogel, Reference Vogel2012). A ToC provides a detailed narrative description of a hypothesised impact pathway – the logical causal chain from input to impact – and how changes are anticipated to happen, based on assumptions as to how the world works. The process of developing a ToC ideally involves a range of stakeholders who try to articulate the linkages and assumptions between inputs and outcomes (Figure 15.1). Progress is continuously monitored and the ToC modified in light of unexpected or unforeseen changes. Approaches based on ToCs hold out considerable promise, even if more robust evidence for their effectiveness in delivering the desired outcomes is needed (Alvarez et al., Reference Alvarez, Jost and Schuetz2014; Thornton et al., Reference Thornton, Schuetz and Förch2017). A ToC is no panacea, but it can facilitate broad commitment to learning from individuals and organisations, widely seen as an essential element of sustainable development.

Figure 15.1 Theory of change cycle (Omore et al., Reference Omore, Kidoido and Twine2019, from O’Flynn & Sonderskov, Reference O’Flynn and Sonderskov2015)

There are several implications of implementing ToC approaches in practice. First, there is the need to formalise a project’s ToC by involving a wide range of stakeholders in its design. Second, the assumptions that underlie the ToC should be examined regularly, and adjustments made if needed – for example, new or different partners may be added, or a particular assumption may simply not hold. Third, an effective and efficient monitoring system must be established, which may be qualitative as well as quantitative. Fourth, space must be provided for project reflection and learning. We revisit such practical issues below.

15.3 Application of Theories of Change in the CGIAR Research Program on Climate Change, Agriculture and Food Security

At different stages in CCAFS’s evolution as a research program, from its design in 2008 to its end in 2021, it utilised various program theories (Figure 15.2 and Table 15.1). At the start, a log frame approach¹ was used to plan and monitor project activities across the portfolio. Projects’ annual plans and reports were collected, harmonised, and consolidated manually (Figure 15.2). Because project teams could make individual adjustments to shared templates, submissions lacked standardisation across the project portfolio. It became clear that another way of planning for and capturing outcomes was required, including engagement and capacity enhancement as key strategic elements of the work. Partners started experimenting with learning-based approaches within AR4D, recognising the need to include mechanisms that challenge ‘business as usual’ and support institutional learning and innovation, to ensure research contributes to development outcomes.

Figure 15.2 History of the use of theories of change in the CGIAR Research Program on Climate Change, Agriculture and Food Security, 2008–21

In 2013–14, the program piloted ToC approaches in one thematic area, involving six new multi-annual projects selected via a competitive process in two regions for gender-focused research (Jost et al., Reference Jost, Kristjanson and Ferdous2015; Thornton et al., Reference Steffen, Broadgate and Deutsch2014). These activities helped support a stronger focus on outcomes, especially behavioural changes in people, and made partners, engagement, people, and actors for change central to implementation following the ‘three thirds’ principle: a third of the effort spent on working with next-users to build relationships and define their needs, a third spent on the research itself, and a third spent on enhancing next-users’ capacity to take up research outputs (Vermeulen & Campbell, Reference Vermeulen and Campbell2015). A key element was encouraging the program’s researchers to consider and plan for the use of their research results by partners and stakeholders and to take responsibility for the findings being used. Reporting on project progress and results was designed regarding the key elements of a ToC: outcomes, outputs, compliance with program core values such as gender and social inclusion, and partnerships. Reporting was complemented with ‘outcome stories’ in which projects could explain the behavioural changes they observed and present evidence of their contribution to these changes. The program invested in building capacity to train project, thematic, and regional portfolio managers in using the ToC approach. The approach was extended to the entire project portfolio after one year of the pilot.

To strengthen the institutionalisation of ToC approaches, the program combined them with an explicit results-based, adaptive management monitoring, evaluation, and learning framework (Schuetz et al., Reference Schuetz, Förch, Thornton, Uitto, Puri and van den Berg2017). This allowed the program to continuously refine both the ToC and project plans, to react in an agile manner to lessons and opportunities as they arose. ToCs were essentially nested at multiple scales: global, regional, country, and project scales. The program invested in the development of an online system to collect planning and reporting data and information. This system enabled implementation of standards, aggregation of results, and guidance of processes in planning and reporting for accountability, learning, and decision-making. Allied with this functionality was a process of annual project performance evaluation. Projects were scored based on their contributions to outcomes – a heavily weighted variable – delivery of outputs, use of inputs, and compliance with key program values such as gender and social inclusion. The evaluation drew on internal leadership and external reviewers.

Several of these elements were modified and expanded in later years, by which time the ToC and adaptive-management approaches had been adopted throughout the program (Box 15.1). Annual planning and reporting were carried out using the online system, which was also adopted by several other research programs as well as CCAFS. The online system was further developed to provide some analysis and business intelligence information for project and program leaders, which could be evaluated against plans on a regular basis. Outcome stories became a key component of annual reporting, and planning and reporting templates stabilised, resulting in a largely consistent set of plans and reports in several programs’ latter years, covering a substantial proportion of the CGIAR system’s work. Project evaluations informed the stage-gating of investment decisions, that is, whether projects were to be stopped or continued.

15.4 Lessons for Success

Regarding ToCs within CCAFS, several factors contributed to their relatively successful use. Mistakes were certainly made, but the program embraced ToCs, including mainstreaming research that focused on linking knowledge with action, leaders who championed the approach, identification of key leverage points in program-impact pathways, and inclusion of local partners to ground the theories in reality and challenge the assumptions. The successful creation and use of ToCs require mindful and intentional implementation by a program’s leadership (Chapter 18). They must lead by example and demonstrate the necessary internal knowledge, skills, and attitudes towards processes for the creation and use of ToCs (Box 15.1).

In terms of using ToCs to lead transformation, identifying transformational leverage points is critical (Chapters 4–14), along with the partners who can deliver change. As such, ToCs that are more specific in pinpointing partners and desired changes will be likelier to lead to success. Strategic partners and targeted tactics to influence the behaviour of key persons and institutions, that is, ‘leverage points’, are essential, and ToC-process teams must consider these. For example, better sourcing transparency along food supply chains will require engaging with the private sector, but a ToC that specifies ‘the private sector’ as a partner will inherently be less successful than one that clearly names a major supplier in a specific chain, and possibly even identifies an individual or unit within that company as the leverage point on which to focus. In the South Asia regional program, for example, efforts to increase the uptake of index-based crop insurance were successful following direct engagement with one of the major insurance companies and its team. This process helped develop new thresholds for determining when policy payouts should occur. Different types of partners will play a role at different stages along impact pathways; therefore, the leverage points will also differ. For example, establishing local relations for site-based participatory research requires a very different leverage point than convincing a ministry of agriculture to include specific climate actions in its upcoming strategic plans. A ToC is useful to help teams think through these processes as clearly and intentionally as possible.

In these planning processes, ToCs can be generated using a top-down approach, involving only key project team members. However, undertaking a bottom-up participatory, inclusive approach that achieves critical buy-in from key stakeholders for implementation can be more rewarding, despite its challenges. In practice, both top-down and bottom-up approaches are needed. The bottom-up approach can provide insights from a broader group of stakeholders that highlight realities not apparent to those who first envisioned the project and that may have hampered success if not understood from the planning stage onwards. The top-down approach keeps sights set on the overall goals and targets that donors expect to see met. The ToC established at the beginning of a program or project may need to be reviewed at intervals, allowing for course corrections with partners before actions lag too far behind or veer off track. At the same time, keeping accountability at the level of outcomes is key for success; taking accountability to a more granular level may compromise the ability of projects to be agile and take advantage of opportunities as they arise, leading to potentially cumbersome administrative adjustments.

Setting up ambitious science-based goals is important for measuring targets with metrics that cover diverse outcomes – for example, productivity, nutrition, gender, and climate – but are also relevant to specific contexts. For a large research program, this may mean setting up a broad, inclusive results framework.

Adherence to a ToC approach requires investments of time and financial resources. It also necessitates persistence and purposeful revisiting of the ToC on an annual or regular basis to adjust as needed, given that it may be difficult to get right the first time. This can mean getting heavily involved in what can be a complex process for relatively short-term projects. For longer-term projects, the investment can pay off when there is enough buy-in from senior management, the process remains flexible, and critical leverage points and partners are identified from the beginning.

Having ToCs at multiple levels with numerous assumptions, ideally co-produced with partners, can be burdensome; some partners, such as the private sector, may prefer extreme streamlining of the ToC approach. In fact, ToC language may never be used when working with such partners but is nonetheless inherent in the collaboration. The bottom line is that a ToC approach must be as simple as possible while still adding value.

15.5 Theories of Change Looking Forward

Recent experience highlights the need for transformation, both of our food systems and how AR4D is done. Below we pose six questions related to key gaps and the next steps in optimising these approaches to foster transformation.

1. 1. How Broadly Do Theory of Change Approaches Need to Be Designed?

   For any large, complex AR4D program with multiple activities and partners, a ‘portfolio’ approach to ToC is appropriate, meaning not all activity areas in such a program need a ToC. For example, while upstream genomics research on organisms may not benefit much from a ToC, downstream plant breeding for traits such as drought or heat resistance would benefit greatly. In that case, a ToC could match the key characteristics of new varieties with the needs of a diverse range of small-scale farmers, to hasten and widen uptake.

2. 2. Is There Adequate Organisational and Institutional Support for Utilising Theory of Change Approaches?

   In designing and implementing new projects and programs, further investment in capacity development will likely be needed. The benefits for researchers and research partners seem clear, though institutional culture itself may need to be transformed for advantages to be realised. The funders of AR4D seem fully on board with the ToC approach and the benefits it can provide (Box 15.2).

3. 3. How Can We Build Theories of Change for Multiple Interventions at the Same Time?

   Transformation of food systems will likely involve bundling, including the bundling of technical interventions such as climate information services plus climate-smart agriculture (CSA), socio-technical bundles, or technology interventions plus their enablers (Barrett et al., Reference Barrett, Benton and Cooper2020; Herrero et al., Reference Herrero, Thornton and Mason-D’Croz2020) (Chapter 8). Examples from the literature of ToCs that address bundling are currently limited. Good examples that could be shared widely and modified for similar challenges and contexts could save research teams considerable time.

4. 4. How Ready Are Agricultural Research for Development Organisations for Transformational Change?

   Historically, AR4D organisations have been very effective in fostering incremental change. Such organisations may need to consider how best to work for transformational change and create enabling environments, including ToCs, that allow it. One challenge for large AR4D organisations working in geographically and politically targeted environments is linking the different initiatives and projects operating in the same environment to maximise synergies among them; pragmatic ToC approaches can help achieve this objective (Box 15.3).

5. 5. What Are Suitable Monitoring and Evaluation Methods?

   We need new tools for the monitoring and evaluation of transformation beyond the so-called gold standard econometric approaches, which may miss many of the complex impacts of the work of AR4D organisations. Examples of such tools are provided by Carneiro et al. (Reference Carneiro, Resce and Ruscica2020) using web analytics and high-level syntheses of outcome stories (Nowak et al., Reference Nowak, Poulos and Chan2021), and there is considerable potential for other big-data-assisted methods. Regarding the power of outcome stories, as noted above, these have developed over time, and their standards have improved. These do not replace impact assessments but are complementary, broadening the evidence base of impacts and increasing inclusivity. One example is the Kenya CSA work, where impacts were assessed using a mixture of soft and hard approaches (Okumu, Reference Okumu2021).

6. 6. What Is the Value Added by Using Theory of Change Approaches?

   We must continue strengthening the evidence base for the effectiveness of ToC approaches. Transformation takes time if it is not to be utterly disruptive, and rapid transformation without all enablers in place may be relatively ineffective. On the other hand, doing all the groundwork with well-facilitated, inclusive participatory processes can take considerable time, yet also contributes to the enabling conditions needed for interventions to succeed. There is little current evidence concerning the trade-off between the degree of use of a ToC, its utility, and its costs; this evidence would be very useful in future food-system transformation projects and program design.

15.6 Way Forward

The AR4D domain is becoming increasingly complex as it grapples with the need for food-system transformation. A ToC can provide critical guidance in planning and implementing projects and programs with respect to engagement, partnerships, and research. To make ToC-based approaches as effective as possible, two gaps in particular must be filled. First, quantitative impact assessment methods can be blended with qualitative methods so that evaluation becomes about both the process and numbers. At the same time, new methods need to be developed for blended evaluation. Second, the evidence base concerning the efficiency, efficacy, and failings of ToC-based AR4D urgently requires further development and synthesis, and the lessons must be applied broadly.