Multi Stakeholder Platforms as System Change Agents A guide for assessing effectiveness

This Guidance Paper is intended for those who are actively working with or for agri-food multi-stakeholder platforms (MSPs). It offers a common language, a robust yet manageable approach, and practical advice showing how MSPs can better use existing evidence and processes to reflect upon their intended system-change role. The guidance is based on ideas from systems theory and evaluation practice, and has been tested in collaboration with four agri-food MSPs: Bonsucro, the Farm to Market Alliance (FtMA), the Scaling Up Nutrition (SUN) Movement and the Zambia Business in Development Facility (ZBiDF). We provide a brief overview of these MSPs on pages 8-9, and we are grateful for their willingness to engage in this learning. Further detailed descriptions of three of these ‘deep dive’ collaborations, along with structured scans of nearly 40 MSPs in food and agriculture, are available here:


Sharpening MSP Effectiveness
What are MSPs and why does assessment matter?
The challenge: Why assessing effectiveness is often unresolved Our approach to overcoming these challenges MSPs with whom the approach was trialled

Putting the Ideas into Practice:
Four steps for assessing MSP effectiveness: Step 1 Defining the effectiveness question Step 2 Developing or refining a theory of change Step 3 Identifying evidence of effectiveness and gaps Step 4 Setting out MSP contribution stories

Sharpening MSP Effectiveness
This Guidance Paper is intended for those who are actively working with or for agri-food multistakeholder platforms (MSPs).It offers a common language, a robust yet manageable approach, and practical advice showing how MSPs can better use existing evidence and processes to reflect upon their intended system-change role.The guidance is based on ideas from systems theory and evaluation practice, and has been tested in collaboration with four agri-food MSPs: Bonsucro, the Farm to Market Alliance (FtMA), the Scaling Up Nutrition (SUN) Movement and the Zambia Business in Development Facility (ZBiDF).We provide a brief overview of these MSPs on pages 8-9, and we are grateful for their willingness to engage in this learning.Further detailed descriptions of three of these 'deep dive' collaborations, along with structured scans of nearly 40 MSPs in food and agriculture, are available here: Deep Dive: The Scaling Up Nutrition (SUN) Movement, Indonesia (http://doi.org/10.18174/541803)Deep Dive: Farm to Market Alliance (FtMA) (http://doi.org/10.18174/541806)Deep Dive: Zambian Business in Development Facility (ZBiDF) (http://doi.org/10.18174/541804) The ultimate aim of the paper is to help those involved in implementing, brokering and supporting MSPs, as well as donors and other funders, to make more strategic choices about effective activities that contribute to system change at scale.We believe that platform leadership must prioritise such assessments as core to strategy development, rather than seeing them as monitoring and evaluation exercises, and that platform funders need to support more systemoriented reflection.The advice in this guidance paper is designed to enable these processes, in order that agri-food MSPs live up to their promise of helping bring about necessary food system transformations.

What are MSPs for and why does assessment matter?
The food and agriculture sector is increasingly facing complex, systemic problems requiring structural change at multiple (global, national, local) levels 1 .MSPs are a recognition that transformation in complex systems (Figure 1) cannot be achieved through simple or technical fixes, which are likely to have insufficient or unintended results.Instead, change requires new forms of governance which bring stakeholders together to plan and act in new ways 2 .Each participant contributes their goals, perceptions, priorities and capacities.Through engagement and dialogue they learn together about the nature of complex problems and potential solutions, and gain a better and more holistic understanding of the context in which these solutions are to be implemented.
Importantly, MSPs cut across traditional public, private and civil society boundaries.For example, the SUN Movement brings together government, civil society, the United Nations (UN), donors, business and researchers across 60 countries globally, with the aim of ending malnutrition in all its forms.While MSPs are similar to partnerships, they are much broader in scope, catalysing structural changes and coordinating the activities of different stakeholders over longer time horizons.
The Zambia Business in Development Facility (ZBiDF), for example, was created as a platform to support, catalyse and scale multiple cross-sector partnerships involving business, government, donors, and civil society, in order to deliver on the country's development goals, such as creating employment and reducing poverty.
Since 2000, much resource and expectation has been invested in agri-food MSPs (Figure 2).As the field has developed, however, so has the call for more critical reflection regarding whether MSPs are effectively addressing the challenges that led to their creation and collectively enabling the realisation of global development goals.'How do MSPs actually contribute to impacts?', 'What are the right monitoring and evaluation approaches to assess these linkages?',and 'How can findings be used for timely course corrections?' are some of these critical questions 3 .As this Guidance is published, the UN's 2021 Food Systems Summit is in preparation; another moment when the global development community will be reflecting on effective mechanisms for achieving system change.

Stakeholders
The challenge: Why assessing effectiveness is often unresolved Despite the growing recognition of the need for this reflection, MSPs still too often rely on the simple narrative that 'complex, systemic challenges can be more effectively tackled by MSPs than by single actors'.As a result, we lack understanding of the effectiveness of MSPs against their intentions and goals, and relative to other ways of working on these goals.Monitoring of MSPs often concentrates on activities, outputs and structures, rather than on outcomes and the contribution of MSPs to such outcomes.This narrow focus is also driven by funder and stakeholder demands for direct, quantitative and attributable results that do not align with the contributory role that MSPs are more likely to play in complex food systems.
The problem is that our usual tools for measuring effectiveness are not fit for this purpose.They rely on a linear logic -generally involving tracking changes in key performance indicators -which is poorly suited for acting systemically.They are often applied retrospectively, in an attempt to account for the results of an MSP, rather than for real time learning.They put the focus on what we can measure -our activities and outputs, and also (in some cases) macro level agri-food system indicators.However, the relationships between MSP activities and system level effects are neither simple nor direct.Many different economic and political factors, including but not limited to the MSP, collectively shape food systems.These other factors can dampen or amplify MSP efforts, meaning that in some cases early achievements will later falter and fail, while in other cases, almost imperceptible changes create pressures that suddenly take off.
Because these challenges seem difficult to resolve, MSPs often ignore them.As long as available indicators are moving in the right direction, we assume that the MSP must be doing some good.Efforts to improve assessment have been piecemeal, with no single accepted methodology.Yet a key justification for investing in MSPs rests on their potential to bring about system change.

2
Source: Authors' own analysis based on structured scans of public information on MSPs in the food and agriculture sector

Our approach to overcoming these challenges
To overcome these challenges and better understand effectiveness, we need an approach that responds to three core questions, both for individual MSPs and for MSPs as a whole.These questions are: 1. What changes in the relevant agri-food system have occurred?2. Are these changes transforming the system towards long-term goals, and to what degree? 3. Has the MSP made a difference to these changes, and if so, how?
Our response, which is detailed in Section 3, is based on building 'contribution stories' to assess and validate or challenge different elements of an MSP's 'theory of change'.
A theory of change (ToC) has been described as 'ideas and beliefs people have -consciously or not -about why and how the world and people change'4 .They are hypotheses, informed by individuals' perceptions of reality, which in turn are guided by a mix of beliefs, evidence and assumptions.In the case of MSPs, the ToC sets out the different short-and longer-term effects through which the platform is expected to catalyse system change.A meaningful ToC enables MSPs to plot their course and measure progress against it, while keeping their strategy under ongoing review, adjusting or redefining it as new experiences and learning emerge.Yet many MSPs seem to lack an explicit and meaningful ToC, relying on instinct and pragmatism where the exact pathway to transformation cannot easily be known.Where ToCs do exist, they are insufficiently reviewed.
Contribution stories are 'snap-shot' narratives that describe observed changes in the agri-food system at a particular point in time.They offer a reasoned discussion of why and how changes are happening, considering the plausibility that the MSP has played a role in contributing to these outcomes, amongst other causal factors.They encourage critical thinking about the dynamics that promote change, make understanding of causal relationships between actions and results more explicit, and offer learning to better align MSP activities and desired impacts.

2
MSPs with whom the approach was trialed  At the time of the deep dive, ZBIDF was facing an uncertain future, as its primary funder, SIDA, had postponed a decision to extend funding, questioning ZBIDF's longer-term effectiveness.
The deep dive focused on the C2F partnership.
Started in 2016, C2F aimed to create employment and reduce poverty by increasing production and processing of cassava, while adding value to the crop.It included large and small business, financiers, NGOs, research institutes and public agencies.

Deep dive question:
How had ZBiDF created value towards its main goal of reducing poverty, by establishing and supporting cross-sector partnerships?

SCALING UP NUTRITION (SUN) MOVEMENT* ZAMBIA BUSINESS IN DEVELOPMENT FACILITY (ZBIDF)*
Putting the Ideas into Practice:

Four steps for assessing MSP effectiveness
In this section, we set out four steps for credibly assessing the effectiveness of MSP contribution to transformational sector changee (Figure 3).We also draw on practical experiences from our 'deep dive' collaborations with Bonsucro, FtMA, the SUN Movement and ZBiDF to illustrate our approach 5 .
3 5] See also on Mayne, J. 2008.Contribution analysis: An approach to exploring cause and effect, ILAC Brief 16.Institutional Learning and Change (ILAC) Initiative; and Patton, M.Q.2012.A utilization-focused approach to contribution analysis.Evaluation, 18(3): 364-377.These sources provided the theoretical and methodological basis for the approach.
Step 1. Define the effectiveness question The question of effectiveness asks whether an MSP and its stakeholders can reasonably conclude that the existence of the MSP and delivery of its activities are making a difference to challenges facing the agri-food sector.The exact formulation of the effectiveness question, however, will depend on a number of factors: • The key audience(s) of the assessment and the level of proof/confidence they need to make relevant decisions.Audiences might include managers, funders and wider publics, for example.• The areas of focus, e.g. an MSP's global operations vs one specific country or specific value chain(s).• The phase of MSP development.In very early stages, assessments may focus on MSP arrangements and capacities.Later, focus will shift to activities and outputs (e.g. products and services), and finally to intermediate or longer-term behavioural changes.However, even at an early stage, it is important to assess whether the conditions for broader transformation are being created, as discussed below under 'targeting scale from the beginning'.
In the case of Bonsucro, for example, the MSP wanted to know whether they were reaching all

What is the MSP theory of change?
4. What is the story of MSP contribution to change?
1. What is the specific effectiveness question? 3 key stakeholders, and not only frontrunners, in line with their strategy to promote sustainable transformation across the full sugar sector.Therefore, the effectiveness assessment focused on the output and intermediate outcome levels, and the causal assumptions between them.
In the case of ZBiDF, despite achieving a number of key results, the government of Zambia was not convinced that it was influencing private sector willingness to collaborate and invest in public-private partnerships.The deep dive provided insights regarding these outcomes and underlying dynamics.
More broadly, effectiveness questions may go beyond MSP actions and resulting changes along intended transformational pathways.They may ask whether these transformational pathways together are sufficient to deliver system change or whether further pathways are required, e.g. for different system elements or geographies.They may also ask whether the initial vision for the transformed future system is still appropriate or needs adjustment, e.g. to avoid unintended negative consequences.

Step 2. Develop or refine MSP theory of change
Having set out the effectiveness question, the next step is to develop or refine the theory of change.This process includes critically reviewing the existing theory of change and its underlying assumptions or developing a new one if needed.Engaging MSP participants and other stakeholders in this process will both strengthen understanding and learning, and enhance accountability.

Define boundaries, goals and pathways
The theory of change needs to define the boundaries of the system that the MSP intends to influence, the transformation goals for this system, and the pathways through which these goals are to be achieved.Who should do what, when and how?In some cases, a well developed global or top-down theory of change already exists, often developed during project design or approval.In this case, the focus is on critically reviewing and updating this existing theory of change, ensuring elements are specific enough to monitor and assess, but not so detailed that it is unwieldy.
In the case of Bonsucro, a theory of change already existed and was critically reviewed.Were Bonsucro's activities, products and services contributing to change?Did they reach the right target groups?Did these groups use them?What were the constraints or barriers to further change?How could these be addressed, and who needed to be involved to make it happen?
The conclusion was that the existing theory of change did not provide enough detail to distinguish the specific MSP outputs and how these were intended to build capacity or enable stakeholder action.A revised theory of change (Figure 4) was built based on Bonsucro tools and services.
For a new MSP starting up, the thought processes would be similar but start with defining higher level objectives, rather than interrogating the existing theory of change, and working backward to plot pathway(s) towards these goals.

Consider different levels
Often, MSPs have ToCs at the broadest possible level.For example, global MSPs have a global ToC; while domestic MSPs will set out a national level ToC.Sometimes these are more of a visual statement of intent than an actual theory of change, developed primarily through top-down processes.However, these broad visions are unable to sufficiently define pathways appropriate for assessing effectiveness in the diverse contexts in which an MSP is active.Instead, ToCs or mini-ToCs should be developed at more granular levels, such as national, local or sectoral.As the boundaries of the level become narrower, the ToC will get less abstract and include more detail regarding activities and outcomes.
In the case of Bonsucro, the focus was not geographic, but on detailing four critical pathways (Figure 4) related to key stakeholder groups: farmers, millers and sugar cane buyers.A fourth 'national' pathway was also distinguished, focused on facilitating and supporting national, integrated sugar cane sector improvement approaches in selected high potential countries.As a result, Bonsucro developed a detailed matrix of outputs, outcomes and impacts, including possible indicators, across these pathways.

Define key assumptions linking MSP activities with expected results
Assumptions are beliefs or expectations that are embedded within the ToC.It is important to make explicit critical causal assumptions, which explain how different parts of the agrifood system are expected to react to specific MSP activities and outcomes generated.Once explicit, these assumptions can be reviewed and refined (or overhauled) over time as better understanding emerges.
For example, in the case of FtMA, the ToC included an assumption that farmers and companies in different segments of the value chain had incentives to follow through on their commitments.On further examination, evidence suggested that this assumption held true for some parts of the value chain, including input suppliers, who identified a clear value proposition for working with FtMA.However, mixed perceptions of benefits among farmers helped explain a lower than expected delivery on contracts.FtMA also assumed that MSP arrangements would support engagement with national governments.However, in Tanzania, government engagement was yet to materialise.
Note that the type of assumptions referred to here are causal 7 , in that they relate MSP processes to changes in actor behaviour and other dynamics in the sector.Table 1 provides example of causal assumptions identified through the deep dives.
Initially, many causal assumptions in the model may either be unrecognised or completely hidden, but later become apparent through experience.By assessing and validating or refuting these causal assumptions on an ongoing basis, i.e. regularly revisiting and revising the ToC and its assumptions, MSPs build better understanding of the system they are trying to influence and how they can effect change.
7] 'Causal' assumptions are different from 'operational' assumptions that deal with the external context which is out of our control but may affect success, e.g.political or market stability.

Target scale from the beginning
The potential for large-scale and systemic change is often what justifies funders' and stakeholders' support for MSPs.Yet system transformation requires that MSPs not only support a small group of participating companies or farmers, or small pockets of good practice, but create change across sectors, value chains and/or geographies.Take the cases of Bonsucro and FtMA, for example.FtMA was seeking the establishment of national alliances in multiple countries, with a goal of empowering 1,500,000 farmers in 10 countries.Bonsucro wanted to increase the percentage of the world's sugarcane land involved in endorsed improvement schemes.As independent, often smallholder, farmers are responsible for half of global cane production, Bonsucro needed to reach beyond larger farmers to also engage this harder to reach segment.
In both cases, however, measuring the number of farmers reached or percentage of land under improvement at any one time, was an inadequate indicator.It measured scale but not scale potential.Instead, the ToC needs to set out from the beginning how it intends to create the conditions for scale.For Bonsucro, for example, the strategy was to work with extension intermediaries and local leaders as a means of engaging smallholder farmers.This strategy and its causal assumptions need to be regularly assessed and reviewed, adapting and redefining strategies over time.

Engage MSP participants and other stakeholders
The process of reviewing and adjusting the ToC will inevitably start with those most involved in the day-to-day operations of the MSP, the secretariat.It will ideally include the management/executive level, functional experts and those with a mandate for monitoring and

Level of assumption Examples
• Stakeholders have a shared interest to transform the food system, but the environment does not enable collaboration evaluation.The quality of the process also depends on having a broader mix of perspectives to inform understanding of the agri-food system, its challenges and potential solutions.It should include adequate regional and national inputs, as well as perspectives from different participants, such as farmers, traders, manufacturers and retailers.Other stakeholders that are not participating in the MSP also have important perspectives, including regarding the likely impact of MSP activities and outputs.
As detailed ToCs can appear quite technical and may not be easy to understand for those not steeped in the jargon of inputs, outputs, outcomes and impacts, it can be helpful to work with narratives.These narratives could capture the whole ToC, or describe specific parts of the system and its pathways, and be used to gather insights from stakeholders.Bonsucro for instance, introduced 'impact stories' for each pathway, in order to test the ToC logic and assumptions.These impact stories sat alongside the ToC, providing a more narrative explanation to show how change was intended to happen.They supported Bonsucro's engagement with stakeholders, strengthening understanding of system change and how it could be assessed.

Step 3. Identify evidence of effectiveness and gaps
Once the ToC is developed or refined, meaningful evidence of changes and of the contribution that the MSP is making to these changes needs to be gathered and assessed.MSPs are often already collecting a significant amount of information, including both quantitative data and other more qualitative evidence, especially for elements that are not easily measured.Table 2 lists examples of quantitative and qualitative evidence identified in the four deep dives against the different levels of their ToC.Interesting examples include: • 'Broker journals' (personal reflection reports by the partnership brokers on events, meetings, lessons and risks).ZBiDF used these to track MSP developments, lessons learned and emerging risks, offering evidence of progress against ZBiDF annual plans.The journals included commentary on the dynamics of relationships among partners and level of partner commitment that could never be captured through quantitative data.
• National level self-assessments implemented through broad stakeholder participation.The SUN Movement uses a standardised reporting template and 'joint annual assessment' (JAA) process.These ask a country to rate its performance on different indicators, using a scale from 0-4, and providing a qualitative explanation justifying each rating.
• Surveys with participants, or data from groups of participants regarding behaviour change.FtMA has access to information on farmers applying improved technologies and better management practices, generated by field officers working directly with farmers' organisations.Bonsucros uses annual member surveys to explain quantitative results collected through its reporting tools.
Once existing evidence is identified, gaps will emerge which require alternative sources of information.In particular, gaps commonly appear at the level of intermediate outcomes.Step 4. Set out MSP contribution story Data alone is insufficient to provide a meaningful understanding of effectiveness, as the key question of how an MSP affects observed results remains unanswered.For example, simply tracking the use of credit does not explain why there is a (low or high) uptake of loans.Is it related to loan cost, perceived risk and/or availability of information?And how did the MSP affect these factors?
Simple cause and effect answers are unlikely, especially at the level of longer-term outcomes.
A role for the MSP cannot be assumed, but needs to be critically reviewed and validated.To do so, causal links and assumptions in the ToC need to be examined in a structured way, assessing the degree to which each is supported or refuted by the evidence.In particular, this process requires a careful consideration of the role that other factors play in the results, and an emphasis on understanding why and how the processes being evaluated work or not.'Contribution stories' can act as the basis for this assessment.

What are contribution stories and why are they important
Contribution stories are logic-based narratives that describe observed changes in the food system, both anticipated and unanticipated, and set out evidence of what contributed to these results, including but not limited to the role of the MSP.While definitive proof of impact is not possible, the emphasis is on exploring quantitative and qualitative evidence, of the types described above, to understand the likelihood or plausibility of a relationship between the MSP and observed changes.This evidence should be critically reviewed to assess whether it offers a sufficiently robust case to be confident of connections at different steps in the chain.
Box 1 provides a summarised example of the contribution story developed for FtMA.

3
Full contribution stories are found in our deep dives.Interestingly, we also found that some MSPs are experimenting with a narrative approach.In particular, SUN has been developing 'Stories of Change in Nutrition'.These are structured case studies to understand what drives impact in reducing undernutrition and how enabling environments and behaviour change can be cultivated and sustained 9 .
The ToC also assumes a good value proposition exist for farmers, although evidence is mixed.Farmers welcomed higher quality inputs and saw prices and predictability of markets improve.However, they complained of inflexible arrangements.Some believed inputs were unsuitable for particular plots, or wanted to sell produce at different times of the year, outside FDC arrangements.In 2016/17, 13,000 MT of maize was committed through FDCs, although only 8,300 MT (64%) was actually sold to FtMA partners.This level compares favourably, however, with 2015/6 when only 15% of maize committed was delivered.
As of late 2018, FtMA had developed national alliances in four countries and reached 142,000 farmers, with new governance arrangements designed to promote expansion and support stronger national alliances.
However, while national stakeholders (input companies, banks, offtakers, farmers) are clearly engaged, the arrangements are reliant on the ongoing involvement of FtMA.No progress was identified on advocacy with the Tanzanian government.As MSP structures were newly developed at the time of the research, the assumption that these arrangements will create the conditions for scale needs further evidence and ongoing scrutiny.

How are contribution stories produced ?
The production of an MSP contribution story involves assembling identified evidence in a draft narrative which assesses impact pathways in the ToC, validating this narrative with stakeholders and identifying evidence gaps, exploring additional evidence to fill gaps and strengthen the narrative, as well as highlighting remaining areas for future data collection 10 .These steps are explained below.
Assemble existing evidence to develop a first outline contribution story: Based on the effectiveness question defined (step 1), available evidence is assembled against the ToC (step 2).This information includes evidence of the role played by MSP activities and outputs in observed changes, as well as other factors that may have influenced results, such as new policy developments, and economic or market factors.This evidence should consider changes along the pathways in the ToC; and also whether these pathways together are sufficient to deliver the desired system change.
In the context of complex systems, the outcomes of interventions cannot be fully known or predicted in advance, so there is significant potential for unintended effects or spillovers not foreseen in the ToC.These may be either positive or negative with respect to the transformation being sought.Therefore, the development of the contribution story with FtMA in Tanzania involved looking not only at expected outcomes, but also sought to identify other significant development.It included a relatively open ended discussion of key changes in the maize sector, as well as exploring the expected outcomes in the FtMA ToC.Through this process we identified, for example, buyer investment in transport and logistics and input provider investment in extension staff as unforeseen outcomes.

3
Review with stakeholders: The assembled evidence leads to a first outline of the contribution narrative as the basis for internal and external consultation.The aim is to test and validate or refute elements of the contribution story.Stakeholders review the draft, provide feedback on whether the connections being described ring true, and offer further learning regarding what has worked and why, as well as confirming or questioning causal assumptions in the ToC.
The quality of this process depends on systematically reflecting on the evidence and feedback from stakeholders, considering the plausibility of assumptions and exploring factors outside the MSP that have influenced the results.Key questions are11 : 1.Where does strong evidence support specific links or assumptions in the theory of change?Indicators of 'strong' evidence include: • the quality of the evidence itself (anecdotal vs consistent); • relevance of the evidence to the core logic of the MSP, and to the link being assessed (e.g. has reasonable time passed between dissemination of new agricultural techniques and observed productivity improvements to support a connection, or is it too soon to reasonably expect that one could have led to the other); and • wide acceptance of its validity among stakeholders with different perspectives on the agri-food system.

What other information challenges these links, such as alternative explanations of results not linked to the MSP?
Stakeholder review may be interactive, such as in a workshop setting.The value of such engagement is that it not only strengthens the contribution story but can contribute to more holistic perspectives, shared learning and trust among MSP participants.However, there are times where a more segregated approach may be valuable.This may be the case if unequal power relations among stakeholders are difficult to manage, with a risk that important issues or conflicts are silenced or sidelined.In this situation, interviews or other individual feedback mechanisms led by the secretariat or an MSP broker may be more productive, although it raises questions of how to fairly weigh up and integrate diverse perspectives.
Identify additional evidence and revise contribution story: After the first review, new questions and gaps in the evidence will emerge.Based on these questions, further information and evidence can be sought, and the contribution story revised following the two steps above.
In some cases, it will also be useful to review and adjust the theory of change.The resulting contribution story will now include more robust conclusions regarding MSP effectiveness, and can be used to identify lessons to inform future strategy and decision-making.
Improve evidence base for future contribution story development: Even in the revised contribution story, a number of areas of the ToC and its assumptions are likely to lack sufficient evidence to deliver strong conclusions regarding these aspects of MSP effectiveness.These gaps should be prioritised, with appropriate indicators developed to fill priority gaps.Once again, both qualitative and quantitative sources should be considered (step 3).As this evidence is developed, it will enable a deeper assessment of effectiveness in future.

Six Lessons and Insights
Section 3 has set out a seemingly neat and linear process of assessing effectiveness using ToCs, existing and new sources of evidence, and contribution stories.However, our deep dive experiences were much more diverse, messier and more iterative.Below we present six lessons and tools derived from these experiences to aid those working with or for agri-food MSPs to have conversations regarding effectiveness.

Strategic embedding
There is a risk that even when MSPs have

Recurrent revision of the theory of change
In our guidance, we make the case for ToCs to be adjusted or redefined over time.This advice might seem counter-intuitive -as if it implies shifting goalposts.However, in a complex and ever-changing system there is only so much that can be understood and planned through initial analysis and design work.The ToC will always reflect only a partial understanding of the agri-food system.This system is also ever changing, in response to external events and also -hopefully -the

Monitoring unintended consequences
It is highly likely that in complex agri-food systems, MSPs will have positive and/ or negative consequences that are not foreseen in advance.In order to understand effectiveness and revise and adapt the ToC accordingly, it is important to be open to identifying these unanticipated changes and feeding them into assessment processes.
Such outcomes might include new business investments, changes in quality of life, the environment or food security, or new gender roles not foreseen in the ToC.
One technique that can help with identifying outcome level changes, both intended and unintended, is called 'outcome harvesting' 13 .
This approach gathers and then iteratively assesses change narratives from a variety of stakeholders with different perspectives.
Since it is a qualitative approach, it is also helpful for identifying outcomes that can be difficult to measure, such as behaviour and relationships changes, or the implementation of new practices.Outcome harvesting can be easily added to the process of contribution story development, since the logic involves asking a set of predefined questions about changes, assembling evidence on these changes and then assessing the contribution of the MSP versus other relevant factors.
Validating the analysis through engaging multiple stakeholder perspectives is also important.

Conclusion
Extensive critical reflection on food systems -global, national, local -point to the need for structural food system change at all levels.As we point out at the start of the guidance paper, this recognition is often the driving force leading to the creation of agri-food multistakeholder platforms.Our interest in MSP 'effectiveness' is therefore not about MSPs being effective in a purely 'operational' sense.We aim to help MSPs better play this system change role.
To do so, MSPs and their stakeholders need to think differently about assessing effectiveness.They need to maintain a clear and constant focus on the bigger picture of necessary system change and avoid getting lost in operational level work, despite inevitable short-term pressures and setbacks.We hope to stimulate MSPs to consider their relevance as they reflect on their effectiveness.We also hope the guidance will provide MSPs with both argumentation and practical options to get active support in this endeavour from funders.
Our guidance is rooted in theory and tested in our 'deep dive' collaborations, which should give MSPs confidence that the approach is both reliable and likely to be implementable in their own context.It emphasises MSPs being more deliberate in their own thinking, and in their engagement with stakeholders, regarding how they are catalysing change over time.This requires a dynamic perspective, with regular updating of the ToC as MSP understanding grows.Ultimately, if an MSP is being effective, the agri-food system will be changing and therefore the ToC will need to change too.
Remember, MSPs are a form of governance intended to respond to the complexity of agrifood systems and their challenges.Their multi-stakeholder design implies more distributed leadership and innovation, and more participatory interactions.Being dynamic also implies an evolutionary logic of trialling actions, monitoring stakeholder responses and then revising the ToC to build on successes and reformulate assumptions where results are not achieved.These elements need to be better reflected not only in the ToC but also MSP governance arrangements, with farmers' organisations, for example, able to participate in a meaningful way 14 .
Finally, a strategic focus on effectiveness needs to extend beyond individual MSPs to the development and agrifood sectors as a whole.For too long we have accepted the mantra that complex challenges require multi actor solutions, without adequately testing this premise.Donors and funders have largely failed to provide the right imperatives for more critical reflection.We need to ask ourselves, are MSPs as a governance form delivering on the vision of more sustainable and equitable food systems?Are they proving to be a better governance form than other more conventional alternatives which have fewer transactions costs?Do MSPs create the conditions for more participatory governance?Before we invest further resources and expectations into MSPs, urgent answers are needed to enable multistakeholder platforms to play appropriate and effective roles in bringing about food system transformation.

2 1 ]
Such as Watts, Nick et al. 2018.The 2018 report of the Lancet Countdown on health and climate change: shaping the health of nations for centuries to come.The Lancet, 392(10163): 2479 -2514; The Food and Land Use Coalition.2019.Growing Better: Ten Critical Transitions to Transform Food and Land Use, The Global Consultation Report of the Food and Land Use Coalition September 2019; FAO, IFAD, UNICEF, WFP and WHO (2020) The State of Food Security and Nutrition in the World 2020.Transforming food systems for affordable healthy diets.Rome, FAO.2] See Rhodes, R.A.W. 1997.Understanding Governance: Policy Networks, Governance, Reflexivity and Accountability.Open University Press, Buckingham; Koppenjan, J.F.M., Klijn, E.-H., 2004.Managing Uncertainties in networks.A network approach to problem solving and decision making.Routledge; Jessop, B. 2000.The Dynamics of Partnership and Governance Failure, in: Stoker, G. (Ed.),The New Politics of British Local Governance.Basingstoke, Macmillan.Press, Buckingham; Koppenjan, J.F.M., Klijn, E.-H., 2004.Managing Uncertainties in networks.A network approach to problem solving and decision making.Routledge; Jessop, B., 2000.The Dynamics of Partnership and Governance Failure, in: Stoker, G. (Ed.),The New Politics of British Local Governance.Macmillan, Basingstoke. 2

Figure 1 :
Figure 1: Visual representation of a system transformation

Figure 2 :
Figure 2: Number of new agri-food MSPs created(1996-2015) Pilot was hosted by the African Management Services Company (AMSCO) under auspices of a Champions and Advisory Group (CAG) of influential stakeholders from Zambia's crosssector environment Deep dive focus: Indonesia Already in its second phase (2016-2020), at the time of the deep dive, the SUN Movement was shifting the focus of its strategy from creating national enabling environments for nutrition to delivering on implementation.Indonesia had been active in the SUN Movement since early in phase 1, with the engagement of all SUN networks (UN, donor, civil society, business, academia), a well-functioning country platform that coordinates government and network efforts, and up to date data.The focus of the deep dive was on the Indonesia's multisector, multi-actor strategy to reduce stunting across the country Deep dive question: How does the SUN Movement use its monitoring, evaluation, accountability, and learning (MEAL) framework to assess and communicate effectiveness in phase 2?

Figure 3 :
Figure 3: Putting ideas into practice

Figure 4 :
Figure 4: Revised Bonsucro Theory of Change • The main barriers are lack of awareness, communication, trust and knowledge; requiring facilitation and technical support • The right stakeholders are being reached through the platform and its members • Sufficient incentives are created in the system for stakeholders to take up products and services • New mindsets, incentives and capacities drive behaviour change; commitments are honoured • Mutual benefits are possible, encouraging different stakeholders to align around common objectives • New information, data and transparency are key drivers of change • Continuous improvements in agreed activities create intended impacts • Mutual benefits support long-term sustainability • The system is resilient to shocks • Sustainable change is achieved at scale • Success breeds wider stakeholder support, replication, scale • Innovations become the new 'normal'
MSPs are by definition meant to be participatory: it's all about bringing together multiple voices around collective action.The approach we describe both depends on, and can contribute to active participation of all members and stakeholders of an MSP.While many members have limited time to engage in an MSP, being part of an assessment process can motivate them to become more active.Developing and verifying a ToC is the first moment where engaging a wide MSP membership is essential.It is the first moment where members say out loud what they expect the MSP to work to achieve, in line with their own priorities and goals, and an opportunity to check for alignment.The more agreement there is about the intermediate outcomes and final impact, the more interested members and stakeholders will be in knowing whether the MSP is making progress.Actively asking members how they see change and what the MSP should do about it will strengthen the voice and influence of all members.The same holds true for the manner in which contribution stories are developed and checked: as explained, this can be done only by or with the secretariat, but it is most effective as a shared process.Three different groups should join in this process: members, stakeholders and funders.All three are potential 'beneficiaries' of the MSP, not just the farmer groups or low income households which are often referred to as beneficiaries.Stakeholders include those affected by what the MSP does, but who may not normally have influence over the MSP.Funders need to be drawn in to help keep minds focused on the higher purpose of the MSP that the assessment considers and give confidence that the right steps are taken to keep MSPs on track for reaching these longer-term goals.
deliberately and clearly articulated and shared their ToC, they may fail to use it to guide strategy.For one thing, it can be difficult to keep wider systemic change goals in mind when focused on short-term operational targets.A focus on initial piloting of activities and meeting short-term funder expectations can lead to a neglect of longer-term ToC elements necessary for system change.Another concern is that ToC development and review is entirely delegated, for example to a monitoring and evaluation team or to external consultants, remaining far removed from strategic decision-making.To be useful, the ToC must have a central place in MSP strategy and governance, even if process management resides with a monitoring and evaluation function.The means to ensure this central role for the ToC will vary between MSPs.One measure, for example, is to integrate a small set of key effectiveness indicators from the ToC within strategy processes.Bonsucro, for example, identified nine key outcome and assumption indicators, taking into account programme priorities, available resources and a realistic timeline.Strategic indicators can be reviewed in governance meetings, incorporated into member surveys, and discussed regularly with stakeholder groups. 4 MSP's activities.As a result, regular adaptation of activities and pathways, indicators and metrics, and eventually also the vision are likely to be required.It is impossible to say exactly how oftenor when it would be appropriate to review the ToC.However, an analogy with another familiar context -that of software releases -may help.In software development, a new software release will be version 1.0.However, the developers' work will continue, adjusting the software as they learn how people are using (or abusing) it and how the software is responding, as well as in reaction to new external developments.Slight 'bug fixes' of the first release will often quickly be required (version 1.0.1),while more substantial changes will come with the addition of significant new features (version 1.1).At some point, however, either these changes will become so numerous or a more fundamental advance will be developed, leading to the introduction of a new version (2.0).For MSPs, the ToC works like an operating system.The first ToC is version 1.0.However, fairly quickly some adjustments to specific activities or interventions will be needed as problems are identified and fixed (ToC 1.0.1).Over time, new pathways are required, with new outputs and outcomes addressing previously neglected aspects of the agri-food system (ToC 1.1).Finally, the question is whether the pathways are adding up to the vision or, as in the case of Bonsucro, a strategic refresh and revised focus is developed.Bonsucro's new 2020 strategy is analogous to a new version release (ToC 2.0).

Table 2 .
Quantitative and qualitative evidence to monitor effectiveness These involve new stakeholder actions and behaviours aligned with MSP goals, and underlying causal assumptions.Behaviours might include the development of new policies or investments in new innovations, which emerge as a result of new understandings, new economic or social norms, or new incentives generated by the MSP.8] Note, however, that this statistical data is often available only after a time lag of perhaps 4-5 years, with implications for its usefulness as part of an assessment.