2.1 Status of research in sustainability tools

Research to date in sustainability tools covers four key spheres: reviews of tools and their application to case studies; comparative studies of specific tools; integration of tools into computational models; and frameworks on how to integrate different tools. There are also a diverse range of factors that can influence the way tools are currently selected.

A diverse scientific and institutional literature exists that reviews tools and their application to case studies. In sustainability, these tools are used to evaluate the environmental consequences of products, services, projects, policies and other anthropogenic activities. They tell us whether our actions, in effect, have consequences for the environment. They encompass established systematic examinations, such as LCA or ecological footprint analysis (Loiseau et al., Reference Loiseau, Junqua, Roux and Bellon-Maurel2012), and also emerging approaches like socio-environmental assessments (Szewrański & Kazak, Reference Szewrański and Kazak2020) and ecosystem services valuation (Costanza, Reference Costanza2020). This literature offers critical insights into each methodology, their applicability across different contexts, and their effectiveness for assessing environmental impacts (Supplementary Information 1).

Some studies compare specific tools and their strengths and weaknesses (Allesch & Brunner, Reference Allesch and Brunner2014; Andersson et al., Reference Andersson, Brynolf, Landquist and Svensson2016; EuropeanCommission, 2010; Finnveden & Moberg, Reference Finnveden and Moberg2005; Hauschild & Olsen, Reference Hauschild and Olsen2018; Lehtinen et al., Reference Lehtinen, Rouhiainen and Azapagic2011; Windsor et al., Reference Windsor, Cinelli and Coles2018). These studies critically evaluate the risks and uncertainties associated with specific tools, and some advise on the most appropriate contexts to apply them. For example, Grout et al. (Grout et al., Reference Grout, Hales, French and Baker2018) compared eight different tools for assessing the environmental impacts of agricultural systems. They selected Health Impact Assessment (HIA) for New Zealand's dairy sector as it was the only tool that covered environmental, health, social, and economic aspects, and that was designed to assess the impacts of a proposed policy. The problem with this approach is that it makes use of the selected tool relatively inflexible – in this case HIA is not appropriate for analyzing alternative scenarios – meaning new tools have to be selected.

Efforts have also been made to integrate sustainability tools together in various ways. These include conducting post-hoc analyses of results from different tools, combining results and modifying the underlying methodology, or incorporating additional indicators to enhance an analysis. Computational predictive tools and models are becoming increasingly important in tool integration. For example, the MIT Integrated Global System Modelling (IGSM) Framework (Reilly et al., Reference Reilly, Paltsev, Strzepek, Selin, Cai, Nam, Monier, Dutkiewicz, Scott, Webster and Sokolov2013), the Integrated Model to Assess the Global Environment (IMAGE) (Teillard et al., Reference Teillard, de Souza, Thoma, Gerber and Finn2016), GCAM v5.1 Model (Calvin et al., Reference Calvin, Patel, Clarke, Asrar, Bond-Lamberty, Cui, Vittorio, Dorheim, Edmonds, Hartin, Hejazi, Horowitz, Iyer, Kyle, Kim, Link, McJeon, Smith, Snyder and Wise2019), E3ME-FTT-GENIE (Mercure et al., Reference Mercure, Pollitt, Edwards, Holden, Chewpreecha, Salas, Lam, Knobloch and Vinuales2018), and SEAMLESS (van Ittersum et al., Reference van Ittersum, Ewert, Heckelei, Wery, Olsson, Andersen, Bezlepkina, Brouwer, Donatelli, Flichman, Olsson, Rizzoli, van der Wal, Wien and Wolf2008). Each is designed to explore potential development pathways and provide a set of scenarios showing the time-dependent evolution of environmental, human and ecosystem health indicators, and other socio-economic parameters. These models can however suffer from imbalances between the dimensions they consider, and may not always scale well. For example there is still uncertainty as to whether SEAMLESS can effectively be upscaled beyond its test case and whether it can cope with the environmental externalities of farming systems assessed with large bio-economic farm models (Grout et al., Reference Grout, Hales, French and Baker2018; van Ittersum et al., Reference van Ittersum, Ewert, Heckelei, Wery, Olsson, Andersen, Bezlepkina, Brouwer, Donatelli, Flichman, Olsson, Rizzoli, van der Wal, Wien and Wolf2008).

Frameworks have been developed to integrate different tools, with prominent examples including the Life Cycle Sustainability Assessment (LCSA) (UNEP/SETAC, 2011) and Environmental Impact Assessment (EIA) (de Ridder et al., Reference de Ridder, Turnpenny and Nilsson2007; EuropeanCommission, 2014, September, 4). For LCSA, while there are case studies of application, the framework is not widely used as it still requires further scientific work on method integration and on harmonization of procedures in presenting and interpreting the results. EIA was implemented by the European Commission with a corresponding Directive 2011/92/EU.

The selection of appropriate tools can be influenced by a range of factors, including the topic or issue to be investigated, the scope and complexity of the project or activity under consideration, the relevant environmental indicators to be considered, legal and regulatory requirements, the need or otherwise to consider stakeholder and public involvement, availability and quality of data, technical expertise required, and the cost of the tools and specialist databases (Hemming et al., Reference Hemming, Camaclang, Adams, Burgman, Carbeck, Carwardine, Chadès, Chalifour, Converse, Davidson, Garrard, Finn, Fleri, Huard, Mayfield, Madden, Naujokaitis-Lewis, Possingham, Rumpff and Martin2022; Wong-Parodi et al., Reference Wong-Parodi, Mach, Jagannathan and Sjostrom2020). For example, in industry, the use of Environmental, Social, Governance (ESG) assessment tools is gaining traction, primarily driven by regulatory reporting requirements in the financial sector (Baratta et al., Reference Baratta, Cimino, Longo, Solina and Verteramo2023); in food production applications, LCA is widely deployed for product and supply-chain assessment where process input and output data are readily available; and at the global level tools such as the IPCC models are more appropriate for assessing and intervening on climate, biodiversity and disease spread issues (Rocklöv et al., Reference Rocklöv, Semenza, Dasgupta, Robinson, Wahed, Alcayna, Arnés-Sanz, Bailey, Bärnighausen, Bartumeus, Borrell, Bouwer, Bretonnière, Bunker, Chavardes, van Daalen, Encarnação, González-Reviriego, Guo and Lowe2023).

Managers may select tools in a variety of ways; they may re-use a tool they are already familiar with, they may ask colleagues for advice or they may make use of some of the reviews or comparative studies described above. Their approach to tool selection will depend on their experience level, available time, and financial resources but, with so many tools available, this can be a daunting task. Problematically, there are many pitfalls to tool selection – ranging from negligence in not investing the required time or resources to research appropriate tools, to more nuanced and complex factors. These include overlooking context-specific factors or ignoring the dynamic nature of environmental systems, such as failing to adequately consider biodiversity dependencies in industrial and agricultural practices (van der Werf et al., Reference van der Werf, Knudsen and Cederberg2020), or failing to consider social dimensions in assessments (Pollok et al., Reference Pollok, Spierling, Endres and Grote2021). Certain well-established tools, such as LCA and ecological footprinting (Mancini et al., Reference Mancini, Galli, Niccolucci, Lin, Bastianoni, Wackernagel and Marchettini2016), may become the default choice for a sector, even when more suitable tools may exist. For example, LCA, while excellent for assessing impact of a production system on the environment, is typically unsuited for the reverse, in considering impacts of the environment on the production system (Hauschild & Olsen, Reference Hauschild and Olsen2018). As our understanding of the interconnectedness of complex systems grows, the deficiencies of these tools become more apparent. Furthermore, in many instances there is a preference for quantitative data, but this may overlook important qualitative socio-economic nuances, stakeholder preferences, and other less tangible considerations. Perhaps the biggest pitfall in tool selection relates to lack of knowledge and lack of capability in a given organization. Organizations may only be aware of the tools that are either prescribed by regulators or are a standard in their industry – there is no robust classification of the tools or a comprehensive overview of what types of issues they can assess – the gap that we address with our research program.

2.2 The need for a new approach to tool selection

Various frameworks and guidelines do exist to assist practitioners in selecting suitable tools in some contexts. These include the ISO 14001 standard for environmental management systems, which offers a broad framework that, while not specifying particular tools, focuses on incorporating environmental considerations into organizational decision-making. Sustainability Assessment Frameworks like the Comprehensive Assessment System for Built Environment Efficiency (CASBEE) (bin Hishammuddin et al., Reference bin Hishammuddin, Teck, Chau and Ho2019), provide a suite of tools for assessing sustainability and making informed decisions based on specific criteria. LCA, although a tool itself, is also a framework offering a structured approach for assessing environmental impacts throughout a product's lifecycle that can incorporate other tools to provide a more holistic perspective. Guidelines from environmental agencies and international organizations, such as the United Nations Environment Programme (UNEP) and the European Environment Agency (EEA), provide additional resources and recommendations for tool selection tailored to specific needs.

However, despite these frameworks and guidelines, and the diverse research on tools, their strengths, weaknesses, and integrations, there is not yet an effective tool selection process or database with global applicability across a wide range of sectors. A large number of studies use sustainability tools (see Supplementary Material 1), but they lack comprehensive explanation regarding the rationale behind tool selection, weakening confidence that these are the most appropriate tools to address the requirements of the intended decision making process (Viola & Marinelli, Reference Viola and Marinelli2016). Where studies do provide rationale on selection, it is typically limited to the specificities of that study, and so lacks the flexibility to adapt to different input or output data, which could lead to decision making errors if implemented elsewhere. Furthermore, current studies frequently do not include an approach to effectively manage practitioner engagement during the development and subsequent roll-out of the tool, which is imperative for tools designed for use by practitioners (Ness et al., Reference Ness, Urbel-Piirsalu, Anderberg and Olsson2007). A robust rationale, flexible application, and method for stakeholder engagement are critical requirements for a tool selection process to have broad applicability in making difficult decisions (Alrøe et al., Reference Alrøe, Moller, Læssøe and Noe2016; Cucurachi et al., Reference Cucurachi, Scherer, Guinée and Tukker2019; Onat et al., Reference Onat, Kucukvar, Halog and Cloutier2017).

The consequences of using inadequate environmental assessment can be severe, ranging from environmental degradation and loss of biodiversity to social unrest and economic losses, underscoring the need for thorough, context-sensitive environmental assessments (Morgan, Reference Morgan2012). Where large infrastructure projects, like dams, roads, or mining operations, have been planned in ecologically sensitive areas, the use of EIAs is commonplace, but they may fail to fully account for biodiversity loss, habitat fragmentation, increased poaching or illegal logging, among other problems. For example, the controversial development of the Belo Monte dam in Brazil, where the construction and 60% river flow reduction caused by the dam operations destroyed the livelihoods of 20,000 people – and a more robust quantification of potential negative impacts following a correspondingly robust tool selection process may have helped to sway political stakeholders to implement further mitigation measures (Kraljevic et al., Reference Kraljevic, Meng and Schelle2013). Similarly, agricultural development projects that rely on simplified environmental assessments may overlook the cumulative impacts on watersheds and water resources, leading to unsustainable water use, degradation of water quality, and conflicts among water users. This was seen in the Aral Sea disaster, where implementation of irrigation schemes using tools that did not adequately consider the downstream effects on water availability for other users or ecosystems (Micklin, Reference Micklin2007).

Given the current lack of an effective tool selection process we aimed to develop a solution. Since it is at the heart of human needs and draws from nature, we used the global food production system as a sustainability case study, working with the UK Centre for Environment, Fisheries and Aquaculture Science (Cefas). Cefas are working with governments, academia, and NGOs to maximize the social, economic and environmental sustainability of food production, collectively under the banner of One Health.

Cefas are spearheading their efforts via their ‘One Food’ project (Cefas, 2024, April 17). This aims to develop a ground-breaking system that places hazard profiling and management at the heart of environmentally, economically, and socially sustainable food system design. The project will develop a systems-based approach to comprehensively map terrestrial and aquatic food sectors and scenarios and their potential to interrupt safe and sustainable supply chains. A comprehensive mitigation plan will be developed by analysis of supply chain options for hazard control. The project aims to visualize and realize the benefits of this approach via calculation of the impact on key indicators associated with the Global Burden of Animal Disease (GBAD), Global Burden of Crop Loss (GBCL) (Rushton et al., Reference Rushton, Bruce, Bellet, Torgerson, Shaw, Marsh, Pigott, Stone, Pinto, Mesenhowski and Wood2018), biodiversity, and climate efficiency.

A core component of the One Food project is the development of the Risk Tool (Cefas, 2023, September 7). The tool aims to generate an understanding of the multiplicity of hazards across food sectors, allowing users to identify priorities for mitigation, explore trade-offs between different hazards and food sectors, and understand the possible outcomes of mitigation interventions. The tool requires the selection of a robust environmental impact assessment method to provide it with information on the environmental implications of food production. The environmental challenges associated with food systems will vary according to country-specific geographical and socio-economic circumstances, meaning wide variations between applications. Consequently, the selected method needs to be adaptable to a multitude of circumstances and sectors. It, much like broader initiatives, thus requires an effective and widely applicable tool selection methodology.

3.1 Scoping

The first step of the methodology is to establish specific objectives and a shared understanding of what is in and out of scope and thus narrow down potential tools. This ‘scoping’ phase is critically important and is the first point of failure in a tool selection process. If the practitioner fails to conceive the system boundaries correctly and the relevant inputs, outputs, risks and hazards, the tool selection may miss key dimensions – leading to failures, as described in 2.2 (Kraljevic et al., Reference Kraljevic, Meng and Schelle2013; Micklin, Reference Micklin2007). Further scope refinement takes place by capturing key stakeholder requirements, corresponding to different decision scenarios, which are amalgamated into a single set of requirements. In the case of the One Food project, this included specific environmental issues and risk categories, and national priorities for reduction of food waste/loss based on the economic benefits and consequences to the environment and identifying least harmful food production methods as targets for new policies. Typically, this scoping process is iterative between the research and stakeholder team members, leading to a detailed outline of the overall study program.

3.2 Creating a catalog of tools

A landscape of potential tools is then created through a rapid, but comprehensive, literature scan conducted in a systematic and transparent way. This can include using academic search engines, such as Google Scholar and Web of Science, via appropriate search terms. Many tools have also been developed outside of academia in multilateral organizations and are deployed more often by industry and policymakers. Thus, institutional and industrial reports and databases can complement the scientific literature (i.e. the gray literature) – examples from our case study include the FAO, MSC (Marine Stewardship Council), Seafish and Mowi (Supplementary Data 1). The broad scope of the One Food project resulted in an extremely large number of hits, and therefore to decide on our search terms, we used pilot searches to give us a broad range of articles from which to launch our literature review. In addition to desk-based literature review, we conducted qualitative research through discussion with our panel of experts to gain additional data outside of the main searches. For the literature review, we applied the principle of diminishing returns, which provided confidence that we identified the majority of reported tools from different sets of literature, and documented the progress with a discovery curve analogous to species recovery curves used for assessing survey completeness (Colwell & Coddington, Reference Colwell and Coddington1994). The output of this activity undertaken as part of this study is provided as supplementary data to this article (Supplementary Data 1), and serves as a comprehensive database for future tool selection activities, subject to periodic review and update.

Next, the identified tools are grouped into categories and organized in a structured way; a ‘taxonomy’. The literature presents various categorizations of environmental and broader sustainability assessment methods, for example, based on indicators or indices suites for specific sectors, product-related groupings, procedural vs analytical approaches, or whether it is a framework with integrated tools (de Ridder et al., Reference de Ridder, Turnpenny and Nilsson2007; Finnveden & Moberg, Reference Finnveden and Moberg2005; Loiseau et al., Reference Loiseau, Junqua, Roux and Bellon-Maurel2012). By combining these approaches, we used the underlying characteristics and applications of the tools and clustered them into groups for the taxonomy. The taxonomy is a tool itself that enables the industrial sector to identify the input metrics it has available (Figure 2), and from this identify the tools that could be integrated to build a more complete analysis approach with greater relevance to the sector (Figure 3). In policymaking by government or regulatory bodies, the taxonomy could provide a method for more efficient use of economic resources and to improve policy complementarity. By using the taxonomy to identify desired output metrics (Figure 2), this can allow the practitioner to select the most efficient possible range of tools that can deliver these outputs, helping minimize investments of time and economic resources.

Figure 2. Taxonomy of tools – Inputs and Outputs. The figure shows the taxonomy of tools containing key inputs and outputs at the fourth tier. This would be used by the practitioner to select relevant tool types based upon available input variables and desired output metrics.

Figure 3. Taxonomy of tools – One Food Tool Examples. The figure shows the taxonomy of tools in which the fourth tier contains the specific tools applicable to the One Food case study. In the One Food case study, the practitioner would use this to identify specific tools having selected the appropriate tool types using Figure 2.

The taxonomy consists of four layers, or ‘tiers’. Tier 1 subdivides the tools into ‘scope models’, focused on an assessment over a specific spatial, economic, or population scale, and ‘flow models’, evaluating input and output flows within pre-defined system boundaries, where any spatial, population, or economic focus is secondary. Tier 2 encompasses six subcategories. Spatial models focus on assessing a system in space; they estimate the impact of activities and interventions within the defined spatial boundaries and provide a zonation structure for management. Economic models focus on assessing the yield or weighing up the costs and benefits of a certain production system or approach. Population models focus on a given population of a species and are used to evaluate risks from a set of activities in order to inform population management. Hazard assessments focus on hazards, which are sources or situations with potential to cause harm to the environment, ecosystems and humans; these tools look at individual or multiple hazards associated with a defined production system or activity. Environmental systems approaches are categorized as ‘a system that is based on the natural environment and includes biotic and abiotic components which interact’ (Park & Allaby, Reference Park and Allaby2013); they incorporate life cycle analyses, which typically quantify the environmental impact of a system/product/service using either a single or multiple indicators, and frameworks integrating additional perspectives, such as economic, and social. Finally, the category of Multi-criteria assessments aims to integrate an interdisciplinary suite of indicators that describe specific aspects of environmental assessment, which are then assessed across a set of criteria to support a specific decision-making process. Further sub-division at Tier 3 groups tools by single/multi-hazard or single/multi-indicator.

A clear definition of the Tier 1 and 2 categories in the taxonomy is fundamental, as these drive the number and type of tools that end up in the final selection by the stakeholder. During the definition process of the taxonomy architecture for the One Food project example, we maintained a focus on a sector-independent character when selecting and defining the levels, as the taxonomy needs to be applicable to a wide range of topics and challenges relevant to the project stakeholders. As the search for the tool landscape was not specifically focused on the food system, we expect the taxonomy to be universally applicable to any type of sector. If the categories are too broad there is a risk that insufficient tools are selected to assess all relevant areas of impact, if categories are too granular this can result in too many overlapping tools, and if a category is missing then an entire tool type or area of impact could be excluded from the final selection. Figure 2 illustrates the taxonomy developed for the One Food project, its substructure based on the tiers and at the lowest level of detail it summarizes the different inputs required by the tools in each category and the main types of the output information and data resulting from the assessments with these tools. Figure 3 shows the taxonomy with its architecture and the different examples of tools within each category; the full list of tools is included in Supplementary Data 1.

3.3 Tool selection

A structured selection process is then used to select a suite of tools for the end user. Any ‘no-go’ or ‘must-have’ stakeholder requirements (in our example applicability to food production or a sufficient maturity level for practitioner deployment) can be used for an initial pre-selection and followed by a more detailed evaluation. Assessing the tools is a substantial part of the process and comparing the various tools within each category to make the final selection involves many trade-offs as all have different strengths and constraints. This could be a very difficult part of the process for users because it requires a reasonably good knowledge of the tools, so a guiding mechanism is needed to support the decisions. These trade-offs will always be project and context specific, but the creation of a set of appraisal criteria provides the mechanism for structuring and prioritizing tools to reach a solution. A facilitated workshop that brings all team members together is an efficient technique for developing appraisal criteria. Through workshop discussions with the stakeholders, we identified ten criteria and reflected their importance to the One Food Risk Platform using weightings as shown in Table 1.

Table 1. Criteria and their weighting as defined for the One Food project example

Note that for application to other projects the level of weighting applied to each criteria may differ.

3.4 Validation

The final step in the methodology is validation. This requires close collaboration between both researchers and stakeholders to stress-test the performance of the tools, to assess if there are gaps in what and how results are presented and whether they are applicable to the stakeholder's aims. In our case study, the selected tools were validated by (1) assessing the ecosystems types (marine, freshwater, and terrestrial) covered; and (2) coverage of various indicator categories, for example ecosystem functions and services, environmental impact categories, spatial and temporal metrics, and biosecurity. We identified 188 tools, which were reduced to 53 tools through taxonomy and screening based on the criteria in Table 1 (Supplementary Data 1), and further screening allowed us to identify five tools most applicable for the specific intended food systems evaluation by the One Food Risk Platform. These were Life Cycle Analysis (LCA) with ReCiPe 2016 and BioScope, Life Cycle Sustainability Assessment (LCSA), Ecosystem Approach to Fisheries Implementation Monitoring Tools (EAF-IMT), Rapid Agricultural Supply Chain Risk Assessment (RapAgRisk), and Soil and Water Assessment Tool (SWAT). Underlying information can be found in the comprehensive database in Supplementary Data 1. The selection above should be treated as an example only, and any given use case will have its own requirements for the best applicable tools. We emphasize that the ‘gap testing’ component of validation is critically important and should be iterated several times during the validation step. There may be instances where a small suite of relatively generalist tools cover the vast majority of categories effectively, but that leave open a small and specific but important impact category, which should be covered by an appropriate ‘add on’ tool.