This chapter deals with public health and pandemic preparedness. It recognises the five stages of a new pandemic (detection, assessment, treatment, escalation and recovery). The chapter also deals with the issue of laboratory preparedness and the need to maintain a critical mass of laboratory and skilled staff expertise at all times in order to be able to respond rapidly and effectively to a new emerging pandemic.

Our book was written during the COVID pandemic. As a result, it was natural to include a chapter on this topic. In line with the overall theme of our book, we highlight aspects close to the understanding and communication of risk. Topics included in more detail are the inherent danger of exponential growth and the need for adhering to the precautionary principle when faced with a new, possibly catastrophic and hence not yet widely understood, type of risk. The precautionary principle enables decision-makers to adopt measures when scientific evidence about an environmental or human health hazard is uncertain and the stakes are high. A question we address to some extent is whether this pandemic happened totally unexpectedly; was it a so-called Black Swan? We present evidence that it most certainly was not. We give examples of early warnings from scientific publications, highly visible presentations in the public domain as well as regulatory measures in force to absorb the consequences of a possible pandemic. In discussions around risk, numbers, especially large ones, and also units of measurement play an important role; we offer some guidance here.

In this essay, we review the Technology and Measurement around the Globe chapters with an eye toward integration and synthesis. We primarily focus on implications for testing, and then make connections to the broader world of nontest assessment. We identify themes of privacy, fairness, workplace applications, and emerging technologies, and offer a research agenda for future investigations that seek to understand culture, technology, and measurement.

An overview of testing and measurement in North America is provided, covering topics related to privacy laws and regulations, online proctoring, artificial intelligence, accommodations, accessibility, and the “opt out of testing” movement that are currently defining measurement in North America. This is not to say that these challenges are unique to North America; in fact, the challenges related to these topics are being faced all over the world in varying degrees and the same opportunities exist, but these topics are of particular importance when it comes to measurement and assessment in North America. Building on these observations, a discussion of how advances in technology and computing power provide an opportunity to challenge the status quo related to assessment; these advancements will allow assessment of skills in more authentic ways that will provide better insight into someone’s knowledge, skills, and abilities. The question we should be asking and attempting to answer is “How can assessment developers leverage the power of the cloud and technology to measure skills more accurately and create higher fidelity in the assessment process?”

Chapter 10 opens by asking readers to choose two kinds of people they might encounter in informal learning settings and to identify questions those people might have about their general topic and about their specific activity. Returning to the fact that a successful conversation is cooperative, this chapter emphasizes asking questions and listening. Asking questions of an audience gives the expert substantive information to listen to. Sets of questions give people choice, and the sets can include questions about explanations of the phenomena being shown. A "juicy question," for example, is one that nonexperts can address by using the materials/examples at hand – in effect, encouraging scientific reasoning. Giving people time to answer questions and then listening carefully as they do so shows respect, as does asking new questions that reflect people’s earlier responses. Readers are cautioned to avoid testing their audience or to feel that they themselves are being tested. The Worked Example finds juicy questions in a map-based demonstration of variation in regional dialects.

We begin by demonstrating the importance of the ideas of Karl Popper to corpus linguistics. On that basis we begin an exploration of his ideas in order to understand both his account of the scientific method and how this method may illuminate aspects of the corpus-based approach to the study of language.

The bulk of medical care system management in the United States does not ground decisions in use of available, high quality evidence. The previous chapters in this book demonstrate both that there sometimes is a rich scientific foundation in health care organization, delivery, and financing that could, if applied, lead to better outcomes, and that sometimes there is little or no evidence on effectiveness of interventions. However, we also observed that the bulk of health care management in the United States does not ground decisions in evidence, using it if available and taking uncertainty into account if not – instead “magical thinking” is often used to make choices.1 Ironically, management holds evidence in high esteem for decision-making by clinicians. In this chapter, we explore why management holds itself to a lower standard regarding its organizational, staffing, and planning choices, seeing experience, intuition, and opinions as good-enough evidence for decisions. We explore what needs to happen for this to change.

Testing vocabulary is similar to testing in other areas of language knowledge and use. The same criteria of reliability, validity, practicality, and washback need to be considered when designing and evaluating vocabulary tests. In some ways testing vocabulary is easier than testing grammatical knowledge or control of discourse because the units to test are more obviously separate. It is not too difficult to identify what a word type is. However, there are problems and issues and we look at these in this chapter. This chapter has two major divisions. The first major division looks at the purposes of vocabulary tests, covering diagnostic, placement, achievement, and proficiency tests. The second major division looks at different test formats, answering questions like: Should choices be given? Should words be tested in context? How can I measure words that learners don’t know well? This section covers a wide range of vocabulary test formats, along with comments on their design and use.

Managerial economics provides a toolbox for solving problems that managers frequently face. It addresses issues relating to any aspect of decision making that ultimately affects the profit of a firm. Although the general methodology of managerial economics has not changed over the decades, there have been rapid and significant changes in the business environment in the last ten years or so, and three new themes have become increasingly important: digitization; behavioural aspects; and globalization. The first of these developments involves aspects of big data and advanced data analytics, the human-machine interface and the interconnectedness of electronic devices. The second relates to psychological aspects of decision making that cause both consumers and managers to engage in behaviour normally referred to as ‘irrational’. The third development is that improvements in technology relating to digitization have made the business world more interconnected. The text makes heavy use of recent case studies involving these three themes, for example on tech firms, Covid-19 and climate change, so students can see how the tools of managerial economics can be applied in real-life situations.

Chapter 8 looks at an Article 36 review of autonomous weapons systems. Having first addressed the progress of debates about potential future regulation of such systems at the international level, it delineates the notions of autonomy, artificial intelligence and learning systems. The chapter shows that military capabilities underpinned by machine learning and deep learning technologies require a new understanding of how provisions of targeting law can be meaningfully translated to the context of autonomous systems and examined in the framework of Article 36. It also demonstrates that determining the novel character of learning autonomous capabilities to ensure timely provision of legal advice, gaining assurance of complex adaptive systems’ performance accuracy and reliability, or identifying a set of requisite control measures, are all issues that ‘traditional’ weapons reviews do not have to grapple with at all, or at least not to the same extent. [143 words]

Chapter 9 focuses on an Article 36 review of (autonomous) cyber capabilities. As in the case of autonomous weapons systems, the most burning questions for a weapons review in the cyber context remain: How to meaningfully incorporate advice on the law of targeting as part of the weapons review? How to determine when a given capability is ‘new’ for review purposes and when to initiate a review? How and when can testing and evaluation processes meaningfully inform the review outcome? Furthermore, some challenges distinct to cyber capabilities also exist. Most importantly, cyber capabilities test the underlying assumptions of the law of armed conflict. The effects of their use may be more deleterious than the consequences produced by traditional weaponry, and yet they may fall outside the legal review requirement because the effects produced do not constitute an ‘attack’ in its conventional interpretation. [142 words]

This chapter gives an overview of the main findings regarding the multimedia principle in testing, which states that testing material that combines text and adequate pictures is more comprehensible and more efficient to process for test takers than text alone. Even though learning and testing with multimedia have much in common, substantial differences exist that are discussed in this chapter. Overall, the findings indicate that adding pictures to verbal test material is mainly associated with facilitative effects. However, the findings differ with respect to different outcome measures and depending on the investigated picture type (i.e., decorative, representational, organizational, and informational). Furthermore, effects also appear to be moderated by item and test-taker characteristics that need to be subjected to future research.