2.1 Example: catastrophic climate change

One example of a challenging policy problem that meets the criteria identified above for an uncertain future is catastrophic climate change. While the occurrence of climate change is widely accepted, the nature and probability of extreme outcomes is much less understood. Forecasted effects occur decades if not centuries into the future and may be irreversible. The scale is global, and worst case “fat tail” scenarios depend on a network of uncertain linkages between human and natural systems. In its 2018 special report, Global Warming of 1.5°C, the Intergovernmental Panel on Climate Change (IPCC) (2018) estimates global temperature increases ranging from 1.46 to 3.66°C by the end of the 21st century. The IPCC warns, however, that temperature increases at or above 2°C increase the likelihood of triggering “tipping points” – critical thresholds beyond which a system can experience significant, often irreversible changes including: “habitat losses for organisms; increases in risks related to flooding from sea level risk; increases in extreme drought; and reductions in rainfall and water availability” (IPCC, 2018, 261).

2.2 Example: systemic financial crisis

As another example, the global financial crisis of 2007 and subsequent “Great Recession” caused monetary authorities to take emergency actions that, in scope and scale, were without precedent. Despite considerable analysis, legislation, and regulation, whether the financial system today is any less vulnerable to a major dislocation is unclear. Much of the ongoing concern centers on institutions deemed “too-big-to-fail”; yet we cannot be certain that ensuring the health of individual institutions will adequately mitigate systemic risk, given the possibility that seemingly small causes may produce a cascade of larger consequences.

The systemic risk embedded in financial markets may be due, in part, to the size and contours of the governing policy institutions, the financial, monetary, and fiscal authorities at the national and supranational level. The struggles to contain the financial crisis in Greece offer one example. But systemic risk also follows from the dense interconnectedness of the modern financial system. In 2015, the U.S. Commodity Futures Trading Commission (CFTC) accused an individual trader in London of causing the trillion-dollar “flash crash” of May 6, 2010 (Whipp & Scannell, Reference Whipp and Scannell2016). Regardless of the merit of the accusation, it should be clear that prosecuting an individual is not an adequate policy response to what is a sign of systemic instability. High-speed algorithmic trading has come to dominate financial exchanges, raising concerns about their vulnerability to sudden disruption (Budish et al., Reference Budish, Cramton and Shim2015). Today’s markets operate so fast that a financial collapse may be catastrophic in its amplitude and global in its scope, proliferating before anyone recognizes what has happened. Authorities continue to struggle with an understanding of systemic risk, options for moderating it, and methods of analyzing it.

2.3 Example: cyberattack on the power grid

Experts have long considered the U.S. power grid a logical target for a major cyberattack. Increasingly sophisticated and interconnected power grids allow electric utilities to use technological advances in networking to operate with greater efficiency and reliability; however, this interconnectedness may also amplify the potential damage caused by a coordinated cyberattack, an electromagnetic pulse weapon, or a large solar storm. Worst-case scenarios include nationwide blackouts that persist anywhere from a matter of days in some places to several weeks in others. Power outages of this magnitude could cause billions of dollars in economic damage and present significant public safety concerns.Footnote ¹

U.S. agencies, including the Department of Homeland Security (DHS) and the Department of Energy (DOE), recognize that the interconnectedness of the U.S. energy grid makes it vulnerable to “cascading failure.” Admiral Michael Rogers testified in 2014 that several countries likely have the capability of shutting down the U.S. power grid (House Select Intelligence Committee, 2014). However, significant uncertainty exists regarding how best to manage this risk. For example, utilities must balance the low-probability but possibly catastrophic risk of a cyberattack against the near certainty of routine weather events affecting their customers. Experts note that utilities operate on thin profit margins, and decisions to increase spending on cybersecurity to prevent low-probability harms require spending less on managing highly-probable, weather-related risks (e.g., investing in burying power lines or raising them above tree lines) (Knake, Reference Knake2017).

For regulated utilities, capital expenditures must be justified to a regulatory commission while the appropriate level of investment required to sufficiently safeguard the grid against cyber attacks is unknown. In its 2015 report, DOE estimated that an investment of $15.2 billion might be sufficient to insulate the U.S. energy grid against cyber attacks (Knake, Reference Knake2017). Yet, its Quadrennial Energy Review released just one year later found that the level of investment necessary actually ranged from $350 billion to $500 billion (U.S. Department of Energy, 2017). These wide-ranging estimates exemplify the deep uncertainty surrounding the appropriate level of investment to safeguard the grid.

Further, as Roberts (Reference Roberts2019) details in his article, a cyberattack might be used to facilitate a physical attack on a target, either to increase the attack’s probability of success, reduce its cost, or increase the magnitude of its effects. Future assessments considering the appropriate level of investment required to secure the grid will require analytical tools that consider the joint effects of “combined attacks.” Additionally, such assessments will also have to account for the synergistic effects of risk mitigation to address multiple threats. For example, methods that might be employed to harden the electric grid could also lessen the consequences of cyber, electromagnetic pulse, solar eruptions, etc.

3.1 Traditional analytical tools may be inadequate

Traditional decision analysis examines a set of decision alternatives and evaluates them by considering the probabilities and utilities of different outcomes associated with each. With this information, choices can be optimized subject to feasibility constraints (e.g., see Raiffa, Reference Raiffa1968; Abbas & Howard, Reference Abbas and Howard2015). However, when reliable information on probabilities is lacking, the analysis becomes much more complicated.

To address these challenges, Cox (Reference Cox2019) reviews the insights of Charles Lindblom (see Lindblom, Reference Lindblom1959), who in 1959 raised concerns about the practicality of “rational-comprehensive decision-making,” such as decision analysis and statistical decision theory, for understanding and managing complex situations. Cox observes that:

Although scholars maintain that BCA “is an indispensable step in rational decision-making in this as in other areas of government regulation” (Posner, Reference Posner2004, 139), they simultaneously recognize that the characteristics of these events make them “intractable to conventional analytic methods” (Posner, Reference Posner2004, 8). It is often difficult to monetize the costs and benefits of both the effects of uncertain risks and of the policies that attempt to address them.

BCA uses prices and pseudo-prices, such as “shadow” and hedonic prices, to capture the value of marginal changes in the supply of goods and services. But scale matters. Prices at the margin cannot be extrapolated without limit: selling one pint of blood for $50 does not mean you would part with two gallons for $800. Further, these methods break down when addressing large irreversible changes such as deliberate species extinction (e.g., see Pugh, Reference Pugh2016; Mannix, Reference Mannix2018); with policies that only make sense on a global scale where uncertainty exists around what authority is making policy and who has standing to influence it; where network effects make systemic failures more likely, such as with financial markets; with long time horizons where determining the appropriate discount rate becomes more complicated; and when probabilities are unknown or unknowable.

While the underlying BCA principles may be sound, data required to complete an analysis are often inaccessible. We can observe prices that are relevant for valuing changes in mortality risk, but there is no market in which to observe a price for the whole planet, as the prices for small patches of real estate are not really informative on this point. Income effects that could be a minor complication in ordinary BCA may come to dominate the results when evaluating extreme cases. Transfer effects can also get very complicated. For example, to what extent should climate policy prompt large transfers among nations, and how will these concerns affect efforts to negotiate a global policy?

3.2 Inaccurate or uninformed risk perceptions complicate decisions

Kahneman and Tversky (Reference Kahneman and Tversky1979) demonstrate that “highly unlikely events are [often] either ignored or overweighted” (p. 283). It is also well established that the public’s perceptions of which risks are most concerning with respect to their likelihoods and impacts differ substantially from risk experts’ estimates (e.g., see Kuran & Sunstein, Reference Kuran and Sunstein1999; Chilton et al., Reference Chilton, Jones-Lee, Kiraly, Metcalf and Pang2006; Viscusi, Reference W. Kip.2009). Additionally, both groups shift their opinions regarding which risks pose the greatest threats to society over relatively short timeframes. For example, Scouras’s article in this issue illustrates that experts hold widely divergent views about the likelihood of nuclear war. Table 1, which summarizes the results of the World Economic Forum’s Global Risks Report survey in 2008 and 2018, illustrates that even experts’ perceptions can change significantly over periods as short as 10 years (World Economic Forum, 2018).

Table 1 World Economic Forum: top 5 global risks in terms of impact.

Source: World Economic Forum, 2018.

Although some experts argue for taking extra precaution when dealing with potentially catastrophic or irreversible risks, the truly difficult issue resides in deciding on the appropriate level and form of policy intervention. For example, Sunstein (Reference Sunstein2006) points out that notions of irreversibility and catastrophe are “exceedingly ambiguous, and [that] it is by no means clear how regulators should understand them.”

As Cox (Reference Cox2019) explains, large-scale real-world decision problems challenge traditional rational-comprehensive decision analysis models because the “reward function” and choice set are unknown or costly to develop. Behavioral research suggests that people – the lay public, political figures, and experts – tend not only to overestimate small and uncertain risks but also are irrationally averse to risk ambiguity. Viscusi et al.’s research in this series reveals that the public over-reacts to very small water-related risks, such as those posed by the herbicide atrazine, prescription drug residues, and BPA. Ambiguity aversion may lead to precautionary choices when, as Viscusi et al. (Reference Viscusi2019, 9) highlight, “the optimal strategy often involves holding off from expensive or irreversible actions and instead learning about the risk based on experience, and considering adaptive behavior that involves switching to other policies if the outcomes with the uncertain choice are sufficiently unfavorable. ”

3.3 Risks exist in analyzing from a single perspective (narrow framing)

Scholars studying risk regulation have long understood that problems result when regulators miss the proverbial “forest for the trees.” For example, 25 years ago, now Supreme Court Justice Stephen Breyer (Reference Breyer1993, 11) observed the propensity for regulators to have “tunnel vision,” referring to the inclination of individuals and agencies to pursue their narrow regulatory objectives while ignoring how their efforts interact within the larger set of societal benefits and costs. Breyer further documented that agencies can lack systematic methods for prioritizing risks and posited that regulators and policymakers tend to be overly attentive to public opinions in deciding how risks should be prioritized (Breyer, Reference Breyer1993, 20). Finally, he noted that regulations crafted with the intention of addressing risks are often responsible for creating additional risks for which the rule makers have not accounted (Breyer, Reference Breyer1993, 22). Accordingly, some research suggests that improvements in the ability to manage risks could be achieved using frameworks that reward learning, experimentation, and interdisciplinary collaboration (Pawson, Reference Pawson2013, 190).

Tunnel vision can be aggravated in the face of uncertain futures because the analysis may easily miss general equilibrium effects or the influence of rational expectations (Sprenger, Reference Sprenger2015). Expensive efforts to improve the welfare of future generations may be frustrated, for example, if the public compensates by reducing other forms of long-term savings and intergenerational wealth transfer.

5.1 Probabilistic vs. strategic risk

Darby (Reference Darby2004) highlights the substantive differences that exist between a probabilistic evaluation of terrorist risk and a similar assessment of other risks such as safety risk.Footnote ³ Primary among them, estimating terrorist risk does not involve “random events associated with ‘dumb’ components; terrorist risk involves intentional acts by malevolent, thinking terrorists.” As a result, the risk depends on many attributes exogenous to the system, including an attacker’s capabilities, ability to adapt, and perception of both a target’s vulnerability and the consequences of a successful attack. Golany et al. (Reference Golany, Kaplan, Marmur and Rothblum2009) categorize the distinction as the difference between modeling probabilistic uncertainty and strategic uncertainty.

Considering the conceptual framework widely employed by DHS in its risk analyses of terrorism where risk is function of threat (T), vulnerability (V), and consequence (C) (i.e., risk = f(T,V,C)), Darby (Reference Darby2004) notes that successfully estimating risk using this approach requires recognizing the dependence among these variables. While researchers have employed various techniques to improve upon this framework, others maintain that it is ultimately of limited use for informing decision-makers responsible for allocating scarce resources to defend against terrorist attacks. For example, the National Research Council (2010) reviewed DHS’s approach to risk analysis in 2010 and concluded:

Examining other uncertain futures often requires overcoming many of the same challenges associated with analyzing terrorism. These include grappling with the aforementioned complexity of assessing strategic risk as well as other challenges including reconciling substantive variation in experts’ risk perceptions, the existence of risk ambiguity, and data limitations. Altogether, these issues point to the need to modify existing frameworks for assessing risks especially related to attacks against critical infrastructure (Depoy et al., Reference Depoy, Phelan, Sholander, Smith, Varnado and Wyss2005). Additionally, given the limited data available to study such attacks, experts identify “qualitative or semi-quantitative approaches … for making interim decisions while identifying data needs” as a promising approach (O’Brien & Cummins, Reference O’Brien and Cummins2011).

5.2 Risk assessment for combined cyber and physical attacks

Another limitation of traditional approaches to conducting risk assessments of terrorist attacks resides in the fact that threats and consequences of physical and cyber attacks are typically considered in isolation (Depoy et al., Reference Depoy, Phelan, Sholander, Smith, Varnado and Wyss2005). In his paper in this series, Roberts (Reference Roberts2019) proposes a qualitative, relative risk assessment framework, noting that “a more modern way of thinking about security is to think of ‘combined’ attacks that include both a physical and a cyber component.” His study assesses the contextual factors under which terrorists might prefer a combined attack on U.S. sporting venues or the international maritime transportation system.

Roberts (Reference Roberts2019) asserts that terrorists are increasingly using cyber attacks as precursors to physical attacks, particularly when that cyberattack serves to: (i) improve the physical attack’s likelihood of success; (ii) decrease the cost of conducting it; or (iii) increase the consequences of that physical attack. His study employs a qualitative framework using these criteria and informed by expert judgment to establish whether an attacker would likely prefer executing a combined or traditional physical attack on a particular target. Admittedly, several combinations of probability, cost, and consequence exist where it is unclear how to assess a potential attacker’s preferences. For instance, it is challenging to determine to what extent a terrorist would accept a lower consequence, such as fewer casualties, in exchange for a higher probability of success.

Roberts (Reference Roberts2019) concludes that the traditional risk = f(T,V,C) probabilistic framework not only does not account for the risk of combined attacks, but also that data are so scarce for these types of attacks that “quantitative analysis of such gaps is questionable at best.” Overall, two advantages of Roberts’ approach are that it offers decision-makers a framework that is technically feasible to implement and permits risk analysis to proceed in the face of strategic uncertainty without relying on quantitative metrics of uncertain validity, especially where there is little solid data.Footnote ⁵

5.3 Nuclear war as a global catastrophic risk

Scouras (Reference Scouras2019) assesses the risks posed by intelligent adversaries involving the use of nuclear weapons. His study notes that analysis of the likelihood of nuclear war is often based on poorly executed expert elicitations that reveal widely divergent opinions. Furthermore, our knowledge of the consequences of nuclear attacks remains underdeveloped with respect to many important effects, despite over a thousand nuclear tests and billions of dollars expended toward research on nuclear impacts. Like Roberts, Scouras asserts that traditional probabilistic risk assessment is insufficient for informing policy responses, and he offers multiple insights for improving the analysis of uncertain futures. Notably, he cautions against deriving overly simplistic policy prescriptions from computations of expected risk, which may be "inadequate, even dangerous [absent] a more nuanced understanding of the risk of nuclear war." (Scouras, Reference Scouras2019).

His article also highlights several contextual factors that have limited the development of comprehensive evaluations of the consequences of using nuclear weapons. For instance, Scouras notes that, particularly during the Cold War, data collection and analysis focused on “the nuclear damage mechanisms of air blast, cratering, ground shock, and similar phenomena” (Scouras, Reference Scouras2019, 11). Accordingly, such an emphasis limited growth of knowledge regarding other important effects of nuclear weapons, including damage to the functioning of economies and societies, the effects of electromagnetic pulses accompanying nuclear explosions, and the possibility of global-scale famine via nuclear winter.Footnote ⁶ According to Scouras, “there are no fundamental barriers to obtaining a better understanding of these important phenomena” (Scouras, Reference Scouras2019, 31). He argues that federal agencies such as the Defense Threat Reduction Agency should be explicitly directed to address these current gaps in our consequence assessments.

Finally, the continued proliferation of nuclear weapons provides additional risks for analysts to consider, including those posed by terrorists and other nonstate actors. Scouras (Reference Scouras2019) posits that it is feasible (and necessary) to improve assessments of these events and argues for research melding ideas from a variety of perspectives and approaches including case studies, experts, risk assessment, and complex systems theory, as “no single analytic approach has proven to be satisfactory.”