2.1. Definitions and applications of robustness

Robustness preserves a system characteristic in the presence of variation of system components (Carlson & Doyle Reference Carlson and Doyle2002). Designers, researchers and users are often interested in system functionality for both engineered and biological systems (Kitano Reference Kitano2004; Clausing & Frey Reference Clausing and Frey2005). Furthermore, Doyle & Csete (Reference Doyle and Csete2011) assert that a large fraction of design effort and system complexity is attributable to features that enhance system robustness. Clausing & Frey (Reference Clausing and Frey2005) support this assertion, reporting that many patent claims improve robustness rather than new functionality. Robustness is studied across disciplines ranging from functionality under extreme or accidental loads in structural engineering (Baker, Schubert & Faber Reference Baker, Schubert and Faber2008) to preserving future options or opportunities under attempts by others to cut off those options in political science (Padgett & Ansell Reference Padgett and Ansell1993). Robustness clearly plays an important role across disciplines, and especially in engineering and design, as it is a critical property of effective systems.

Research over the last several decades has identified four types of design robustness for engineered systems, each corresponding to a type of uncertainty that can affect a significant system characteristic. Taguchi proposed a design philosophy that reduces variation in performance due to exogenous sources of uncertainty (Type I) (Chen et al. Reference Chen, Allen, Tsui and Mistree1996). Chen et al. (Reference Chen, Allen, Tsui and Mistree1996) developed methods to simultaneously design for Type I uncertainty and endogenous variation such as in design variable values (Type II uncertainty). Choi et al. (Reference Choi, Austin, Allen, McDowell, Mistree and Benson2005) augment Chen et al.’s methodology to account for variability in the system performance model used to evaluate candidate designs (Type III uncertainty). Seepersad et al. (Reference Seepersad, Fernandez, Panchal, Choi, Allen, McDowell and Mistree2004) identify changes in design specifications and variables and the ‘combined effect of analysis tasks’ as sources of design process uncertainty (Type IV), resembling a combination of Type II and Type III uncertainty. In summary, the objective of these four types of robustness is to preserve system performance or functionality under uncertainty from exogenous sources, design variable values, system performance models and system design processes.

Past research treats designers or system decision-makers (collectively referred to as system actors) as a source of uncertainty in technical factors such as design variable values (Chen & Lewis Reference Chen and Lewis1999; Kalsi, Hacker & Lewis Reference Kalsi, Hacker and Lewis2001; Allen et al. Reference Allen, Seepersad, Choi and Mistree2006). This perspective does not address strategic decisions, such as system actor participation in a team or collaborative structure, and is an appropriate approach only for design of a single system where there is little strategic uncertainty.

2.2. System-of-systems collaboration

System actors designing a single system ostensibly share a common goal. While their design decisions affect the utility delivered by each other’s design elements, project managers can theoretically resolve conflicts within project constraints to achieve an effective design and mitigate decision conflict. However, unity of purpose does not similarly characterize all SoS because they are comprised of complete, independently managed and operated systems (Maier Reference Maier1998).

SoS types exhibit different levels of management centralization. Maier (Reference Maier1998) categorizes SoS as directed, collaborative or virtual. OUSD AT&L (2008) adds ‘acknowledged’ SoS to Maier’s list. Only directed SoS are built and managed for a particular collective purpose or set of purposes. Many defense SoS are directed systems. Constituents of an acknowledged SoS share a common purpose managed separately from each system’s goals. However, the constituents retain managerial independence. Collaborative and virtual SoS form voluntarily, and without SoS-specific management, because each constituent system can increase its utility by doing so, either by increasing performance, reducing cost or both. Virtual SoS lack a shared purpose, whereas collaborative SoS retain this feature. The upside benefit of participation depends on mutual collaboration of other actors to overcome initial resource expenditure to enable interoperability.

The combination of upside benefit generated through the SoS and downside risk tied to upfront investments creates a large variation in potential outcomes relative to an independent system. Awareness of these risks, especially as they affect potential collaborative partners, may create uncertainty about the actions of other system actors. These conditions create an environment, particularly for collaborative and virtual SoS, in which system actors desire successful collaboration, but are not certain it will occur and are unlikely to unconditionally commit to collaboration.

Consequently, each system actor is motivated to influence others to adopt a locally favorable policy and to simultaneously pursue a policy that yields the highest utility in response to other actors’ unknown decisions. The risk of coordination failure may be mitigated through contractual agreements which penalize defection and spread risk across the parties or through information exchange which increases confidence that collaboration will be successful. However, contracts are not always viable or effective options, for example, when no formal relationship exists between the actors. Furthermore, contracts generally alter strategy dynamics to reduce the favorability of defection, not increase the favorability of collaboration. Information sharing alone is insufficient to ensure collaboration. For example, the shared information may be negative, and inhibit SoS formation, or it might not be trusted.

The aim of robust SoS design is to reduce the consequences of coordination failures (increasing the favorability of collaboration, conforming to traditional Type I robust design) and to shape the collaboration environment to reduce the probability of coordination failure. Existing design space exploration and optimization methods alone are inadequate to achieve these goals because they do not include the interactive decision models required to assess strategic issues or the effect of design on strategy dynamics.

2.3. Game-theoretical equilibrium selection

Grogan & Valencia-Romero (Reference Grogan and Valencia-Romero2019) developed a system design model which represents the collaborative dynamic as a type of bipolar game commonly referred to as stag hunt. Bipolar games are two-strategy normal-form games with two strict Nash equilibria (Selten Reference Selten1995). Applied to SoS, the strategy space $ {\mathcal{S}}_i=\left\{{\phi}_i,{\psi}_i\right\} $ denotes actor $ i $’s ‘independent’ and ‘collaborative’ strategies respectively. SoS formation requires that successful collaboration is mutually beneficial. This work further assumes that independence is strictly superior to investing in a failed collaboration, completing the conditions required for a stag hunt. Table 1 shows normal-form payoffs $ {V}_i^{s_i,{s}_j} $ for an example stag hunt game using the Bluetooth and Greycloak technology transition labels where numeric payoff values encapsulate actor utilities including other factors like risk aversion. To note, the game payoffs in this section only illustrate equilibrium selection concepts and the bi-level system design model. They are not intended to represent a particular technology transition.

Table 1. Normal-form payoffs for a symmetric two-actor technology transition game modelled after a stag hunt

The equilibria for two actors $ \mathcal{A}=\left\{1,2\right\} $ are $ \phi =\left\langle {\phi}_1,{\phi}_2\right\rangle $, all independent, and $ \psi =\left\langle {\psi}_1,{\psi}_2\right\rangle $, all collaborate. As an analogy for SoS design, each actor has a choice between a safe strategy $ {\phi}_i $ with a relatively low payoff that does not require collaboration (canonically referred to as ‘hare hunting’), and a riskier strategy $ {\psi}_i $ that requires successful collaboration to achieve a higher payoff than hare hunting (canonically referred to as ‘stag hunting’). Stag hunters with defecting partners suffer a loss relative to what could be obtained if they had hunted hare.

In the context of the mobile computing technology transition example, the hare hunting strategy retains the Bluetooth technology and is implemented by a Mk I product design. The stag hunting strategy transitions to the Greycloak technology with superior technical potential and is implemented by a Mk II product design. A pair of interacting system actors successfully collaborate when both adopt the Greycloak technology.

Technology transitions in a commercial environment are also influenced by competitive forces, that is, actors may adopt a new technology to gain or maintain a long-term competitive advantage. Therefore, competition exerts forces on system actors beyond the direct value of short-term gains in utility. Furthermore, the interaction between complementary and substitute goods produces complex dynamics that may accelerate or inhibit transition to a new technology. This work focuses strictly on the complementary relationship between system actors, rather than competitive forces, to reduce assumptions with overlapping impacts and improve the interpretability of the results.

Design and strategy choices are linked in the context of the application case, that is, implementing a Mk II design indicates an actor intends to collaborate in the market with others adopting the Greycloak technology. However, this is not universally true of SoS. For instance, diverse defense systems may adopt a set of common communication protocols to enable combined operations in an SoS but the systems may intermittently or never be used as an SoS. In such a context, the design enables a strategy but it does not determine it.

Rational actors select a strategy in games such as a stag hunt on the basis of two criteria: payoff dominance and risk dominance (Harsanyi & Selten Reference Harsanyi and Selten1988). The payoff-dominant strategy is an element of a Pareto efficient equilibrium, yielding the highest possible payoff for each actor. In the examples used here, the Greycloak strategy $ {\psi}_i $ is always payoff-dominant. The risk-dominant strategy exhibits the lowest risk when considering interactive effects and, as subsequent sections demonstrate, has the greatest basin of attraction, meaning actors tend to select it in repeated games on a multiactor network (Kandori, Mailath & Rob Reference Kandori, Mailath and Rob1993; Grogan & Valencia-Romero Reference Grogan and Valencia-Romero2019).

Risk dominance is analogous to expected value maximization under specific conditions. The normalized deviation loss $ {u}_i $, in Eq. (1), denotes the threshold probability $ {p}_j $ of actor $ j $ selecting strategy $ {\psi}_j $ that equalizes the expected value of $ {\phi}_i $ and $ {\psi}_i $ for actor $ i $. For symmetric games, $ {u}_i={u}_j $, and therefore $ \phi $ (conversely, $ \psi $) maximizes expected value and is risk dominant for $ {u}_i>0.5 $ (conversely, $ {u}_i<0.5 $) if actors play strategic alternatives with equal probability.

(1)$$ {u}_i=\frac{V_i^{\phi_i,{\phi}_j}-{V}_i^{\psi_i,{\phi}_j}}{\left({V}_i^{\phi_i,{\phi}_j}-{V}_i^{\psi_i,{\phi}_j}\right)+\left({V}_i^{\psi_i,{\psi}_j}-{V}_i^{\phi_i,{\psi}_j}\right)} $$

The weighted average log measure (WALM) of risk dominance, $ R $, proposed by Selten (Reference Selten1995), extends the concept of risk dominance to asymmetric and many actor games with linear payoff functions by accounting for interactive effects

(2)$$ {\displaystyle \begin{array}{rcl}R& \equiv & \sum \limits_{i=1}^n{w}_i(A)\ln \frac{u_i}{1-{u}_i}\end{array}} $$

where $ {w}_i(A) $ measures the weight of actor $ i $ based on an influence matrix $ A $, noting that $ {w}_i(A)=\frac{1}{2} $ for all two-actor games. See Selten (Reference Selten1995) for a general derivation of these values. $ R>0 $ indicates that $ \phi $ is risk-dominant and $ \psi $ is risk-dominated, while $ R<0 $ indicates the opposite, namely $ \psi $ risk-dominates $ \phi $. Furthermore, the magnitude of $ R $ indicates the strength of dominance.

Using payoffs from Table 1, Figure 1 illustrates $ {u}_i=\frac{2}{3} $ as the intersection between expected value curves for the $ {\phi}_i $ (blue solid line) and $ {\psi}_i $ (red dashed line) strategies. Following symmetry, the risk dominance criterion evaluates to $ R=2\cdot \frac{1}{2}\ln \left(\frac{2/3}{1/3}\right)=0.69 $, indicating that the Bluetooth equilibrium $ \phi $ is risk-dominant. In general, symmetric games with $ {u}_i>0.5 $ result in $ R>0 $ but asymmetric games depend on the relative magnitude of all $ {u}_i $ values and the influence weights of each actor.

Figure 1. Expected value of actor $ i $’s strategies under uncertainty in actor $ j $’s strategy for the baseline scenario in Table1 and the scenario with the Mk II-B design in Table 3.

Whether payoff or risk dominance should ultimately determine equilibrium selection has been a matter of inquiry for game theorists. Harsanyi & Selten (Reference Harsanyi and Selten1988) defined the concepts of payoff and risk dominance, asserting at the time that payoff dominance is the relevant equilibrium selection criterion for stag hunt games. However, Harsanyi later reversed his position, concluding that risk dominance should be the governing selection criterion (Harsanyi Reference Harsanyi1995). Carlsson & van Damme (Reference Carlsson and van Damme1993) likewise concluded that risk dominance is the only rational strategy for incomplete information bi-stability games, such as stag hunt.

The importance of risk dominance also makes sense in systems design settings because using payoff dominance to guide decisions would simply optimize solutions to maximize utility under successful collaboration, ignoring downside risks. Payoff dominance tells a designer nothing about the stability of collaboration or strategic robustness of the system. Such designs are likely to be over-optimized and fragile. Instead, system actors may use risk dominance, specifically the risk dominance criterion $ R $, to guide payoff-dominant collaborative solutions to a more robust state.

2.4. Strategically informed design

Design can influence a system actor’s payoffs under the possible strategic outcomes, thereby modifying the relative risk and payoff dominance of available strategies. For example, actions to reduce losses under a coordination failure can increase risk dominance of a payoff-efficient strategy. Such a design achieves robustness by directly mitigating the impact of a coordination failure but also by reducing the probability of coordination failure if partners also use risk dominance to inform strategy selection. Therefore, an emanation of Selten’s risk dominance criterion for engineering is that system design can reduce losses under coordination failure and influence strategy dynamics to reduce defection by system actors as a form of environment modification (Whitacre Reference Whitacre2012) that increases utility robustness. Grogan & Valencia-Romero (Reference Grogan and Valencia-Romero2019) used this relationship to develop a bi-level model of SoS design in which system actors each select an upper-level strategy $ {s}_i\hskip1.5pt \in \hskip1.5pt {\mathcal{S}}_i $ and a lower-level design $ {d}_i\hskip1.5pt \in \hskip1.5pt {\mathcal{D}}_i $. The value function $ {V}_i^{s_1,\dots, {s}_n}\left({d}_1,\dots, {d}_n\right) $ maps decisions to payoff values for each system actor.

Consider a new game in which each system actor has two decisions. The first is an operational design decision: a choice between technical implementations in design space $ {\mathcal{D}}_i=\left\{{\alpha}_i,{\beta}_i,{\gamma}_i\right\} $ with assigned labels Mk II-A, Mk II-B and Mk I. The second is, as before, a choice between strategies in strategy space $ {\mathcal{S}}_i=\left\{{\phi}_i,{\psi}_i\right\} $. Table 2 shows the payoffs $ {V}_i^{s_i,{s}_j} $ for each strategy pair-design combination. To simplify the presentation, payoffs are only a function of the design and strategic context, not the partner’s design. For comparison, the prior game in Table 1 adopts the Mk I design $ {\gamma}_i $ under the Bluetooth strategy $ {\phi}_i $ and the Mk II-A $ {\alpha}_i $ design under the Greycloak strategy $ {\psi}_i $.

Table 2. Actor $ i $’s strategy-specific design payoffs for a technology transition design game

In a trade typical to design (Doyle & Csete Reference Doyle and Csete2011), adopting the Mk II-B design $ {\beta}_i $ under the Greycloak strategy sacrifices a small amount of upside utility relative to Mk II-A for a large reduction in downside risk, enhancing robustness to coordination failure. Despite being dominated in both the $ \left\langle {\phi}_i,{\phi}_j\right\rangle $ and $ \left\langle {\psi}_i,{\psi}_j\right\rangle $ strategic contexts, the Mk II-B design reduces the normalized deviation loss to $ {u}_i=\frac{1}{4} $, as shown in the normal form game in Table 3 and the green dash-dot line in Figure 1. If the Mk II-B design is selected by both actors, the risk dominance criterion decreases to $ R=2\cdot \frac{1}{2}\ln \left(\frac{1/4}{3/4}\right)=-1.10 $, indicating that the payoff-dominant Greycloak equilibrium $ \psi $ is also risk-dominant.

Table 3. Normal-form payoffs for a symmetric technology transition design game instance

An asymmetric scenario in Table 4 combines the two cases. If one actor’s strategy-design pairs are ($ {\phi}_i $, $ {\gamma}_i $) and ($ {\psi}_i $, $ {\alpha}_i $) and the others are ($ {\phi}_j $, $ {\gamma}_j $) and ($ {\psi}_j $, $ {\beta}_j $), normalized deviation loss alone suggests a different ‘best’ strategy for each actor. However, neither $ \left\langle {\psi}_i,{\phi}_j\right\rangle $ nor $ \left\langle {\phi}_i,{\psi}_j\right\rangle $ are equilibria. Risk dominance resolves the decision dilemma by accounting for the influence each actor has on the other’s utility and the relative preference of each strategy for both actors. In this example $ R=\frac{1}{2}(0.69)+\frac{1}{2}\left(-1.10\right)=-0.20 $, indicating that the Greycloak equilibrium $ \psi $ is risk dominant.

Table 4. Normal-form payoffs for an asymmetric technology transition design game instance

Assuming risk dominance drives strategy selection in the bi-level model reveals a valuable insight for strategic robustness and stability. The risk dominance criterion $ R $ indicates which equilibrium has the larger basin of attraction and its relative strength. Specifically, the magnitude of $ R $ captures the stability of the risk-dominant strategy, namely to what degree equilibrium selection (and thereby strategy selection) resists change due to uncertainty or perturbation. As the preceding bi-level design example illustrates, lower-level design decisions that trade system efficiency for robustness can influence strategy dynamics by changing the utility resulting from a particular strategic outcome, thereby increasing the basin of attraction of the collaborative strategy (Valencia-Romero & Grogan Reference Valencia-Romero and Grogan2020). Strategic stability further increases utility robustness by reducing the probability of coordination failure in a positive feedback loop. Consequently, system actors can use $ R $ as an indicator of both the stability of a strategy and the strategic robustness of constituent system designs in an SoS.

However, these observations naturally lead to questions regarding the viability of $ R $ to describe strategy selection and the influence of design decisions on strategy dynamics in more realistic environments. The effect of robust design implementation on strategy selection by system actors with fragile design options is of particular interest (e.g., the effect of one actor selecting the Mk II-B design on the strategy selection of another actor). The following subsections present models and research questions to investigate the effect of strategically robust designs on actors’ strategy selections in multiactor networks.

2.5. Evolutionary games and strategy selection

Evolutionary game theory provides methods to determine the stability of strategies and to recommend or evaluate equilibrium selection criteria for multiactor, repeated games as more representative settings for decision-making than two-actor single-shot games as in the preceding section. Agent-based (actor-based) evolutionary game theory models simulate the fitness and spread of strategies (or other actor attributes) by representing actors as nodes in a graph and relationships among actors as edges between nodes (Szabó & Fáth Reference Szabó and Fáth2007). Evolutionary game theory models actor interactions as games and cumulative or single-round payoffs indicate the fitness of each actor. That is, evolutionary games use relatively many actors in a network, interactions restricted by the network topology, and actor property updates based on a fitness measure. Researchers study different phenomena by adjusting network topology, strategy update rules, frequency and synchronicity, and payoff structures to represent various real-world conditions. Studies generally investigate the distribution or prevalence of strategies under equilibrium conditions.

Kandori et al. (Reference Kandori, Mailath and Rob1993) found that large, well-mixed populations of simulated actors converge to the risk-dominant equilibrium in coordination games (of which stag hunt is a type) with infrequent myopic best-response strategy updates and low probability mutations of actors’ strategies. Consistent convergence to the risk-dominant equilibrium with small strategy selection perturbations demonstrates that it is the stochastically stable equilibrium, also known as evolutionarily stable. Without mutations, convergence is path dependent and determined by the mixed strategies and the proportion of each strategy in the initial population. Referencing the normalized deviation loss $ {u}_i $ computed from payoff values, Roca, Cuesta & Sánchez (Reference Roca, Cuesta and Sánchez2009b) demonstrated that a well-mixed population without mutations using a best-response update rule, playing a symmetric stag hunt game, will converge to the $ \psi $ equilibrium if the proportion of actors playing $ {\psi}_i $ is greater than $ {u}_i $.

Roca, Cuesta & Sánchez (Reference Roca, Cuesta and Sánchez2009a) performed a comprehensive study of the effects of temporal, spatial, and strategy update rules on strategy selection and stability in evolutionary games, relaxing the well-mixed population and slow update assumptions of Kandori et al. (Reference Kandori, Mailath and Rob1993). Many works prior to Roca et al. (Reference Roca, Cuesta and Sánchez2009a) attempted to draw universal conclusions about the effect of individual features, such as topology or selection pressure on strategy distributions. However, Roca et al. (Reference Roca, Cuesta and Sánchez2009a) demonstrated that the effects caused by a given feature are generally convolved with other factors. The results presented by Roca et al. (Reference Roca, Cuesta and Sánchez2009a) extend the findings of Kandori et al. (Reference Kandori, Mailath and Rob1993) to confirm that strategy selection in stag hunt games is described by risk dominance on a variety of network topologies, including random, scale free and lattice networks, when a best-response strategy update rule is used. Therefore, the combinations of details such as the network topology, update rule, and payoff structure are critical for achieving meaningful results.

Braha (Reference Braha2020) further supports the importance of graph topology. His work reports that real-world problem-solving networks are often characterized by high frequencies of three- and four-node directed subgraphs that accelerate convergence to a network-wide solution state, that is, coordination, in a stochastic problem-solving model. These coordination-facilitating graph motifs work to reduce cycling between states. The findings of Braha (Reference Braha2020) may play an important role in establishing the system actor communication networks of future SoS to maximize coordination.

2.6. Research objectives

The bi-level design model presented in the preceding section assumes that risk dominance, as expressed in $ R $, accurately models strategy selection in the stag hunt game for utility-maximizing actors. Classic game-theoretical works such as Carlsson & van Damme (Reference Carlsson and van Damme1993) and Harsanyi (Reference Harsanyi1995) assert that risk dominance is the relevant equilibrium selection criterion in a single shot stag hunt game. Furthermore, evolutionary game theory, subject to specific model features, also indicates that risk dominance is the governing equilibrium selection criterion in stag hunt games with returns that increase linearly with the number of collaborating neighbors. The support from Roca et al. (Reference Roca, Cuesta and Sánchez2009a) is particularly valuable, as their models examine a variety of network types and policy update rules that may be applicable to SoS development and operations. However, the application of risk dominance to evolutionary games with increasing or decreasing marginal returns to collaboration, a common attribute of real-world SoS, has not been examined. This work poses the following question, testing an expanded application of risk dominance as a measure for subsequent work:

1. RQ1. How does risk dominance predict selected strategies in single-level, multiactor evolutionary games representing SoS with payoff functions exhibiting (a) constant, (b) increasing and (c) decreasing marginal returns to the number of collaborators?

Subquestions (a–c) address scenarios when each additional collaborator is of equal value to the focal actor, increasing value to the focal actor, and decreasing value to the focal actor, respectively.

Therefore, the following hypothesis is made for RQ1:

1. H1. Strategy selection in single-level, multiactor network games representing SoS follows the risk dominance criterion, with Pareto efficient strategies selected with greater frequency as $ R $ decreases.Footnote ²

Further refined, the majority of system actors are hypothesized to select the Pareto efficient strategy if $ R<0 $ (the threshold value of $ R $) for constant marginal returns to the number of collaborators. The threshold value of $ R $ is less than zero for increasing marginal returns due to the relatively low value of initial collaborators. Lastly, the threshold value of $ R $ is greater than zero for decreasing marginal returns due to the high value of initial collaborators. Examining all three scenarios is important to establish the relationship between $ R $ and strategy selection across a spectrum of payoff functions representative of SoS returns.

The bi-level model and the relationship between strategic robustness and stability critically depend on the concept that design decisions in an SoS context that increase robustness to others’ decisions also reduce the probability of a coordination failure by reinforcing others’ favorable strategy selection. Conclusions drawn by Grogan & Valencia-Romero (Reference Grogan and Valencia-Romero2019) based on the work of Selten (Reference Selten1995) support this concept. However, no published research has tested the idea in a dynamic setting representative of SoS. Furthermore, direct support from the body of literature in evolutionary game theory provides insufficient support because relatively little research has been done on equilibrium selection in asymmetric games or with actors with asymmetric options, especially in an engineering context. This gap leads to the second research question:

1. RQ2. Does adoption of strategically robust designs by a subset of a population in an evolutionary bi-level design game increase collaboration amongst all system actors, including those with relatively fragile designs?

Testing RQ2 requires games with asymmetric payoffs and asymmetric options, as expressed in the design decision layer. Combining the findings of Roca et al. (Reference Roca, Cuesta and Sánchez2009a) with the theory presented by Selten (Reference Selten1995), the following hypothesis is made for RQ2.

1. H2. Designs that increase strategic robustness reinforce favorable strategic behavior from others, increasing the likelihood of collaboration by all actors in a multiactor bi-level design game representation of SoS.

The research proposes two computational simulation experiments based on a common multiactor model to evaluate the hypotheses. The first experiment evaluates how well risk dominance describes equilibrium selection in a multiactor network, confirming the foundation for the second experiment. The second experiment tests whether designs that increase strategic robustness (system robustness to coordination failure), lowering $ R $, increase collaboration among all actors, including those with relatively fragile system designs.

3.1. Model constructs

The fundamental features of each model are a set of actors, a game with payoffs, and a network connecting the actors. Each actor’s neighborhood is defined by the network and includes all actors with whom they are connected directly by an edge. Actors play the game with each neighbor simultaneously, using the same strategy and design (as applicable) in all interactions within the same round of play. Actors may update their design and strategy decisions between each round of play solely on the basis of past decisions (their own and neighbors’), corresponding payoffs, and possible future payoffs as a function of strategy and design options. Rounds of play continue until termination criteria are met.

Each round of the simulation represents the outcome of a complete set of design and strategy decisions. Simultaneous move games are a proxy for a scenario in which actors commit resources to a particular policy based on beliefs about the future actions of other actors, but only discover what path has been taken after some time has passed. In real-world SoS, this may occur because actors signal the intent to collaborate, but do not follow through, or because collaboration breaks down before system realization. The simulations do not represent the design process itself. In other words, all design and strategy decisions culminating in system deployment are combined into one round of the simulation. Therefore, simulated actors learn their best one-shot actions, not the best action sets for a repeated game representation of an SoS design process. Strategy is used here to refer to the nontechnical decision to collaborate (embodied in the application case as a decision to adopt the Greycloak technology) in the SoS or not. Actor action spaces are defined by their design and strategy decision spaces. An actor’s policy, composed of a design and strategy choice, is updated using the policy update rule described in a later subsection.

Design choices in this context determine strategy, that is, if an actor implements Greycloak, then it is also collaborating in the market with others that do the same. However, this is not universally true of SoS. For instance, diverse defense systems may adopt a set of common communication protocols, enabling combined operations in an SoS, but the systems may intermittently, or never be used as an SoS. In such a context the design may enable a strategy (whether to collaborate), but it does not determine it.

Additionally, strategies are evaluated and selected on the basis of their utility under the possible strategic outcomes, and not with a view to long-term business objectives, or competitive concerns that are typically included in an organization’s long-term planning activities. These considerations could be included in a utility measure used to evaluate design and strategy options, but are excluded from this work to increase simplicity and interpretability of the results.

3.2. Network topology

Each model implements three network topologies, each with 30 actors, as a contextual variable. Including network topology helps to test the hypotheses over a broad range of conditions, thereby increasing the generalizability of common conclusions. The effect of topology itself is not a subject of interest for this work, and is therefore not considered one of the independent variables in the experiments to test each hypothesis.

The three network topologies include a random network, a random network with preferential attachment, and a small-world network (Watts & Strogatz Reference Watts and Strogatz1998; Barabási & Albert Reference Barabási and Albert1999). All three networks are prevalent in the literature on multiactor simulations and evolutionary game theory (Roca et al. Reference Roca, Cuesta and Sánchez2009a). A random network randomly places edges between nodes with equal probability (Erdös & Rényi Reference Erdös and Rényi1959); in the case of this work, until each node has a minimum number of edges. Random networks with preferential attachment are constructed by adding nodes to a minimal network (as few as two nodes and one edge) by connecting the new node to an existing node with probability equal to the ratio of the existing node’s edges and total number of edges in the network (Barabási & Albert Reference Barabási and Albert1999). Small-world networks are formed from a ring lattice in which each node is connected to specified number of nearest neighbors. Edges are randomly rewired to a new node with a specified probability, without forming self-loops or duplicate connections (Watts & Strogatz Reference Watts and Strogatz1998). Small-world networks increase node clustering relative to random graphs, without large average path lengths between nodes. The random networks are simulated with minimum node degree between 1 and 2.

Braha & Bar-Yam (Reference Braha and Bar-Yam2007) studied new product development networks for a vehicle, operating system software, a pharmaceutical facility, and a 16-story hospital and found that all had networks with high-clustering and short path lengths, typical of small-world networks. Therefore, this work assumes that small-world networks are the most representative of those typical in SoS development. However, as relevant network structures likely vary between industries and specific projects, comparisons across the three topologies bound outcomes and help compare to previous results.

3.3. Update rule

The policy update rule is a critical feature of any evolutionary game to update decisions in response to environmental stimuli. The policy update rule determines which actions actors make in generations subsequent to the seed generation. This work uses the myopic best response (MBR) update rule proposed by Blume (Reference Blume1993) and Kandori et al. (Reference Kandori, Mailath and Rob1993) with an implementation similar to Sysi-Aho et al. (Reference Sysi-Aho, Saramäki, Kertész and Kaski2005). Montanari & Saberi (Reference Montanari and Saberi2010) also used the best response update rule in their study of the spread of innovations using Markov chain models.

Consider the set of actors $ \mathcal{A}=\left\{1,\dots, n\right\} $. Every actor $ i $ is connected to a finite set of other actors (its neighborhood) denoted by $ {\nu}_i $. The number of collaborating actors in actor $ i $’s neighborhood, $ \mid \left\{j\hskip1.5pt \in \hskip1.5pt {\nu}_i:{s}_j=\psi \right\}\mid $, is denoted by $ {c}_i $. Actor $ i $ chooses a strategy $ {s}_i^{(t)}\hskip1.5pt \in \hskip1.5pt {\mathcal{S}}_i $ and a design $ {d}_i^{(t)}\hskip1.5pt \in \hskip1.5pt {\mathcal{D}}_i $ from a set of alternatives at time $ t $, plays the game with each neighbor, and receives total payoff $ {G}_i^{(t)}\left({s}_i^{(t)},{d}_i^{(t)}\right) $.

$ G $ takes one of three functional forms: for constant marginal returns with $ {c}_i $:

(3)$$ {G}_i^{(t)}=\sum \limits_{j\hskip1.5pt \in \hskip1.5pt {\nu}_i}\;{V}^{s_i^{(t)},{s}_j^{(t)}}\left({d}_i^{(t)}\right), $$

for increasing marginal returns with $ {c}_i $

(4)$$ {\displaystyle \begin{array}{rc}{G}_i^{(t)}=& {V}^{s_i^{(t)},{\phi}_j}({d}_i^{(t)})+[{V}^{s_i^{(t)},{\psi}_j}({d}_i^{(t)})-{V}^{s_i^{(t)},{\phi}_j}({d}_i^{(t)}){+1]}^{c_i^{(t)}/|{\nu}_i|}-1,\end{array}} $$

or for decreasing marginal returns with $ {c}_i $

(5)$$ {G}_i^{(t)}={V}^{s_i^{(t)},{\phi}_j}\left({d}_i^{(t)}\right)+\left[{V}^{s_i^{(t)},{\psi}_j}\left({d}_i^{(t)}\right)-{V}^{s_i^{(t)},{\phi}_j}\left({d}_i^{(t)}\right)\right]\cdot {\log}_{\mid {\nu}_i\mid +1}\left({c}_i^{(t)}+1\right). $$

The three payoff function forms represent possible network effects in the SoS. Eq. (3) represents the case when an actor’s payoff grows linearly with the number of collaborators in the SoS. Eqs. (4) and (5) represent more complex network effects in which many or few potential collaborators must participate in the SoS for collaboration to be profitable. Real-world SoS may also be characterized by networks with interactions with varied weights, that is, the collaboration of some actors is more valuable than that of others, regardless of whether marginal returns are constant, increasing, or decreasing.

Actors select new $ {s}_i $ and $ {d}_i $ at time $ t+1 $ with probability $ p $ using MBR by

(6)$$ {s}_i^{\left(t+1\right)},{d}_i^{\left(t+1\right)}=\left\{\begin{array}{ll}\underset{s\hskip1.5pt \in \hskip1.5pt \mathcal{S},d\hskip1.5pt \in \hskip1.5pt \mathcal{D}}{\arg \hskip0.1em \max }{G}_i^{(t)}\left(s,d\right),& \mathrm{if}\;k<p\\ {}{s}_i^{(t)},{d}_i^{(t)},& \mathrm{otherwise}\end{array}\right., $$

where $ k $ is a uniform (0,1) distributed random variable. In other words, each actor adopts the design and strategy to those that would have maximized $ G $ in the previous round of play with probability $ p $. The probabilistic formulation disrupts simultaneous updates and mitigates the possibility of cycling between strategies or designs. The MBR update rule rationally, but short-sightedly, maximizes payoff by selecting strategies as if each round of play were the last (Kandori et al. Reference Kandori, Mailath and Rob1993).

As an example, consider the strategic network state in Figure 2 where dark blue actors play the Bluetooth strategy $ {\phi}_i $ and yellow actors play the Greycloak strategy $ {\psi}_i $ (ignoring design decisions in this example). Assume that marginal returns are constant and actor $ i $ realizes payoffs per Table 3 with $ {s}_i^{(t)}={\phi}_i $, receiving a total payoff $ {G}_i^{(t)}=0.4\cdot 2+0.8\cdot 2=2.4 $ for this network state. Alternatively, if actor $ i $ had selected $ {s}_i^{(t)}={\psi}_i $, then the corresponding payoff would have been $ {G}_i^{(t)}=0.35\cdot 2+0.95\cdot 2=2.6 $. Therefore, the MBR rule updates the strategy to $ {s}_i^{\left(t+1\right)}={\psi}_i $ with probability $ p $.

Figure 2. Example game graph. Yellow actors play $ {\psi}_i $ and dark blue actors play $ {\phi}_i $.

MBR has several features that make it well suited for application in an SoS design model and justify its implementation here. Simple imitation rules like ‘follow the best’ require actors to know the best actor with highest payoff, either in the neighborhood or globally, and the best actor’s selected design and strategy. MBR requires only local information; namely, an actor’s own actions and the effect of its neighbors’ actions. Actors’ strategies and the effect of their actions on their neighbors are synonymous in most evolutionary game theory models. In the model used in this work, only neighbors’ strategies affect actor payoffs, and are therefore visible and possible to imitate. Neighbors’ design decisions are not known and cannot be imitated. Furthermore, real-world interactions seldom reveal complete and perfect information about a design actor’s strategy, design, or resulting payoffs, even if neighbors’ designs do affect one’s payoff. Lastly, MBR represents innovative, rational, but limited strategic behavior of the type expected from a process such as decision making in business or design, while maintaining simplicity, continuity with previous research, and interpretable results.

3.4. Model limitations

The implementation of each model feature discussed in this section introduces limitations to the application of results to engineering design. Several are discussed here and others are left for discussion in context of the experiments presented in the following sections.

The basic model construct introduces several limiting features. MBR is used because it produces actor behavior that is short-sightedly rational and requires only local information. However, as van Duinen et al. (Reference van Duinen, Filatova, Jager and van der Veen2016) demonstrate, an actor selecting new technology strategies may switch between policy update processes using repetition, deliberation, social comparison and imitation depending on the actor’s level of satisfaction and uncertainty. MBR is a deliberative policy update rule. As cited earlier, imitation (and social comparison, which is imitative) requires knowledge of neighbors’ past policies. Strategy update rules representative of behavior amongst design actors in an SoS setting is an area requiring additional research. Therefore, this work employs MBR as an adequate proxy for human decision making rather than introducing a complex rule that is not empirically supported.

Second, the model only considers populations with homogeneous payoff functions. Actors in real-world SoS will often have different payoff functions and be influenced to varying degrees by the other actors in the network. Heterogeneous payoff functions and varied actor weight would also introduce additional assumptions that complicate the interpretation of the results, and are, therefore, not considered in this work.

Lastly, the structure of the networks also influences the dynamics by creating clusters that can protect risk-dominated solutions, effectively biassing the local probability of collaboration. Real-world networks are sometimes dynamic (Braha & Bar-Yam Reference Braha and Bar-Yam2006), which could minimize or amplify the impacts of clustering or transfer them temporally and spatially to a new network region. Like other excluded features, the inclusion of dynamic networks would increase model complexity and required assumptions and therefore are reserved for future work.

4.1. Experimental design

The first experiment uses the network topologies and strategy update rule presented in the preceding section to evaluate the effect of risk dominance on strategy selection. Variable payoffs represented by $ S\hskip1.5pt \in \hskip1.5pt \left[-2,0\right] $ and $ T\hskip1.5pt \in \hskip1.5pt \left[\mathrm{0,0.975}\right] $ in Table 5 form the independent experimental variables. The notation for $ S $ and $ T $ aligns with social dilemma game theory literature to represent the payoffs known as the ‘sucker’s payoff’ and ‘temptation to defect’. Varying $ S $ and $ T $ over the corresponding domains produces a range of risk dominance $ R $ values in Figure 3.

Table 5. Normal-form payoffs for a symmetric technology transition game with variable payoffs $ S $ and $ T $

Figure 3. Risk dominance ($ R $) contours as a function of variable payoffs $ S $ and $ T $.

The dependent variable is the density of collaborative strategies selected in equilibrium, measured as the ratio of the number of actors playing the collaborative strategy $ {\psi}_i $ to the total number of actors.

Each combination of topology and payoffs is simulated 1000 times with each actor initially equally likely to implement either strategy. Each of the 1000 trials is initialized with a new graph and a new set of actors, that is, the graph structure is unique for each trial; decisions and dynamics do not carry over from trial to trial. Actors play their neighbors, collect payoffs according to the specified form of $ G $, and then update their strategies using MBR with probability $ p=0.9 $. Each trial is terminated when the number of actors playing each strategy is unchanged in four consecutive rounds of the game. Results for each trial and experimental condition are output as a CSV file with a record of the dependent and independent variables. Theory and past research suggest that, for an initial probability of each actor selecting $ \psi $ of $ 0.5 $, games with $ R>0 $ will converge to $ \phi $ while games with $ R<0 $ will converge to $ \psi $ when the payoff function is linearly increasing with the number of collaborating neighbors.

5.1. Experimental design

The second experiment uses the same network topologies and update rule as the first experiment but the update rule affects both strategy and design selection simultaneously. Simultaneous update is an intentional simplification of real-world collaborative design, in which strategy selection or modification may occur asynchronously with design modification. The ease of updating either strategy or design and the resulting rate at which updates are possible varies between design efforts. This work starts with the simplest case, simultaneous update. Furthermore, because each generation represents the complete sequence of events leading to system deployments, asynchronous updates do not increase realism.

The experiment introduces one design for Bluetooth strategy and two designs for the Greycloak strategy with a variable payoff for one design and initial probability of collaboration as two independent factors. Whereas the first experiment demonstrates how $ R $ affects strategy selection in a population initially composed of actors equally likely to start as defectors or collaborators by varying $ S $ and $ T $ for all actors, the second experiment investigates how varying options for a single payoff for a subset of the population and varying the probability that each actor is a collaborator in the initial population affects collaboration in the equilibrium state. The goal of the second experiment is to understand how design updates by a subset of a population affect collective dynamics under initial conditions (the probability that actors are collaborators) that vary the favorability of collaboration.

The population contains two types of actors assigned with equal probability. Both types have two strategies $ {\mathcal{S}}_i=\left\{{\phi}_i,{\psi}_i\right\} $. Type 1 actors have design spaces with two alternatives $ {\mathcal{D}}_1 $ = {MkI, MKII –A} associated with the Bluetooth and Greycloak technologies respectively. Type 2 actors have design spaces with three alternatives $ {\mathcal{D}}_2 $ = {MkI, MKII –A, MkII – B}. Type 1 actors are technically unable to implement the Mk II-B design because it uses the Bluetooth and Greycloak transceivers and is, therefore, too large for their products.

Design payoffs are given in Table 6; all are fixed values except for the Mk II-B design in context $ \left\langle {\psi}_i,{\phi}_j\right\rangle $ which takes on variable value $ C $ between 2.0 and 4.9. With this formulation, games always exhibit the stag hunt dynamic.

Table 6. Mobile technology transition design options

Assuming the Mk I design is used with $ {\phi}_i $, pairing the Mk-IIA design with $ {\psi}_i $ yields $ {u}_i=\frac{2}{3} $ while pairing the Mk II-B design with $ {\psi}_i $ yields $ {u}_i $ values between $ \frac{1}{16} $ and $ \frac{2}{3} $. Risk dominance reaches a threshold value $ R=0 $ at $ C=4.25 $ for Type 2/Type 1 pairs and $ C=3.5 $ for Type 2/Type 2 pairs.

The initial probability of collaboration $ {p}_{\psi } $ varies between 0 and 1. Decreasing $ {p}_{\psi } $ increases the severity of the payoff perturbation experienced by an actor playing $ {\psi}_i $. The probability of any actor initially using the Mk II-B design is always 0. All actors starting with the $ {\phi}_i $ strategy select the Mk I design and those starting with the $ {\psi}_i $ strategy select the Mk II-A design. Type 2 actors may select the Mk II-B design in subsequent rounds of play.

The dependent variable is the fraction of Type 1 actors that select strategy $ {\psi}_i $ in the equilibrium state. While the fraction of strategy $ {\psi}_i $ among Type 2 actors is also of interest, the result is expected to follow the patterns observed in Experiment 1. Of greater interest is the effect that robust design implementation has on strategic behavior of neighbors with relatively fragile designs (Type 1 actors). The risk dominance strategy selection concept predicts that actors are more likely to collaborate if their neighbors have lower $ {u}_i $ values such as that provided by robust design solutions in an engineering context.

Each combination of topology, $ C $, and $ {p}_{\psi } $ is simulated 1000 times following the same process as the first experiment. Each simulation is terminated when every actor plays the same strategy and design in four consecutive rounds of the game.

6.1. Limitations to generalizability

In accordance with strategy selection following risk dominance, the positive effect of strategically robust designs on the collaborative behavior of system actors observed in results depends on the design actors possessing or developing a sufficiently accurate knowledge of the strategy dynamics. The selected model informs actors through repeated interactions. While repeated interactions between multiple system actors are characteristic of SoS design, decisions, particularly strategic decisions governing SoS formation, are often separated by months or years, limiting the opportunities for learning in repeated interactions with a fixed set of partners. Furthermore, early interactions may not produce rewards and penalties consistent with the overarching coordination game dynamic, limiting the learning that can be achieved from those interactions. Instead, it is more likely that learning occurs over a longer, multiprogram or multisystem timescale. Therefore, initial decisions informed by an assessment of SoS strategy dynamics are critically important because strategic exploration may be prohibitively costly.

Whilst strategically robust designs minimize the impacts of coordination failure independent of the strategy dynamic, incomplete information about the dynamic in real-world applications limits the hypothesized stabilizing effects. Lack of understanding of the level of robustness needed for a particular application may lead to overly conservative designs that diminish upside potential gains or overly optimistic (fragile) designs that expose a high risk of losses. Likewise, system actors are not encouraged to collaborate unless they are aware that others pursue strategically robust designs. This issue is exacerbated by the generally asymmetric positions of the constituent system actors. Each will usually have a unique payoff function and will differentially value the collaboration of the other system actors. While the three payoff function forms studied here begin to explore this issue, they do not replicate the actor-to-actor variability present in real systems. Furthermore, additional information, such as risk or loss aversion, reference points, and probability weights are required to transform payoffs into actor-specific utilities. Collecting such information about potential participants in an SoS presents difficulties, as has been the experience of many economists in other domains (Barberis Reference Barberis2013). Further research is required to determine the necessity and practicality of assessing SoS strategy dynamics using actor-specific utility functions.

Two methodological adjustments are required to close this gap and enable practitioners to implement strategically robust designs that also enhance the stability of collaboration. The first is a method for estimating the true strategy dynamics from incomplete information and characterizing the uncertainty of the estimate. Bayesian games (Harsanyi Reference Harsanyi1967) and team reasoning (Bacharach Reference Bacharach1999) are promising methods to characterize payoffs under incomplete information and can be used to formulate games with probability distributions of utility as a function of a design, business alliance or other factors creating uncertainty. Significant research is required to develop and validate design methods, which incorporate such features.

The second is a method to communicate one’s strategic robustness to potential partners with the aim of increasing their likelihood of committing to collaboration. In intra-organization collaborations, such as those common within agencies of a government, signalling one’s strategic robustness may be relatively simple because information can be shared more freely. Signalling strategic robustness in commercial endeavors, particularly those with collaborators that are also competitors, is comparatively difficult due to concerns about maintaining or gaining competitive advantage. Additional research building on relevant economics literature (e.g., Crawford & Sobel Reference Crawford and Sobel1982; Spence Reference Spence2002) may help understand how to communicate strategic intent in different environments.

Finally, it must be stated that it is not yet known if human designers are more likely to collaborate when their collaborative partners adopt strategically robust designs, that is, whether human decision-makers use risk dominance as a decision criterion in a collaborative design context from both descriptive and normative perspectives. This is a critical knowledge gap that should be investigated by design experiments with human subjects building to real-world case studies to test the key ideas presented in this work.

6.2. Implications for engineering design

Notwithstanding the limitations in the preceding section, there are a few key implications of this research for design practitioners. The first experiment demonstrates that, subject to local network effects, simulated system actors following myopic decision rules maximize utility by selecting the risk-dominant strategy in a stag hunt game as a model of technology transition SoS with uncertain collaborative behavior. Design practitioners, especially policy makers, can learn from this simple experiment that the mere existence of a Pareto-efficient SoS design is insufficient grounds for each actor to commit to collaboration. Even in cases with the prospect of mutual benefit for all parties, the spectre of a technical failure or discovery of a superior independent strategy by one actor could lead to a coordination failure and corresponding utility loss. Risk dominance represents a significant single factor that differentiates strategic stability of equilibria.

The second experiment builds on the first, demonstrating that designs which are robust to coordination failures increase collaboration by all system actors, thereby reinforcing their expected utility. However, because strategy selection for each actor is still driven by risk dominance and the initial probability of collaboration, even actors with robust collaborative designs sometimes maximize value through defection. Two lessons can be drawn from this result. First, decisions to form an SoS should be supported by analysis of strategy dynamics to determine if collaboration is favorable and, if so, whether options exist to increase its stability. Second, even if it fails to achieve a sufficient stabilising effect, pursuing strategically robust designs reduces losses in the event of a coordination failure. Implementing strategically robust designs maintains the operational independence characteristic of SoS (Maier Reference Maier1998).

For design researchers, the second experiment clearly demonstrates that a strategically robust design for one or more constituent systems enhances collaboration amongst all constituents in a robustness feedback loop, assuming that each is aware of the robustness of the others’ designs and influenced by risk dominance (limitations previously cited). Intuitively, the robustness feedback loop is built on the idea that system actors with little to lose and much to gain through collaboration are likely to do so. Design actors with a riskier value proposition can collaborate with higher confidence knowing that others are committed. The extent to which this robustness feedback loop is operative amongst human designers is a potentially fruitful area of research.