Agents ready with their solutions in their mind

The design team activity consists of idea generation and idea selection. In idea generation, agents generate solutions based on a set of rules that are given in Singh et al. (Reference Singh, Cascini and McComb2021). Agents generate solutions in their minds based on (a) their way to explore solution space, (b) memory and recalled ability, (c) learning from their past events and (d) the effect of the influencers. Singh et al. (Reference Singh, Cascini and McComb2021) show how an agent’s experience and influencers in a team affect an agent’s idea generation approach. Idea selection comprises team interaction and decision-making (Figure 3). The team interaction starts after the design team agents are ready with their solutions to communicate with the team (step 1 in Figure 3). The idea selection process that can be seen in Figure 3 is elaborated in the sections below.

Figure 3. Flow of processes during idea selection in a design session.

Proposing solutions

As it is known that communication is key in the design process and communication depends on the individual’s self-efficacy level. Self-efficacy mediates the five big personality traits, including extrovertism (Stajkovic et al. Reference Stajkovic, Bandura, Locke, Lee and Sergent2018) and other studies like Cao et al. (Reference Cao, MacLaren, Cao, Dong, Sayama, Yammarino, Dionne, Mumford, Connelly, Martin, Standish, Newbold, England and Ruark2020), also found that talkative agents (an extrovert characteristic in agents) are perceived as more influential. Therefore, even though all individuals generate ideas, some might not be enough confident to propose to their peers as seen in step 2 in Figure 3. In the current model, there is a low probability that all the agents will propose solutions. Agents, who have higher self-efficacy than others, communicate their ideas more often. However, the possibility of a low self-efficacy agent proposing its solution to the team is not completely eliminated. The number of agents who are selected to propose their solutions is given in Eq. (1).

(1) $$ {N}_{\mathrm{Min}}\hskip0.35em \le \hskip0.35em \left({N}_{\mathrm{SA}}\right)\hskip0.35em \le \hskip0.35em {N}_{\mathrm{tot}}, $$

$ {N}_{\mathrm{Min}} $ are the minimum number of agents that should at least propose solutions. Based on educational experiences as described by authors in their studies (Reid & Reed Reference Reid and Reed2000; Lahti, Seitamaa-Hakkarainen & Hakkarainen Reference Lahti, Seitamaa-Hakkarainen and Hakkarainen2004) as well as a common observation made during the experiment presented in this study that more than 30% of individuals in small design teams (i.e., 5–7 individuals) propose solutions during a session (step 2: proposing ideas to the team in Figure 3). Since the model simulates a team of six agents, $ {N}_{\mathrm{Min}} $ here was taken to be 3. $ {N}_{\mathrm{SA}} $ is the number of selected agents that propose solutions out of the total number of agents in a team N _tot.

The probability of an agent being selected to propose its solution (P _SA) depends on how the self-efficacy is varied in the team Var (Team_SE).

(2) $$ {P}_{\mathrm{SA}}\alpha\;\mathrm{Var}\left({\mathrm{Team}}_{\mathrm{SE}}\right), $$

(3) $$ \mathrm{Var}\left({\mathrm{Team}}_{\mathrm{SE}}\right)\hskip0.35em =\hskip0.35em \frac{\sum_{i=1}^{N_{\mathrm{tot}}}{\left({\mathrm{SE}}_i-\mu \right)}^2}{N_{\mathrm{tot}}}, $$

where μ is the mean of the self-efficacy of agents in a team and SE is the self-efficacy of an agent i. This means that there is a high probability of all the agents who are selected (or selected agents, SA) to propose their solution have high self-efficacy when the team self-efficacy variance is high. When the team self-efficacy variance is low, that is, all the agents have either low self-efficacy or high, agents are randomly selected to propose their solutions. Cases when then N _SA is higher than the number of agents with high self-efficacy, additional low self-efficacy agents are selected randomly selected to propose their solutions. Agents who did not propose their solution still have their solution in their minds that will be regarded as opinions for the further decision-making process.

Merging of solutions

After individually generating ideas, collective work is required. Likewise, at this stage at step 3 in Figure 3, agents who have proposed their solutions merge similar ones. The similarity between the solutions is computationally defined as the distance between the solution points on s design space, that is, the two solutions are similar if they are close to each other on a solution space. Merging similar solutions uses the k-means clustering method. k-means clustering is a popular and simplest cluster analysis method in data mining that uses Euclidean distances between points for a given number of k (clusters). However, in order to define the value of k, one approach is to try different values of k, for example, one can start with k = 1, k = 2 and so on until k = total number of solutions (data points), by comparing the variation within the clusters (Steorts Reference Steorts2017). In this work, k is chosen randomly and lies between 2 < k < $ {\mathrm{Num}}_{\mathrm{totSoln}}-2 $ where $ {\mathrm{Num}}_{\mathrm{totSoln}} $ is the total number of solutions. By choosing k within this range would give optimal and near-optimal values of k. The using near-optimal values of k in the model could be justified by the presence of environmental noise or other unaccounted factors, which affected the optimal way of merging similar solutions (data points).

For the clustering (i.e., merging) similar solutions proposed by the agents, python’s Scikit-learn machine learning library was used. The flowchart for getting the solutions that should be merged is given in Figure 4. The merged solution that becomes the new common proposed for the selected agents in the cluster is the centroid. k-means aims at minimising an objective function given in Eq. (4).

(4) $$ J(V)\hskip0.35em =\hskip0.35em {\sum}_{i\hskip0.35em =\hskip0.35em 1}^c{\sum}_{j\hskip0.35em =\hskip0.35em 1}^{c_i}{\left(\left|\left|{S}_i-{v}_j\right|\right|\right)}^2, $$

where ||S_i − v_j || is the Euclidean distance between S_i and v_j (S is the set of positions of the solutions points and v_j is the positions of the centroids. c is the number of clusters and c_i is the number of solutions in cluster i.

Figure 4. Flowchart for merging similar solutions.

Coalition groups of like-minded agents

Most of the literature focuses on strategies for decision making that is used to assist decision-makers to solve problems in a systematic and consistent manner to reach optimal solutions. However, here the goal of this study is not to simulate agents to make an optimal decision but to understand how different team dynamics affect the design outcome. Petit & Bon (Reference Petit and Bon2010) identified some rules related to the individual (based on inter-individual differences in physiology, energetic state, social status, etc.) and (or) self-organised (context and group size) state that could govern the collective decision-making in animals. Some of the rules stated in their work were selected to have a simple and useful simulation. These are related to individualization where self-efficacy state and social status (degree of influence as perceived by others) were considered, and others related to self-organisation was the size of the majority group. Taking this into consideration steps 4 and 5 in Figure 3 shows that during decision-making the individuals’ opinion could be affected by the (a) the influencer’s effect (stated as expert effect ‘induced by the presence of a highly confident individual in the group’) or (b) the majority effect ‘caused by the presence of a critical mass of laypeople sharing similar opinions’ (Moussaïd et al. Reference Moussaïd, Kämmer, Analytis and Neth2013).

The majority effect (in Steps 4 and 5 of Figure 3) is based on Cartwright (Reference Cartwright1971) model of choice shift that explains why group decision making is more complicated than just taking an average of group members’ decisions. Cartwright (Reference Cartwright1971) stated two subtypes of majority influence processes: coalition process and pure majority process. Coalition (step 4) takes place when the judgements (opinions) of individuals are close to each other and it tends to dominate the group judgement process. Majority process (step 5), where the judgement of a larger (majority) group of individuals influences the judgement of other team members. Both coalition and majority process contribute to majority influence such that the influence of individual team members or influencer(s) is less effective during decision-making. However, in the case when all the individuals are closer (have a similar opinion) to that of an influencer, its effect would be exaggerated.

1. (i) Coalition groups of ‘like-minded’ agents:

   Some studies of social network show that opinions in a social network suffer locality effect, that is, they get localised to given groups without infecting the whole society (Wu & Huberman Reference Wu and Huberman2004) The team members in the real-world communicate during the decision-making process and they would be aware of each other’s thoughts on the solutions. Similarly, each agent at this point after step 3 is aware of the other agents’ proposed solutions and opinions and form coalition groups with (as shown in Figure 5) (a) the agent(s) who have proposed the solution or (b) to the agent(s) who did not propose any solutions. (c) Agents could also not be a part of these coalition groups. The (a) and (b) occur when their opinion on the problem is similar to that of the other (Cartwright Reference Cartwright1971; Read & Grushka-Cockayne Reference Read and Grushka-Cockayne2011). Computationally this means that the distance between their proposed solution/opinion (not a proposed solution) and other members’ solution/opinion are close such that they form a coalition group. The groups formed based on similar opinions behave collectively when evaluating proposed solutions. Case (c) occurs when an agent’s opinion is not similar (close) to the others, hence it stays alone.

2. (ii) Deciding which solution to choose:

   In collaborative design sessions, the design team is not directly aware of the quality of their solutions, was inspired from the real-world representation where, when the designers start working (and they have no past experience similar to the agents in this case), they go by trial and error. However, unlike humans who have an intuition or long-term memory that help them to get an idea about the quality value of their solutions, agents in the model are not capable of doing so. Agents in the model are also not informed of the design solution space, that is they do not know the quality values of the solutions. Thus, in this perspective, when deciding which solution to choose, the agents go by the confidence level of the proposing individual/group. The majority of the models of opinion formation in groups are based on the confidence level of agents (Hegselmann & Krause Reference Hegselmann and Krause2002). Therefore, the two behavioural factors that were considered when deciding on which solution(s) to select are the number of agents in the coalition group and their self-efficacies. The first factor was chosen as it is known that influence increases with the number of individuals in the group (to a certain point) (Bond Reference Bond2005), hence affecting other team members’ actions during decision-making. The second factor was chosen as individuals with lower self-esteem, those who are dependent on and those who have a strong need for approval from others are also more conforming (Jhangiani & Tarry Reference Jhangiani and Tarry2014). Thus, similar to the real world, the model also behaves in a manner where more and more individuals have the same opinion, a less confident individual is likely to act like a ‘sheep’ and a more confident one is independent to think for itself when evaluating the proposed solutions. For example: if there are three agents with low/average self-efficacy and two agents with high self-efficacy, the sum of self-efficacies of agents in a group is calculated (Figure 5). If the three agents with low/average have higher or similar self-efficacy sum than the other group (i.e., in this case, two agents with high self-efficacy), they would not change their mind and will stick to their solution. If the sum of three agents with low/average self-efficacies is lower than two agents with high self-efficacy, the former group would go with the solution proposed by the group of two agents (explained in detail in the following sections).

This majority influence is explained by the coalition and majority process, where the behaviours and beliefs of a larger (majority) of individuals in a coalition group influence the behaviours and beliefs of a smaller group (Nemeth Reference Nemeth1986). This happens when the group of ‘like-minded’ agents have higher cumulative self-efficacy than those who proposed the solution. Hence, it is more likely that the latter group of agents (or an agent) will agree with the former group (DeRue et al. Reference DeRue, Hollenbeck, Ilgen and Feltz2010). On the contrary, when the cumulative self-efficacy of the coalition group is less than other groups/individuals, minority influence occurs, and the group is likely to agree with the proposed solution of the influencer(s). Similarly, studies like Hegselmann & Krause (Reference Hegselmann and Krause2002) simulated symmetric or asymmetric confidence in team agents. They found that when all agents had similar confidence (symmetric confidence) the opinions split into many. In contrast, asymmetric cases resulted in convergence into the direction that was governed by asymmetric confidence profiles (Hegselmann & Krause Reference Hegselmann and Krause2002).

Figure 5. An example of a decision-making scenario during Idea selection.

As an example shown in Figure 5, whether an agent is in a coalition group or alone, self-efficacy decides whether to agree with the proposed solution or not. When the self-efficacy is lower than the compared group/individual, how much an agent (i) agrees (A) with the other agent’s (j) proposed solution depends on two factors is given in Eq. (5). Agreement is often represented as binary (i.e., yes/no or agree/disagree) and it is not possible to distinguish between moderate and extreme values. In the social opinion formation model, individuals are affected by their peers in a socially connected system (Nguyen et al. Reference Nguyen, Xiao, Xu, Wu and Xia2020). Thus, they could have a range of agreement values, in other words, they could agree more with some and slightly less with others (Sîrbu et al. Reference Sîrbu, Loreto, Servedio and Tria2017). Based on this, the agreement in the model is not binary but a continuous value that depends on the following factors (as hypothesised at the beginning):

1. (i) The amount of influence (I) ‘proposed solution agent’ j has on the agent i who is evaluating its support to the proposed solution.

2. (ii) The past amount agreement ( $ {P}_A $ ) agent j had while deciding on agent i’s proposed solution.

(5) $$ {A}_i^j\left(I,{P}_A\right)\hskip0.35em =\hskip0.35em {w}_1\left({I}_i^j\right)+{w}_2\left({P_A}_j^i\right). $$

The weights ( $ {w}_1\hskip0.20em and\;{w}_2 $ ) used in Eq. (5) were taken as 0.5.

(6) $$ {I}_i^j\left(\Delta \mathrm{SE},\mathrm{SE},T\right)\hskip0.35em =\hskip0.35em {w}_3{\left(\Delta {\mathrm{SE}}_{i-j}\right)}^{1.5}+{w}_4\left({\mathrm{SE}}^j\right)+{w}_5\left({T}_i^j\right), $$

where $ I $ is the degree of influence from agent j perceived by agent i, which is given below as a function of $ \Delta \mathrm{SE},\mathrm{SE}\;\mathrm{and}\;T. $ is the difference in self-efficacy of agent i and agent j, T is the degree of trust agent i has on agent j. $ \mathrm{SE} $ is the self-efficacy of an agent j (a more detailed discussion on this formulation is given in Singh et al. (Reference Singh, Cascini and McComb2021)). The self-efficacy of the two agents are compared, that is, if SE_i > SE_j then the $ {\Delta \mathrm{SE}}_{i-j} $ = 0. Otherwise, the absolute value of $ {\Delta \mathrm{SE}}_{i-j} $ is calculated and the influence of j on i is calculated as shown in Eq. (6). This considers the impact of $ \Delta \mathrm{SE}\;\mathrm{and}\;\mathrm{SE}\;\mathrm{on} $ a low self-efficacy agent, and the impact on $ \mathrm{SE} $ on a high SE agent when they perceived influence from their peers (as proposed in Singh et al. Reference Singh, Cascini and McComb2021).

Controller agent actions

Before the solution(s) are communicated to the controller agent who is analogous to the project manager, project leader, professor or others in a similar position (step 6), the total agreement (A _total) on a proposed solution (ps) is calculated (as Eq. (7)) for all the proposed solution.

(7) $$ {A}_{{\mathrm{total}}_q}\hskip0.35em =\hskip0.35em {\sum}_{i\hskip0.35em =\hskip0.35em 1}^{N_A}{A}_i, $$

∀q∈{ps ₁, ps ₂…, ps_Q} where Q is the number of proposed solutions. N_A is the total number of agents who agreed with the proposed solution (ps) and i as the initial starting index.

At times, it is seen that the design team selected one final or multiple concepts. Here the maximum number of solutions was chosen to be 3 that was based on common observation as well as the teams in the empirical study were asked to select up to 3 concepts final concepts for their presentation to the experts. To simulate similar behaviour in the model to decide whether one or multiple solutions are proposed to the controller agent, the distribution of the total agreement (A _total) for all proposed solutions is calculated as in Eq. (8). In other words, the probability of the design team to propose one final solution to the controller agent is more when the distribution is high (i.e., high agreement on some proposed solutions) (Sîrbu et al. Reference Sîrbu, Loreto, Servedio and Tria2017). Conversely, there is a higher probability that the agents would propose three solutions to the controller agent when the distribution of the agreements on the proposed solution is low (i.e., similar agreement values on all the proposed solutions), thus, there is no clear dominant solution.

(8) $$ \mathrm{distribution}\left({A}_{\mathrm{total}}\right)\hskip0.35em =\hskip0.35em \sqrt{\frac{\sum_{q\hskip0.35em =\hskip0.35em 1}^Q{\left({A}_{{\mathrm{total}}_q}-\mu \right)}^2}{Q}}, $$

Q = number of proposed solutions at the end of step 1 (Figure 3), q is the starting index and μ is the mean.

When the other team agents select one or more solutions to communicate to the controller agents, the self-efficacy of the agents whose solutions or merged solutions were selected increases. While the self-efficacy of those whose solutions were not selected decreases.Footnote ¹

The controller agent can assess the solutions proposed by the team and give feedback (e.g., a senior designer, project manager, leader and others in similar roles who evaluate the outcome of a team of novice or less experienced designers or students) as seen in Eq. (9). The feedback is based on a probability ( $ {P}_{\mathrm{feedback}} $ ) that a randomly generated number will be smaller than the solution quality value. There is a higher probability of getting a higher feedback value when the quality of the solution is high. However, the probability of getting lower feedback value on a high-quality solution is not completely eliminated as in real-world teams often fail despite having a good concept due to external factors (such as wrong market timing or change in the consumer behaviour). Alternatively, there is a very small chance of getting good feedback on a bad quality solution.

(9) $$ \mathrm{feedback}\hskip0.35em =\hskip0.35em {P}_{\mathrm{feedback}}. $$

(10) $$ {P}_{\mathrm{feedback}}\alpha \hskip0.35em \mathrm{solution}\ \mathrm{quality}. $$

In this case, when multiple concepts are proposed to the controller agent, it picks the concept with the highest quality and shares that information with the team. Good feedback (above average, in this case, 0.5) results in an increase in self-efficacy of the design team agents and low feedback decrease self-efficacy. The amount of increase and decrease in self-efficacy depends on the current self-efficacy level of an agent (Singh et al. Reference Singh, Cascini and McComb2020). Besides affecting self-efficacy, the reputation of agents is also updated (Singh et al. Reference Singh, Cascini and McComb2021).

Summary of the empirical results

The model assumes that the perceived degree of influence by an individual and the past agreement their peers had with them are some of the factors that affected their agreement on the proposed solutions by their team members. The empirical study findings, besides hinting at the presence of coalition groups that are formed based on the similarity of solutions/opinions that lead to the majority effect, show that the influencer effect could also be seen when individuals were agreeing with other proposed solutions, hence supporting the assumption. These findings show the presence of the two social factors considered in the model during decision–making in idea selection. The results from the empirical study support the assumption by showing that an individual’s (j) perceived degree of influence by another individual (i) is correlated to an individual’s agreement (A ∝ I) and was also associated with individual j’s agreement when individual i proposed its solution. The data collected during the empirical study did not capture how many times the teams generated and selected ideas. If the team members proposed and selected solutions multiple times or were asynchronous (i.e., some individuals proposing solutions in one session and some others in the other sessions), then using the term ‘past agreement’ (P_A) is accurate. Thus, assuming that the agreement of one individual is affected by the agreement its peer had when this individual proposed a solution in the past, (A ∝ P_A). Hence, the study supports the assumption that these two factors could be associated with an individual’s agreement during idea selection.

Exploration related findings

The exploration of the agents is measured in three different ways; spread, exploration quality index and local exploration quality index. These three different ways were chosen as it would be useful to know how diverse were these solutions, the quality of the explored solutions with respect to the design space and the number of good solutions proposed by a team.

Spread is the dispersion of the solutions (Eq. 11)). It is calculated by getting the distance between each solution from the centroid of all the solutions in the design space. The variation in these distances (i.e., the distance between a solution and centroid) gives the idea about how the solutions are located in the design space. The spread shows how different the solutions are from each other; in other words, it exhibits variety in the solutions. If S is a set of n proposed solutions on a design space having two design variables, $ S\hskip0.35em =\hskip0.35em \left\{\left({x}_1,{y}_1\right),\left({x}_2,{y}_2\right),\dots .,\left({x}_n,{y}_n\right)\right\} $ . The coordinates of a centroid $ c\hskip0.35em =\hskip0.35em \left({c}_1,{c}_2\right), $ are calculated as $ \left({c}_1,{c}_2\right)\hskip0.35em =\hskip0.35em \left(\frac{1}{n}{\sum}_{i\hskip0.35em =\hskip0.35em 1}^n{x}_i,\frac{1}{n}{\sum}_{i\hskip0.35em =\hskip0.35em 1}^n{y}_i\right) $ . The average distance $ \mu $ from that centroid is $ \mu \hskip0.35em =\hskip0.35em \frac{1}{n}{\sum}_{i\hskip0.35em =\hskip0.35em 1}^n\left|\left|{S}_i-c\right|\right| $ , where $ \left|\left|{S}_i-c\right|\right| $ is the Euclidean distance d given as $ d\hskip0.35em =\hskip0.35em \sqrt{{\left({x}_i-{c}_1\right)}^2+{\left({y}_i-{c}_2\right)}^2} $ . The spread or the variety among the solutions can be calculated as the standard deviation of these distances from the centroid (as given in Eq. (11)). Where N is the total number of distances between the solution coordinates and the centroid.

(11) $$ \mathrm{Spread}\hskip0.35em =\hskip0.35em \sqrt{\frac{1}{N}\sum_{j\hskip0.35em =\hskip0.35em 1}^N{\left({d}_j-\mu \right)}^2.} $$

Exploration Quality Index (EQI): Exploration quality index is the ratio of the number of solutions proposed on a lower resolution solution space (solns_r) above a certain threshold, t (in this case t is above 0.6, where 0 is a minimum and 1 is a maximum solution quality value) to the total number of solutions (totSoln_r) available on the design solution space greater than the threshold value (Eq. (12)). The lower resolution of solution space means that the original solution space (100 × 100) is decreased in size by a factor (5 in this case) so that the resultant is a smaller space (20 × 20). This means that if an agent explores five neighbouring solution cells, the average is calculated. It was done to avoid having an inaccuracy that could arise, for example, when an agent explores five immediate neighbour cells to an agent exploring five cells at a larger distance.

(12) $$ \mathrm{EQI}\hskip0.35em =\hskip0.35em \frac{\mathrm{s}{\mathrm{olns}}_r}{{\mathrm{totSolns}}_r}. $$

Local Exploration Quality Index (LEQI): Local exploration quality index is the ratio of the number of solutions proposed (solns) that are above a certain threshold, t (in this case t is above 0.6) to the total number of solutions proposed (totSoln) (Eq. (13)) on a given design space (i.e.,100 × 100).

(13) $$ \mathrm{LEQI}\hskip0.35em =\hskip0.35em \frac{\mathrm{solns}\;}{\mathrm{totSoln}\hskip0.24em }. $$

The results related to the different exploration measures explained above are shown and discussed below. ANOVA was used for analysis where all the data samples were normally distributed, which was the case with the spread, EQI, LEQI, and quality values for 1 and 3 solutions for various influencer-team compositions. Therefore, a pairwise T-test was used when doing a pairwise comparison. Others like quality and contribution had at least one sample that was not normally distributed therefore, Kruskal–Wallis was performed and for the pairwise comparison, Conover’s test was done.

Figure 10 shows a different spread or the variety of the final selected solutions for various influencer-team compositions (ANOVA F = 11.24, p < 0.001). The pairwise difference comparison in the spread values can be seen in Figure 11. In general, it can be noticed that teams with no well-defined influencers (i.e., the second scenario with no and all influencers) behave differently than the teams with well-defined influencers (i.e., one, two, and three influencers) in their spread values (having p < 0.05). Even though the teams with no influencer have the most variety, they do not differ significantly from the teams of 1 influencer in their proposed solution spread values. The teams with well-defined influencers behaved similar to each other in the exploration of the design space when proposing their solutions to the controller agent, hence no significant difference could be seen in their spread values. Teams where all the agents had similar and high self-efficacy (i.e., all influencers), produced the least variety in their proposed solutions. The all influencer team’s variety values differ significantly from the rest of the influencer-team compositions.

Figure 10. Spread of the proposed final solution.

Figure 11. Post hoc pairwise T-test after ANOVA to do pairwise comparisons of the spread values (Holm–Bonferroni correction was performed on the data before the T-tests and Conover’s tests).

Figure 12 shows the EQI and LEQI values. Even though it can be seen that the EQI of the teams with no well-defined influencers had more alteration (i.e., all influencers had the highest and no influencers had the lowest EQI), there was no significant difference in the EQI values for all the team compositions (ANOVA F = 1.66, p = 0.16). LEQI on the other hand for all the team compositions differ significantly (ANOVA F = 3.399, p = 0.009). This significant difference in the LEQI values was mainly due to all influencer teams who had the highest LEQI. All influencer team’s LEQI differed significantly in comparison to the other teams with well-defined influencers and also with teams with no influencer (Figure 13). LEQI of the teams with well-defined influencers and no influencer had no significant difference in their values.

Figure 12. EQI and LEQI of the teams with different influencers.

Figure 13. Post hoc pairwise T-test after ANOVA to do pairwise comparisons of the LEQI of the proposed solutions.

Quality findings

The quality of the solution is the value of a point on a design solution space. The quality of the final solution is the value of the single solution that the team of agents proposed to the controller agent or the best solution according to the controller agents in case of multiple solutions. The final solutions in the various team composition differed significantly in the quality values (Kruskal–Wallis H = 15.35, p = 0.004). It can be seen from Figure 14 that all influencers teams had the best quality of the final proposed solutions throughout the design project, while the other team compositions had minor differences. However, not much difference can be seen towards the end of the project. From Figure 15, it can be noticed that the quality of the final solutions by all influencer teams differ significantly from other team compositions. The difference in the quality values of all influencer teams was comparatively lesser with the teams where agents had no well-defined influencer and all had low self-efficacy (i.e., no influencer) and the teams where half of the agents had higher self-efficacy than others (i.e., three influencers) than the teams with well-defined influencers especially one and two influencers.

Figure 14. Final quality of the solution with the standard error of the teams with different numbers of influencers.

Figure 15. Post hoc Conover’s test used after Kruskal–Wallis to do pairwise comparisons on the final quality values.

As discussed in Section 3 that depending on the disparity in the total agreement value, a team could propose single or multiple solutions to the controller agent. Figure 16 (top) shows the number of times the teams proposed multiple solutions alternatives (in this case 3) or single solutions to the controller agent.

Figure 16. Single versus multiple solutions (3) final solution count (top)and quality (bottom).

To see more variations in the results the two extremes (i.e., one or three solutions) were extracted and analysed. All influencer teams proposed more multiple solutions to the controller agent while other team compositions mainly proposed single solutions. Proposing multiple solutions results in better solution quality as the controller agent has the liberty to select the most promising solution out of the multiple alternatives proposed (Figure 16, bottom). The solution quality differed significantly when different teams proposed single (ANOVA F = 4.22, p = 0.002) as well as multiple solutions to the controller agent (ANOVA F = 4.64, p < 0.001). This difference in the quality was significant between all influencer teams and other team compositions when they proposed single solutions to the controller agent (Figure 17, top). The difference in the quality of the solution when multiple solutions were proposed was most significant between all and no influencer teams and teams with two or three influencers (Figure 17, bottom). An interesting thing to see in Figure 17(bottom) is that no significant difference in the multiple proposed solution quality between all and 1 influencer was found.

Figure 17. Post hoc pairwise T-test after ANOVA to do pairwise comparisons of the quality of the 1(top) and multiple (bottom) proposed solutions to the controller agent.

Additional team behaviour findings

Agreement, as described in the Model description, is the amount an individual agrees with the other’s proposed solution. The mean agreement value in the teams of different compositions changes throughout the project as seen in Figure 18 (Kruskal–Wallis H = 13.5, p = 0.009). Teams with well-defined influencers have higher and a different pattern of agreement values than in teams with no well-defined influencers. However, a very weak correlation could be found between agreement values and solution quality of these teams (Kendall τ = 0.2, p-value = 0.04).

Figure 18. Agreement values and final solution quality throughout the sessions.

In the context of the model, the contribution is defined as the number of times an agent proposed its solution to the other team members, which was important in knowing team behaviour. Low contribution distribution as shown in Figure 19 indicates that more or less all agents equally proposed solutions in the team. On the other hand, a high value of contribution distribution indicates that only some agents often proposed solutions as seen from all influencers and three influencers team compositions (Figure 19). The values of the distribution of agents’ contribution in various team compositions differ significantly (Kruskal–Wallis H = 27.06, p < 0.001). However, this significant difference in the contribution distribution values is due to all and three influencers team composition (Figure 20). There was no significant difference for teams of one and two influencers, three influencer teams with no influencers in the distribution of the contribution by their agents. However, within the no well-defined influencer setting, all and no influencer teams behave significantly different from each other in the contribution by their agents.

Figure 19. Contribution distribution and average quality of the final solutions.

Figure 20. Post hoc Conover’s test used after Kruskal–Wallis to do pairwise comparisons on the contribution distribution values.