Book contents
- Frontmatter
- Contents
- List of Panels
- Preface
- PART I A LONG-PONDERED OUTFIT
- PART II THE EVALUATION DISCORDANCE
- PART III THE ALGORITHMIC CONFLUENCE
- PART IV THE SOCIETY OF MINDS
- 12 Cognitive Development and Potential
- 13 Identifying Social Skills
- 14 Communication Abilities
- 15 Evaluating Collective and Hybrid Systems
- PART V THE KINGDOM OF ENDS
- References
- Index
- Plate section
15 - Evaluating Collective and Hybrid Systems
from PART IV - THE SOCIETY OF MINDS
Published online by Cambridge University Press: 19 January 2017
- Frontmatter
- Contents
- List of Panels
- Preface
- PART I A LONG-PONDERED OUTFIT
- PART II THE EVALUATION DISCORDANCE
- PART III THE ALGORITHMIC CONFLUENCE
- PART IV THE SOCIETY OF MINDS
- 12 Cognitive Development and Potential
- 13 Identifying Social Skills
- 14 Communication Abilities
- 15 Evaluating Collective and Hybrid Systems
- PART V THE KINGDOM OF ENDS
- References
- Index
- Plate section
Summary
In one or two generations, kids are practically born with their brains connected.
– Sara Seager, in Is the Universe a Hologram? And Other Conversations with Leading Scientists (Plasencia, 2017)WHEN SEVERAL SYSTEMS are governed, momentarily or permanently, by common goals, they can work as a team. From the outside, if the team has a unified interface, its evaluation should, in principle, be the same as that of an individual. However, if the members of the team can perform different actions and have different observations, the task setting changes slightly and so must change the evaluation. In both cases, what can we say when we compare the group of individual policies versus the joint policy? How does the performance of the group depend on the behavioural features of its members? We will see that the members’ features and the way the team is arranged and communicates are key, but we will also see that diversity, mostly in terms of the psychometric profile of the group, might be equally important. The general open question, which extends to hybrids and large collaborative platforms, is what psychometric profiles must be selected for mustering a team that meets an overall desired psychometric profile.
CHARACTERISING COLLECTIVE TASKS
In Chapter 13 we analysed the way in which an agent can be analysed, and measured, in a social context. One natural setting for evaluation is a situated test, a multi-agent system, where several agents interact. In particular, in Section 13.3 we saw that those agents sharing rewards (a common utility or payoff) were considered to be in a team, having an incentive to co-operate. Of course, having a shared reward or goal does not imply that the agents will co-operate. The motivation may be insufficient (e.g., in a large group agents can become lazy, since their effort may be unnoticeable), the task may make it impossible (e.g., agents are in isolated rooms) or the cognitive abilities of the group may be inappropriate for co-operation (e.g., lack of communication). For instance, in Figure 13.4 we saw that a single random agent can be very disruptive to a set of agents working in the same team.
- Type
- Chapter
- Information
- The Measure of All MindsEvaluating Natural and Artificial Intelligence, pp. 392 - 414Publisher: Cambridge University PressPrint publication year: 2017