Pools of machines

The wisdom of groups has been much studied. The jury system for legal decision-making is one practical example of this notion. However, why can a group be wise, or more precisely, when can a group act wisely and when might it not, is the real issue for us. Thus, theoretical studies lead to so-called ensemble methods, or committee decision rules, as often provably better than single classifiers. This research reveals not only the wisdom of group decisions, under certain carefully noted circumstances, but much more: groups can act quite wisely even when individuals in the group are not especially wise at all. That basic classifiers which are not particularly good when acting alone, can when acting jointly become quite good, is not at all intuitive. But this insight leads to an important class of decision-making tools, and we discuss the wisdom of ensembles in this chapter.

Weak correlation with outcome can be good enough

In Chapter 8 we introduced the idea that predictors that are only weakly correlated with the outcome can, acting jointly, be quite strong. This certainly seems counter-intuitive, but has been known for some time, since at least 1785; see Note 1.