1.1. Proving the reliability of autonomous experts?

Dellsén imagines individual experts as functions that map hypotheses and bundles of evidence to the binary outcomes: accept or not. X _i(H, E) is the outcome that expert X _i accepts a hypothesis H in light of the evidence E. P(X _i(H, E)) is the probability that expert X _i will accept H in light of E. Dellsén equates degrees of epistemic autonomy among experts with degrees of probabilistic independence between X _i(H, E) for a group of experts X: = {X _i : i ∈ [1, n]}. He then compares two groups, $X^A: = \{ {X_i^A \, \colon \, i\in [ {1, \;n} ] } \}$ and $X^B: = \{ {X_i^B \, \colon \, i\in [ {1, \;n} ] } \}$, where $X_i^B ( {H, \;E} )$ are more (positively) dependent on each other than $X_i^A ( {H, \;E} )$. This means that:Footnote ⁹

(1)$$\displaystyle{{P\left( {\bigwedge_1^n X_i^A ( {H, \;E} ) } \right) } \over {\prod _1^n P( {X_i^A ( {H, \;E} ) } ) }} < \displaystyle{{P\left( {\bigwedge_1^n X_i^B ( {H, \;E} ) } \right) } \over {\prod _1^n P( {X_i^B ( {H, \;E} ) } ) }}$$

Dellsén then makes two assumptions. First, given H is true, the two groups are equally likely to agree that H is true. Second, the experts in each group can be matched pairwise so that the ith expert in each group will be equally likely to accept H in light of E. Stated formally:

$$P \big ({\mathop \bigwedge \limits_1^n X_i^A ( {H, \;E} ) \vert H} \big) = P\big ({\mathop \bigwedge \limits_1^n X_i^B ( {H, \;E} ) \vert H} \big)$$

$$P( {X_i^A ( {H, \;E} ) } ) = P( {X_i^B ( {H, \;E} ) } ) , \;\forall i\in [ {1, \;n} ] $$

Both of these assumptions are stated as equalising the prior distributions of expertise between the two groups. The aim is to isolate the effects of the different levels of dependence. (I will argue, below, that assumption 1 actually does much more than this and that it is the source of the problems for Dellsén's argument.)

Given these assumptions and (1), a straightforward application of Bayes's Theorem gives:Footnote ¹⁰

(4)$$P\big ({H\vert \mathop \bigwedge \limits_1^n X_i^A ( {H, \;E} ) } \big)> P \big ({H\vert \mathop \bigwedge \limits_1^n X_i^B ( {H, \;E} ) } \big )$$

Thus, the probability that H is true given that those in X ^A all accept H is higher than the probability that H is true given that those in X ^B all accept H. In words, a hypothesis is more likely to be true when a group of more epistemically autonomous experts agree that it is than when a group of less epistemically autonomous experts agree that it is.

1.2. An impossible task

If correct, the proof above offers a powerful result. By rearranging a few equations, we seem to have shown that groups of epistemically autonomous experts are, qua epistemically autonomous experts, more reliable.

But it is also a very surprising result. It seems odd that probabilisitic independence (which Dellsén treats as coextensive with epistemic autonomy) should entail greater reliability. After all, if two outcomes, a and b, are positively dependent, the only thing this tells us is that the probability of both occurring is greater than the product of the probabilities of each occurring [$P( {a\wedge b} ) > P( a ) P( b )$]. On its own, more or less dependence between a and b should have no bearing on a third outcome, c. For that to be the case, some further claims about how a and b relate to c would have to be made. Setting a and b to whether or not individuals accept a hypothesis and c to whether or not that hypothesis is true does not change this fact. Further claims about how the acceptances of the individuals involved relate to the truth of the hypothesis must be made for greater dependence between X _i(E, H) and X _j(E, H) to bear on P(H).

How does this square with the proof Dellsén offers, which is formally speaking correct? Dellsén's proof works because he introduces the required relation between X _i(E, H), X _j(E, H) and H in his first assumption, equal agreement on truth. But rather than giving a substantive reason to accept it, he suggests that it is only holding the prior distributions of expertise between the two groups even. This, however, is not correct.

1.3. An uneven comparison

Pairwise equal likelihood of acceptance holds the prior probabilities of each individual expert's acceptance of H equal between the two groups. This means that any difference in the probabilities that the groups collectively accept H are due to the differences in how the individuals are combined. Given the aim of comparing the impact of different ways of combining experts, pairwise equal likelihood of acceptance is thus a fair ceteris paribus assumption.

Equal agreement on truth is also offered as a ceteris paribus assumption. Dellsén's aim is to ensure that the only difference between the two groups is their relative dependence, and not differences in levels of expertise. As he puts it:

To add force to this point, Dellsén points out that one way in which equal agreement on truth could be violated is if one of the groups was more knowledgeable about the issues around H. Dellsén is trying to evaluate the impact of relative dependence between experts. If one group has more relevant expertise than the other, then that is going to stack the evaluation in their favour.

The problem is that equal agreement on truth does more than simply ensure that neither group has more relevant expertise for H than the other. If all it were doing was keeping levels of expertise fixed then the converse, equal agreement on falsity [$P\left( {\bigwedge_1^n X_i^A ( {H, \;E} ) \vert \neg H} \right) = P\left( {\bigwedge_1^n X_i^B ( {H, \;E} ) \vert \neg H} \right)$], should be similarly neutral. But assuming equal agreement on falsity would completely reverse the result.Footnote ¹¹ So what is actually going on?

Once pairwise equal likelihood of acceptance is assumed, more dependent groups will be more likely to agree in general. That is, given assumption 2, (1) implies:

(5)$$P\big ({\mathop \bigwedge\limits_1^n X_i^A ( {H, \;E} ) } \big )< P\big ({\mathop \bigwedge\limits_1^n X_i^B ( {H, \;E} ) } \big )$$

This means that it is not true that ‘if one of the group[s] is more likely to agree that H is true when H is indeed true, then this would already favor that group's consensus over the other group's as an indicator of the truth regarding H’. If one group were potentially more likely to agree when H is true but also potentially more likely to agree when H is false, as is the case for more dependent groups of experts, then them agreeing on H gives us no better guide as to whether H is true.

Moreover, assuming that the hypothesis in question can only be true or not true, then (5) can be split up into:

(6)$$\eqalign{& P\big ({\mathop \bigwedge \limits_1^n X_i^A ( {H, \;E} ) \wedge H} \big ) + P\big ({\mathop \bigwedge \limits_1^n X_i^A ( {H, \;E} ) \wedge \neg H} \big )\cr & < P\big ({\mathop \bigwedge \limits_1^n X_i^B ( {H, \;E} ) \wedge H} \big ) + P\big ({\mathop \bigwedge \limits_1^n X_i^B ( {H, \;E} ) \wedge \neg H} \big )} $$

In words, the higher probability that those in X ^B will agree on H in general could come from a higher probability of them agreeing when H is false and/or a higher probability of them agreeing when H is true. Nothing about the degree of independence in the probability of accepting H can tell us which way this will go. In Dellsén's result, however, equal agreement on truth assumes that dependent groups are no more likely to agree on H when it is true, meaning that they must be more likely to agree when H is false.Footnote ¹²

Equal agreement on truth does not just hold the levels of expertise around H even between both groups. Groups of dependent experts are more likely to agree. Adopting equal agreement on truth entails assuming that this can only apply to false hypotheses. Dellsén gives no justification for assuming this. Nor should such an assumption be made if the goal is to evenly compare the reliability of both groups.Footnote ¹³ To justify an assumption of this sort, a substantive claim about how more and less dependent groups reason would be required.Footnote ¹⁴

Thus, although Dellsén's conclusion seems to follow from the nature of probabilistic independence, the work is being done by an assumption that is insufficiently justified. This offers a cautionary tale for purely formal epistemological arguments – one with significance because Dellsén's result has gained traction.Footnote ¹⁵ I will argue that autonomy between experts is valuable. My argument differs from Dellsén's purely formal approach, however, by stating clearly the mechanism that makes autonomy valuable.

2.1. Autonomy and independence

In the definition of expert epistemic autonomy, above, the word ‘direct’ is key. Recall (emphasis added):

An expert is autonomous when they do not accept H simply because another tells them that they should. It is possible, however, for autonomous experts to be indirectly influenced by one another. By explaining her reasoning or ways of combining the available evidence, the jth expert, Jill, may convince the ith expert, Iris, to accept H without undercutting Iris's epistemic autonomy. As long as Iris utilises her own critical capacities to analyse what Jill tells her before accepting H, and does not simply accept H because Jill does, then she does not count as being directly influenced by Jill.

The distinction between direct and indirect influence is important as complete isolation between experts does not seem desirable or even possible. Experts within groups will very often have existing connections to one another and interaction between them may help each clarify and refine their own views.Footnote ¹⁶ This distinction disappears, however, when Dellsén moves to probabilistic independence. If Jill can influence Iris's acceptance of H then X _Iris(H, E) and X _Jill(H, E) will be dependent, even if the influence Jill might have over Iris is only indirect.Footnote ¹⁷ There are even situations in which greater dependence between X _Iris(H, E) and X _Jill(H, E) can lead to greater reliability. In a group that covers a wide range of relevant expertise, some dependence in the probabilities that the experts will accept hypotheses may aid the group to reliably agree on true hypotheses by triangulating their different perspectives. Iris and Jill may be able to put their perspectives together to rule out alternative hypotheses and consequently both agree on H, when independently neither could rule out enough alternatives. This can be the case without any loss of epistemic autonomy. Jill may be able to explain why an alternative hypothesis cannot be true using knowledge Iris already has but via an argument that she had not thought of (and vice versa).

Thus, it is not true that greater epistemic autonomy is coextensive with greater probabilistic independence. Moreover, in some cases, it is possible for greater probabilistic dependence to lead to greater reliability. Given this, I will stick to Dellsén's original definition of epistemic autonomy and not talk in terms of probabilistic independence.

2.2. Critical triangulation

In discussing the intuition behind his defence of epistemic autonomy, Dellsén draws on the nineteenth-century English natural philosopher William Whewell's idea of consilience: if a theory can be supported by evidence from many different directions (‘different classes of facts’, Reference Whewell1858: 88), then that offers a reason to infer the truth of that theory. If Whewell is correct, then something similar should apply to testimonial evidence. Testimony in favour of a hypothesis from autonomous (or more autonomous) sources of expertise should be taken to provide a strong (or stronger) reason to believe that hypothesis. This is an interesting suggestion and I will show how consilience can be used to highlight the value of expert autonomy. The problem with Dellsén's proof is that it missed a key part of Whewell's idea: difference.

Whewell's suggestion is that taking evidence from different directions gives more reason to believe a theory than taking evidence from one direction. To apply this to the case of epistemic autonomy, testimony from (more) autonomous experts must be analogous with evidence from (more) different directions. The first thing this requires is that the experts involved are different in some relevant sense. Ten identical experts deciding autonomously from one another do not fit the logic of consilience. Their autonomous testimony would be more akin to checking the same evidence ten times, rather than gathering evidence from ten different directions.Footnote ¹⁸

Taking this on board, imagine again two groups of experts that decide whether to accept a hypothesis given a bundle of evidence. For consistency, retain Dellsén's labels. Call the groups $X^A : = \{ {X_i^A \,\colon\, i\in [ {1, \;n} ] } \}$ and $X^B: = \{ {X_i^B \, \colon \, i\in [ {1, \;n} ] } \}$, the hypothesis H and the bundle of evidence E. Imagine also that the experts in X ^A are more epistemically autonomous than those in X ^B.

To add the difference that drives consilience, assume that the experts within both groups differ from one another in the skills and background knowledge they use to assess H (the skills and knowledge of $X_i^A$ differ from $X_j^A$ and so on). Adding such an assumption is not particularly costly. It is clear that experts do often possess different collections of skills and knowledge. Most of those that discuss expert testimony assume as much and assume that this variation is compatible with genuine expertise.

Because their skills and knowledge differ, the experts within both groups offer different perspectives on E and H. It is these different perspectives that seem analogous to Whewell's evidence from different directions. A group that can make better use of the range of perspectives of its members draws on a wider set of interpretations of the available evidence. The logic of consilience then suggests that we have greater reason to believe any conclusions such a group agree on.

How might epistemic autonomy play a role in this? Imagine our simple case with two experts, Iris and Jill, again. What is at issue when Iris accepts H simply because Jill asserts that she should? One issue seems to be that in accepting H on Jill's say so, Iris's own skills and knowledge are bypassed. She no longer thinks with her own mind but instead relies on Jill's.Footnote ¹⁹ If Iris and Jill are truly experts on H (as assumed) and they differ in the skills and knowledge they bring to the assessment of H, then there may be reasons against H that are accessible to Iris but not Jill. These reasons are ignored when Iris's skills and knowledge are bypassed. And H is, consequently, scrutinised from less perspectives than it might have been.

In more general terms, any group of experts that bypasses the skills and knowledge of some of its members utilises less expert perspectives than it might have done. Such groups use a narrower range of skills and knowledge in assessing H, and so consider fewer critical viewpoints on H than they might have.Footnote ²⁰ Such groups will consequently be more likely to incorrectly accept H when it is false because they are less likely to notice an error or poor judgment in the reasoning from E to H. We can summarise this idea as the following mechanism:

If the mechanism of critical triangulation is correct, it gives us a reason to value epistemic autonomy between experts.Footnote ²¹ More specifically, when (a) the experts within X ^A and X ^B differ in the skills and knowledge they possess, (b) both groups are made up of experts with the same distribution of skills and knowledge, and (c) the experts in X ^A are more epistemically autonomous than those in X ^B, critical triangulation gives us reason to believe that those in X ^B will be more likely to accept H in error.Footnote ²² This is because less epistemically autonomous groups utilise a narrower range of the perspectives available to them than more epistemically autonomous groups. And, given two groups that contain equivalent perspectives, the group that utilises less of those perspectives will develop less critical viewpoints on H, and so, via critical triangulation, will be more likely to accept H in error.

2.3. Technical aside

In the language of probability, critical triangulation gives us reason to believe that the combined probability of groups of less autonomous experts agreeing on H and H being false is higher than the equivalent probability for more autonomous groups. That is, given critical triangulation and (a–c):

(7)$$P\big ({\mathop \bigwedge\limits_1^n X_i^A ( {H, \;E} ) \wedge \neg H} \big )< P \big ({\mathop \bigwedge\limits_1^n X_i^B ( {H, \;E} ) \wedge \neg H} \big )$$

This is not equivalent to saying that the probability that H is true given that more epistemically autonomous groups of experts agree that it is is greater than for less epistemically autonomous groups – i.e. (4). I suggest that critical triangulation and (7) give us reason to value epistemic autonomy between experts. But, if the less autonomous group of experts (X ^B) are more likely to agree on H in general – this seems likely because both groups have the same spread of expertise but experts in the less autonomous group rely on one another more – then (7) is weaker than (4).Footnote ²³ How much does this weaken the conclusion that epistemic autonomy between experts is valuable?

The first thing to note is that (7) leads to (4) under certain conditions. If more autonomy between experts decreases the chances of false agreement at a greater rate than it decreases the chances of agreement, then (4) follows.Footnote ²⁴ Alternatively, (4) also follows if the difference between the probability that the more autonomous experts will agree and the probability that they will falsely agree is larger than or equal to the same difference for the less autonomous group.Footnote ²⁵ Both of these conditions will be satisfied if the extra perspectives that are available in X ^A – due to the fact that the experts think more for themselves – decrease false agreement more than they decrease agreement. This could be justified by assuming a veritistic understanding of expertise – entailing that the perspectives the experts in both groups bring track truth better than average (Goldman Reference Goldman2001). Thus, although any extra expert perspectives are likely to decrease the chances of agreement, those perspectives should hopefully decrease the chance of agreement when H is false even more.

Given that (4) can be derived from (7), why do I not make the stronger claim that hypotheses are more likely to be true when more autonomous groups of experts agree on them, than when less autonomous experts agree on them? The reason why I conclude with (7) rather than adding ‘and under most conditions this also leads to (4)’ is that the idea within (7) is simpler and easier to work with. Less autonomous groups are more likely to agree on falsehoods. That is enough of a reason to value autonomy between experts. The reader may disagree on this point. In which case, the more detailed reasoning above suggests that under plausible conditions (4) also follows. It is just that that requires more complicated checks, which would be impractical in reality. One of the virtues of the argument in 2.2 is that it clarifies, in a simple manner, the key value of epistemic autonomy: that it lowers the chances of false agreement via critical triangulation.

2.4. The value of autonomy between experts

Critical triangulation offers a reason to believe that groups of more epistemically autonomous experts are less likely to reach consensus in error. This is a conclusion similar to the one that Dellsén aimed at but failed to show. It also clarifies what it is about epistemic autonomy that is valuable: it reduces the risk of incorrect consensus by enabling the full difference within expert groups.

This conclusion does not follow from the logical features of epistemic autonomy alone. Rather, it relies on the claim that critical triangulation happens. I have given reasons to believe that critical triangulation is a factor in group reasoning. But there is no guarantee that these reasons correctly identify what happens in all cases. There may also be other mechanisms at play that pull in the opposite direction. This should not, however, undermine the value of my point. It should not be surprising that a consequential conclusion – that epistemic autonomy between experts is valuable – should rest on a substantive claim. It is a virtue of my argument that I clarify the substantive claim that underpins the argument.

On my telling, the value of epistemic autonomy rests in ensuring that all the viewpoints in a group are heard. In addition to clarifying its value, this highlights a limitation of epistemic autonomy between experts. No matter how autonomously two experts decide, if they are trained in the same skills, at the same institution and under the same people, the perspectives they offer are likely to be broadly similar. If the power of epistemic autonomy lies in enabling difference, then the significance of its effects depends on how different the experts in a group are. This opens the door to a second interpretation of independence between experts: as offering different perspectives.