DM and EUM rules

The DM rules can be naturally motivated by the necessity of achieving a seamless integration between our quantitative and qualitative belief states. When binary beliefs significantly deviate from the credal state, it would be difficult for an agent to organize their beliefs coherently and to guide their actions consistently. Thus, it is beneficial to facilitate a coherent linkage between credence and belief by seeking the most similar binary belief corresponding to credence. However, harmonious integration of different types of beliefs is not the sole goal of our epistemic life. So, let us motivate DM rules with respect to the truth-tracking norm.

Binary beliefs aim at the truth. Thus, our objective is to provide belief binarization rules that track the truth well—the closer to the truth, the better the rule. However, perfectly rational agents in our framework might lack the knowledge of the actual truth; they possess only subjective probability functions, which are rationally permissible credal states for truth tracking (Joyce Reference Joyce1998) and aim at the truth as well. Given that probabilistic beliefs are more fine-grained and more informative than binary beliefs, rational belief binarization methods should strive to identify binary beliefs as close to the probabilistic beliefs in question as possible. In this sense, this method tracks the truth by tracking probabilistic beliefs and thus belongs to a sort of implicit method adhering to the aiming-at-truth norm.

We can also conceive of belief binarization methods that explicitly consider the truth-tracking norm for beliefs. As our agents lack access to the truth but only to subjective probabilities that encapsulate all internally accumulated evidence about the truth, the agent, at best, can expect her binary beliefs to be accurate based upon her credal state. Arguably, this method can be viewed as a sort of explicit method to track the truth. In epistemic decision theory, minimizing the expected inaccuracy of credence has been employed to justify epistemic norms (Greaves and Wallace Reference Greaves and Wallace2006; Oddie Reference Oddie1997). We apply the minimization of the expected inaccuracy of belief, as decision rules, directly to the belief binarization problem (Levi Reference Levi1967). In conclusion, to get “close” to the truth, the best approaches for an agent with a probability function to determine what to believe would involve either (i) the ways to get “close” to the probability function, or (ii) those to expectedly get “close” to the truth. The former is the DM rules, and the latter is the EUM rules.

Distance between binary beliefs and credences

The DM rules determine binary beliefs that minimize the distance from a given probability function $P$ . To measure the distance from a probability function to a belief set, we will utilize divergences between probability functions. However, a belief set is not a probability function. Our key concept involves linking a belief set $Bel$ with a probability function: the uniform distribution $U\left( B \right)$ over the belief core $B$ of $Bel$ (the conjunction of all believed propositions), which is guaranteed to be believed by the assumption of deductive closure. Building on this, we can assess the discrepancy between a probability function and a belief set $Bel$ by measuring the divergence between the probability function and $U\left( B \right)$ . Accordingly, the distance from $U\left( B \right)$ to $Bel$ becomes zero, and the DM rules map $U\left( B \right)$ to $Bel$ . This does not mean that $U\left( B \right)$ is the probabilistic representation of a rational belief set $Bel$ . Following the process of belief binarization, we might generate possibly infinitely many probability functions corresponding to specific binary beliefs $Bel$ . The probability function $U\left( B \right)$ can be interpreted just as the most representative credal state corresponding to $Bel$ .

To justify our methodology, we need to explain why $U\left( B \right)$ is the most natural probability function generating the set $Bel$ of all the supersets of $B$ as the belief set. We suggest an epistemic principle, termed the suspension principle, which posits that belief binarization rules ought to map $U\left( B \right)$ to $Bel$ . Let’s delve into the plausibility of this principle. In cases where $B$ is the singleton set $\left\{ w \right\}$ of a world $w$ , $U\left( B \right)$ represents the probabilistically certain belief that $w$ is the actual world. Thus, the resulting belief set $Bel$ should indeed be the set of all supersets of $\left\{ w \right\}$ . When $B$ is not a singleton, $U\left( B \right)$ represents the probabilistically certain belief that any world lying outside of $B$ is not the actual world. Therefore, we can exclude those worlds. Moreover, since $U\left( B \right)$ is uniform over $B$ , it represents that the agent lacks an opinion regarding which world in $B$ is the actual one. Thus, the agent should suspend judgment about whether the actual world resides in any strict subset of $B$ , and thereby $B$ should be the belief core. And since $B$ has a probability of $1$ , all supersets of $B$ should be believed. By contrast, since probability functions other than $U\left( B \right)$ break the symmetry—regardless of how slight a perturbation may be—we cannot use this symmetry consideration for them.

Based on this framework, our characterization theorem of DM rules establishes that upholding the suspension principle amounts to employing our DM rules. It is worth noting that the suspension principle is not demanding and serves as a minimal requirement when permitting the suspension of judgments. This condition is lenient enough to embrace almost all threshold-based rules in the relevant literature. Nevertheless, our DM rules do not encompass every conceivable distance-based belief binarization approach, and there might be other ways to measure distance for belief binarization. But our characterization theorem of DM rules implies that if alternative distance-based rules satisfy the suspension principle, they are subsumed under our one. And the violation of the suspension principle demonstrates a significant drawback inherent in them. In fact, it can be easily checked that the Hamming rule presented by Dietrich and List (Reference Dietrich, List and Douven2021) violates the suspension principle.

DM(Bregman) and EUM(SP) rules

Now the problem is reduced to how one can measure the distance from a probability function to the uniform distribution on a belief core, i.e., a non-empty set of possible worlds. Many kinds of distance measures exist. Could there be some rationality norm to guide one in selecting distance measures? Our first main theorem will answer that question: if we employ a Bregman divergence for the DM rules, which will be called DM(Bregman), these rules can be represented by the expected utility maximization procedure, where the disutility is the Bregman divergence from the truth. If we agree that maximizing expected utility is a standard strategy to choose binary beliefs given a credal state, this theorem reveals a good reason why we should seriously consider distance minimization methods. ^{Footnote 1}

Let us elaborate further on this. Within our framework, the disparity between a world $w$ and a rational belief set $Bel$ can be assessed by measuring the divergence between the omniscient probability at $w$ and $U\left( B \right)$ . Furthermore, this divergence can be regarded as an epistemic inaccuracy of a belief set $Bel$ at a world $w$ . Since the most relevant feature of Bregman divergences concerning our study is that minimizing a Bregman divergence from $P$ is tantamount to minimizing the expected Bregman divergence from the true world in light of $P$ , any DM(Bregman) rule can also be viewed as an EUM rule. Additionally, the epistemic inaccuracy of a belief set at a world, as determined by a Bregman divergence, is a strictly proper score. So we now shift our focus to the EUM rules with strictly proper scores, called EUM(SP).

Strictly proper scores are prominently featured inaccuracy measures in epistemic decision theory and widely adopted as utilities in various other domains, such as probabilistic forecasting and belief elicitation. Strict propriety requires that the expected utility of a probability function according to $P$ is uniquely maximized at $P$ . Therefore, to attain maximal expected utility, we must opt for $P$ when it is provided, leading to the suspension principle in our framework. This observation implies that EUM(SP) rules can be regarded as DM rules. Moreover, building upon the fact that expected strictly proper scores generate Bregman divergences, our second main theorem demonstrates that EUM(SP) rules are DM(Bregman) rules.

Figure 1 depicts the relationships between the aforementioned rules. Our findings underscore that EUM(SP) rules, or equivalently DM(Bregman) rules, are the ways to get close to the probability and to expectedly get close to the truth at the same time. Both DM and EUM rules possess their own merits: the former can build a harmonious relationship between credence and belief by seeking the most similar binary beliefs to credence; the latter, on the other hand, guide in finding binary beliefs that approximate the truth. So, it would be beneficial to identify a class of rational belief binarization rules satisfying two principles simultaneously. Indeed, our representation theorems substantiate the existence of an extensive range of rational belief binarization rules that uphold these dual principles.

Figure 1. DM and EUM rules.

Generalization

We aim to develop our distance- and utility-based approaches within a comprehensive framework that can accommodate a wide range of scenarios. In the literature concerning belief binarization, probability functions are typically represented by points within a probability simplex. In our work, we not only utilize the probability simplex for representing probability functions, but we also incorporate De Finetti’s coherent polytope. In the former approach, each world’s probability (we will assume worlds to be finite) will be relevant, while the probabilities of some focused events, called an agenda, will be relevant in the latter approach. The latter approach exhibits greater generality than the former. If all the singleton sets of worlds constitute the agenda, then the latter approach turns into the former. This broadened perspective aligns our framework with the literature on epistemic decision theory or belief aggregation (Pettigrew Reference Pettigrew2015; Dietrich and List Reference Dietrich and List2017a ) and provides a new viewpoint for the belief binarization problem. In this framework, we can consider an agenda for the belief binarization problem. The agenda does not need to form an algebra. Thus, when we apply our DM rules in this framework, we do not need to have all the probabilities of propositions to determine what to believe. Even when we have the probabilities of only the propositions in the agenda, we can employ the DM rules, while other belief binarization theories in the literature cannot be applied in this case. Even if a probability function is given on an algebra, we may exclusively focus on certain basic propositions or premises. Then, our framework allows us to model a form of premise-based belief binarization, similar to the case of generalized probabilistic opinion pooling (Dietrich and List Reference Dietrich and List2017b ). Moreover, our methods can deal with disutility functions that depend solely on the probabilities of the focused events, although acquiring the probabilities associated with possible worlds is essential for calculating expected utility for the EUM rules.

Further challenges in this study are twofold. Firstly, we aim to refine conventional definitions of Bregman divergence. Specifically, we intend to adapt Bregman divergence to address belief binarization problems, allowing for infinite divergence within certain boundary regions of a probability simplex or De Finetti’s coherent polytope. Secondly, we do not impose the constraint of proposition-wise additivity on scoring rules. The aforementioned rules and claims will be elaborated and proved based on these technical settings. The rest of this paper elucidates these ideas with detailed mathematical explications.

Representations of probabilities

The remaining aspect of measuring distance involves addressing how to represent probability functions in ${\mathbb{R}^m}$ . Two approaches have been adopted in this regard. In certain contexts, such as belief binarization theories or statistics, probabilities of worlds are employed. In other contexts, like belief aggregation or epistemic decision theory, a set of propositions is initially given, with only their respective probabilities being pertinent. We incorporate both of these approaches in our discussion. Let $W = \left\{ {{w_1}, \ldots, {w_n}} \right\}$ . Our first approach is to represent a probability function $P$ by a point $p$ in ${\mathbb{R}^{\left| W \right|}}$ such that

$$p = \left( {{p_{{w_1}}}, \ldots, {p_{{w_n}}}} \right) = \left( {P\left( {{w_1}} \right), \ldots, P\left( {{w_n}} \right)} \right) \in {\Delta ^W},$$

where ${\Delta ^W}$ denotes the set of all points representing probability distributions. We say that $p$ is the representation point of $P$ in ${\Delta ^W}$ . According to this representation method, we can represent an omniscient credence function ${V_w}$ at $w \in W$ —assigning $1$ to $w$ —by a point ${v_w}$ on ${\{ 0,1\} ^{\left| W \right|}}$ where the $w{\rm{'}}$ th coordinate is given by ${({v_w})_{w{\rm{'}}}} = {V_w}\left( {w{\rm{'}}} \right) = 1$ if $w{\rm{'}} = w$ and ${({v_w})_{w{\rm{'}}}} = {V_w}\left( {w{\rm{'}}} \right) = 0$ otherwise. Thus, ${\Delta ^W}$ is the convex hull of the representation points of all omniscient credence functions because  $p = \mathop \sum \nolimits_{w \in W} P\left( w \right){v_w}$ . Note that ${\Delta ^W}$ is nothing but the usual $\left( {\left| W \right| - 1} \right)$ -dimensional probability simplex.

We now move to the second approach. We introduce a non-empty subset ${\mathscr F} = \left\{ {{A_1}, \ldots, {A_m}} \right\} \subseteq {\mathscr P}\left( W \right)$ , and call it the set of focused events or an agenda. Even though probabilistic and binary beliefs are functions from ${\mathscr P}\left( W \right)$ , there can be some situations where we are only interested in the focused events in ${\mathscr F}$ . In this case, we can represent probability functions in ${\mathbb{R}^{\left| {\mathscr F} \right|}}$ and measure distances between them in this space. Note that probabilistic and binary beliefs are assumed to be functions not from ${\mathscr F}$ but from ${\mathscr P}\left( W \right)$ even in this approach as well, which can be relaxed later. ^{Footnote 3} We represent a probability function $P$ by a point $p$ in ${\mathbb{R}^{\left| {\mathscr F} \right|}}$ such that

$$p = \left( {{p_{{A_1}}}, \ldots, {p_{{A_m}}}} \right) = \left( {P\left( {{A_1}} \right), \ldots, P\left( {{A_m}} \right)} \right) \in {\Delta ^{\mathscr F}},$$

where we denote by ${\Delta ^{\mathscr F}}$ the set of all points representing probability distributions. We say that $p$ is the representation point of $P$ in ${\Delta ^{\mathscr F}}$ . In this approach, an omniscient credence function ${V_w}$ at $w \in W$ is represented by a point ${v_w}$ on ${\{ 0,1\} ^{\left| {\mathscr F} \right|}}$ where the $A$ th coordinate ${({v_w})_A} = {V_w}\left( A \right)$ is $1$ if $w \in A$ and $0$ otherwise. Accordingly, ${\Delta ^{\mathscr F}}$ is the convex hull of $\{ {v_w} \in {\{ 0,1\} ^{\left| {\mathscr F} \right|}}|w \in W\} $ . Notice that ${\Delta ^{\mathscr F}}$ is a 0/1-polytope in ${\mathbb{R}^{\left| {\mathscr F} \right|}}$ (a polytope whose vertexes are on ${\{ 0,1\} ^{\left| {\mathscr F} \right|}}$ ).

It is interesting to compare the two approaches. In the case where ${\mathscr F} = \{ \left\{ w \right\} \in {\mathscr P}\left( W \right)|w \in W\} $ , both ways are the same. If $\left| {\mathscr F} \right| \lt \left| W \right|$ , one point in ${\Delta ^{\mathscr F}}$ represents several distinct probability distributions, i.e., a probability distribution is not uniquely determined by a point: a point in ${\Delta ^{\mathscr F}}$ represents a convex set of probability functions. This is because if $P,P{\rm{'}}$ have the same representation $p$ , then for all $A \in {\mathscr F}$ we have $P\left( A \right) = P{\rm{'}}\left( A \right) = \alpha P\left( A \right) + \left( {1 - \alpha } \right)P{\rm{'}}\left( A \right)$ for all $\alpha \in \left[ {0,1} \right]$ , which means that any linear combination of them has the same representation. Therefore, we can regard a point in ${\mathscr F}$ as a convex set of probability functions. Note that many definitions, theorems, and statements in this paper will be formulated using not only the representations in ${\Delta ^W}$ but also the ones in ${\Delta ^{\mathscr F}}$ . To express this, we will use ${\Delta ^M}$ . Hence $M$ will be considered to be $W$ or ${\mathscr F}$ throughout this paper.

Representations of belief cores

Now let us turn to the representations of the uniform distribution $U\left( B \right)$ on a non-empty belief core $B \in {\mathscr P}\left( W \right)\backslash \left\{ \emptyset \right\}$ . We will use the small letter $b$ to refer to the representation point of $U\left( B \right)$ in ${\Delta ^M}$ and call it the corresponding point of $B$ in ${\Delta ^M}$ . Moreover, we denote by ${U^M}$ the set of all corresponding points of non-empty belief cores, i.e., ${U^M} = \{ b \in {\Delta ^M}|\emptyset \ne B \subseteq W\} $ . With this in place, we will, hereafter, let a BR $G$ be a correspondence from $\mathbb{P}\left( W \right)$ to ${U^M}$ . In other words, $G\left( P \right)$ refers to a set of points $b$ in ${U^M}$ . Since we address binarization rules based on distance minimization or expected utility maximization, a BR $G$ will take the form of ${\rm{argmi}}{{\rm{n}}_b}\,g\left( {P,b} \right)$ , which is the set of all points $b$ in ${U^M}$ that minimize $g\left( {P,b} \right)$ , where $g$ is an extended real-valued function. Note that the uniform distribution on a belief core might not be uniquely determined by a point in ${\Delta ^{\mathscr F}}$ as explained above, which can lead to the under-determination of a belief core. However, even if a point that corresponds to several belief cores is selected, one could combine it with a tie-breaking rule. Moreover, the following lemma shows that even though different belief cores give us different belief sets, if their corresponding points in ${\Delta ^{\mathscr F}}$ are the same, then they yield the same result concerning any event in ${\mathscr F}$ .

The statement (i) gives us a geometrical intuition. If ${b_A} = 1$ , then $A$ is believed; if ${b_A} = 0$ , then ${A^{\rm{c}}}$ is believed; if ${b_A} \ne 0,1$ , then neither $A$ nor ${A^{\rm{c}}}$ is believed. This explains why binary beliefs on the focused events in ${\mathscr F}$ are invariant under the same output representation (IOR).

Let us now illustrate some examples of belief binarization problems. Consider the following two binarization problems: (1) $W = \left\{ {{w_1},{w_2},{w_3}} \right\}$ , and (2) ${\mathscr F} = \left\{ {\left[ {{a_1}\left], \ \right[{a_1} \wedge {a_2}} \right]} \right\}$ and $W = \left\{ {{w_1}\left( {| \!\!\!= {a_1},{a_2}} \right),{w_2}\left( {|\!\!\!= {a_1},\neg {a_2}} \right),{w_3}\left( {| \!\!\!= \neg {a_1},{a_2}} \right),{w_4}\left( {| \!\!\!= \neg {a_1},\neg {a_2}} \right)} \right\}$ , where ${a_1}$ and ${a_2}$ are atomic formulas in the standard propositional logic, and for any formula $\phi $ , $\left[ \phi \right]$ is the set of all valuations under which $\phi $ holds. We write $w| \!\!\!= {\phi _1},{\phi _2}$ when ${\phi _1}$ and ${\phi _2}$ hold under the valuation $w$ . Figure 2 depicts (1) and (2) in ${\Delta ^W}$ and ${\Delta ^{\mathscr F}}$ , respectively. Each point represents the uniform distribution on a belief core. In the right panel, some of them represent several uniform distributions. The solid circles express that there are extra uniform distributions. Note that the points that are surrounded by a dotted circle give us the same beliefs/disbeliefs of the focused events in ${\mathscr F}$ by Lemma.

Figure 2. Belief binarization problems.

DM rules and the suspension principle

Now we deploy a divergence $d$ in ${\Delta ^M}$ and the representation methods discussed above in order to formulate distance minimizing binarization rules (DM rules) as follows.

So a DM rule with a divergence $d$ in ${\Delta ^M}$ is a correspondence that takes as input any probability function $P$ and outputs the points $b$ in ${U^M}$ that minimize the distance from $p$ . The following theorem states that the DM rules are characterized by an epistemic principle, what we call the suspension principle. Here is the formal definition of the suspension principle.

This means that if $P$ is a uniform distribution on certain worlds, then binarization rules should result in the belief core that consists of those worlds. For the characterization theorem, we will need the following condition, which provides some control over the cases where points in ${\Delta ^M}$ can represent several probability distributions.

If $G$ satisfies IIR in ${\Delta ^{\mathscr F}}$ , then the binarization results depend only on the probabilities of the focused events in ${\mathscr F}$ . Thus, the representation point $p$ plays the role of the input of $G$ . This amounts to dealing with a generalized agenda, which does not need to be an algebra, and probabilistically coherent beliefs—a function extendable to a probability function on the algebra generated by the generalized agenda. Let us compare this with the invariance under the output representation in Lemma 1. IIR means that two probability functions with the same representation input point give us the same output point, which is associated with several belief cores. On the other hand, IOR in Lemma 1 shows that binary belief functions corresponding to an output point give us the same beliefs or disbeliefs about the focused events in ${\mathscr F}$ . The following theorem says that the suspension principle characterizes the DM rules if $G$ satisfies IIR.

We remark that DM rules in ${\Delta ^W}$ always satisfy IIR. Thus, a BR $G$ is a DM rule in ${\Delta ^W}$ iff $G$ satisfies the suspension principle. From this theorem, it can be easily checked that the threshold-based rules in Leitgeb (Reference Leitgeb2017), Lin and Kelly (Reference Lin and Kelly2012b), and Goodman and Salow (Reference Goodman and Salow2023) can also be seen as a DM rule in ${\Delta ^W}$ .

The most natural DM rule can be given by using the squared Euclidean distance ${D_{{\rm{SE}}}}$ in ${\Delta ^W}$ . ^{Footnote 4} We call this the DM(SE) rule in ${\Delta ^W}$ . Figure 3 illustrates DM(SE) for the case where $\left| W \right| = 3$ . The seven dots represent the uniform distributions $b$ of the seven belief cores. The dotted lines divide the simplex into seven regions. Each region is the preimage region ${G^{ - 1}}\left( b \right)$ $( = \{ p \in {\Delta ^W}|b \in G\left( P \right)\} )$ of the point $b$ under $G$ . For example, consider a probability distribution $P$ such that $P\left( {{w_1}} \right) = 0.8$ and $P\left( {{w_2}} \right) = 0.2$ , which can be represented by a point $p = \left( {0.8,0.2,0} \right)$ . According to DM(SE), $P$ should be assigned to the belief core $\left\{ {{w_1}} \right\}$ because the squared Euclidean distance between $p$ and $\left( {1,0,0} \right)$ is much less than the distances of $p$ to the other six corresponding points of belief cores. Additionally, it is also easily checked that if we use the inverse Kullback–Leibler divergence (IKL) ^{Footnote 5} in ${\Delta ^W}$ , the resulting belief binarization, called DM(IKL), is the same as the probability 1 proposal (we ought to believe a proposition $A$ iff $P\left( A \right) = 1$ ). In the following section, we will generalize the cases of DM(SE) and DM(IKL) to the DM rules with a Bregman divergence.

Figure 3. DM(SE) when $W = \left\{ {{w_1},{w_2},{w_3}} \right\}$ .

Refined Bregman divergence

First of all, we refine the definition of Bregman divergence to make it applicable to the belief binarization problems. In the typical definition of Bregman divergence in most of the literature, the domain of the second argument is an open set—e.g., the (relative) interior of a probability simplex ${\Delta ^W}$ or ${\mathbb{R}^m}$ —or the value of Bregman divergence cannot be infinity. For our purposes, however, we need to define it in a closed set ${\Delta ^M}$ , because many uniform distributions on belief cores are located on the boundary of the set. Furthermore, we should allow infinity as a possible value of divergence, because we want to embrace some asymptotically divergent distance measures like the inverse Kullback–Leibler divergence. For this reason, we need to extend the definition of Bregman divergence with infinity to the (relative) boundary of ${\Delta ^M}$ .

Considering that our definition includes infinity on the boundary, a comparison can be drawn with the works of Adamcik (Reference Adamcik2014) and Predd et al. (Reference Predd, Seiringer, Lieb, Osherson, Vincent Poor and Kulkarni2009). In Adamcik (Reference Adamcik2014), Bregman divergence is defined in ${\Delta ^W}$ and its value can be infinity. ^{Footnote 6} Our definition is more general because it embraces not only Bregman divergences in ${\Delta ^W}$ but also those in ${\Delta ^{\mathscr F}}$ . Turning to Predd et al. (Reference Predd, Seiringer, Lieb, Osherson, Vincent Poor and Kulkarni2009), Bregman divergence is defined in ${\Delta ^{\mathscr F}}$ and its value can be infinity. However, the additivity of Bregman divergence is presupposed. Our intention is to develop definitions and theorems in a more comprehensive manner without the assumption of additivity. Thus, our definitions and theorems work not only in a simplex ${\Delta ^W}$ but also in a general 0/1-polytope ${\Delta ^{\mathscr F}}$ dealing with not only additive but also non-additive divergence.

Let’s elaborate on the main desired features of our proposed definition. We aim to extend the definition of finite Bregman divergence in the interior of ${\Delta ^M}$ to encompass the boundary, even considering infinite values. However, our intent is not to have infinite distance along the entire boundary, but rather to carefully determine where finiteness and infiniteness should be applied. For instance, take two points on the boundary that reside within the relative interior of the same lower-dimensional face. In this scenario, we want the divergence between them to remain finite, just like the divergence between two points in the interior of the 0/1-polytope. Moreover, we want the divergence to be continuous in the region where it should be finite, just as the Bregman divergence is continuous in the interior of the 0/1-polytope. On top of that, we aim to retain key properties—e.g., the relation between distance minimization and expected score maximization—and well-known instances of Bregman divergence—e.g., the squared Euclidean distance and inverse Kullback–Leibler divergence. This enables one to use the existing results about Bregman divergences. In a nutshell, our definition is designed to ensure that the behavior of divergence on the boundary mirrors that in the interior. This is reasonable in the context of the belief binarization problem because we should maintain the same belief–credence connection principles even when narrowing our attention to a specific region on the boundary by assigning zero probability to some worlds or propositions.

To this end, we provide a way to denote faces not only in the simplex ${\Delta ^W}$ but also in the 0/1-polytope ${\Delta ^{\mathscr F}}$ . Within ${\Delta ^W}$ , we can employ ${\rm{Supp}}\left( Q \right)$ , defined as $\{ w \in W|Q\left( w \right) \ne 0\} $ , to signify the set of the worlds corresponding to all vertexes of the lowest-dimensional face on which $q$ $\left( { \in {\Delta ^W}} \right)$ lies. We need to extend this notion to indicate the faces of ${\Delta ^{\mathscr F}}$ , where a point in ${\Delta ^{\mathscr F}}$ can represent multiple probability functions.

So, $w \in {\rm{MSupp}}\left( q \right)$ means that there exists a probability function $Q$ represented by $q$ such that $Q\left( w \right) \ne 0$ —i.e., a probability represented by $q$ assigns to $w$ a non-zero probability (see figure 4). Using the notion of maximal support, we can designate the lowest-dimensional face ${\Delta _q}$ on which $q$ lies, which is the convex hull of the maximal support of $q$ . We can easily check that ${\Delta _q}$ is a (sub-)0/1-polytope that is the set of the points whose maximal support is a subset of $q$ ’s maximal support. In contrast, ${\mathbb{F}_p}$ is the set of the points whose maximal support is a superset of $p$ ’s maximal support (see figure 5).

Figure 4. In (a), $q$ is the representation of ${Q_1}$ , ${Q_2}$ , and ${Q_3}$ where ${\rm{Supp}}\left( {{Q_1}} \right) = \left\{ {{w_1},{w_2},{w_3}} \right\}$ , ${\rm{Supp}}\left( {{Q_2}} \right) = \left\{ {{w_1},{w_2},{w_4}} \right\}$ , and ${\rm{Supp}}\left( {{Q_3}} \right) = \left\{ {{w_1},{w_2},{w_3},{w_4}} \right\}$ . Thus, ${\rm{MSupp}}\left( q \right) = \left\{ {{w_1},{w_2},{w_3},{w_4}} \right\}$ . In (b), ${\rm{MSupp}}\left( p \right) = \left\{ {{w_1},{w_3}} \right\} \subseteq {\rm{MSupp}}\left( q \right) = \left\{ {{w_1},{w_2},{w_3},{w_4}} \right\}$ .

Figure 5. When ${\Delta ^M} = {[0,1]^3}$ , ${\Delta _q}$ is the thick gray line including the end points and ${\mathbb{F}_p}$ is the union of the relative interior ( ${(0,1)^3}$ ) of ${\Delta ^M}$ and the gray area excluding the dotted boundary.

Now we are ready to formulate our definition of Bregman divergence $D$ . We modify its typical definition to the extent that $D\left( {p,q} \right)$ is finite and continuous so far as $q \in {\mathbb{F}_p}$ , i.e., ${\rm{MSupp}}\left( p \right) \subseteq {\rm{MSupp}}\left( q \right)$ , which means that all worlds that are probabilistically possible according to some probability function represented by $p$ are also probabilistically possible according to some probability function represented by $q$ . Loosely speaking, we aim for Bregman divergences to remain finite as far as $q$ does not exclude any world that $p$ does not exclude.

This definition is compared with the conventional ones of Bregman divergence in the relative interior of ${\Delta ^M}$ , denoted by ${\rm{ri}}\left( {{\Delta ^M}} \right)$ , as follows. As usual, Bregman divergence is defined in terms of a convex function ${\rm{\Phi }}$ called a Bregman divergence generator. What distinguishes our definition from the conventional ones is parts (i) and (ii). In the conventional definitions, part (i) is applied in ${\rm{ri}}\left( {{\Delta ^M}} \right)$ , which is the whole domain in the conventional ones, and (ii) is not needed. By contrast, we apply part (i) to the region where $q \in {\mathbb{F}_p}$ , and extend this continuously, infinity being allowed as limits, to the rest of the domain.

Note that we use the directional derivative instead of the gradient in the conventional definition. This is because we need to define divergence not only in the interior but also on the boundary, where gradients are not well defined. In ${\rm{ri}}\left( {{\Delta ^M}} \right)$ , ${\nabla _{p - q}}{\rm{\Phi }}\left( q \right) = \nabla {\rm{\Phi }}\left( q \right) \cdot \left( {p - q} \right)$ because ${\rm{\Phi }}$ is differentiable from part (i) by the convexity of ${\rm{\Phi }}$ . ^{Footnote 7} In the interior of any lower-dimensional face ${\Delta _q}$ , we could also say that ${\rm{\Phi |ri}}\left( {{\Delta _q}} \right)$ (the restriction of ${\rm{\Phi }}$ to ${\rm{ri}}\left( {{\Delta _q}} \right)$ ) is differentiable in the sense that it is differentiable in the lower-dimensional space (note that the affine hull of ${\Delta _q} - q$ ( $: = \{ x - q|x \in {\Delta _q}\} $ ) is a subspace of ${\mathbb{R}^m}$ ). In this sense, we could conclude that our definition of $D\left( {p,q} \right)$ coincides with the conventional one not only for $p,q \in {\rm{ri}}\left( {{\Delta ^M}} \right)$ but also for $p,q$ in the relative interior ${\rm{ri}}\left( {{\Delta _q}} \right)$ of any lower-dimensional face.

Now let us consider another way to extend the conventional definitions of Bregman divergence to the boundary to compare this with our definition. Instead of parts (i) and (ii), we could have defined Bregman divergence as follows:

According to this definition, the divergence is finite and continuous in ${\rm{ri}}\left( {{\Delta ^M}} \right)$ , and it does not guarantee finite and continuous divergence between two points in the relative interior of a face on the boundary. Figure 6 shows the problem cases that could arise if we defined Bregman divergence according to (*). In these cases, the divergence $D\left( {p,q} \right)$ can be infinite although $q$ does not exclude any world that $p$ does not exclude. We will see later that if we do not prevent these cases, then we cannot prove the relation between Bregman divergences and proper scores in Theorem 2. For this reason, it is hoped that the region where Bregman divergence should be finite and continuous is extended from ${\rm{ri}}\left( {{\Delta ^M}} \right)$ to ${\mathbb{F}_p}$ .

Figure 6. These cases can occur according to the definition of (*). The dashed lines in each polytope represent where $D\left( {p, \cdot } \right)$ is infinite.

Continuous strictly proper score

Now we define a continuous strictly proper score in our setting where probability distributions are represented in ${\Delta ^M}$ . We include infinity as a value of scores on some boundary region, and when we talk about continuity of a score including infinity, we will also regulate the region where it should be finite and continuous.

where $p$ is the representation point of $P$ in ${\Delta ^M}$ .

According to our definition above, continuous scores $I\left( {w, \cdot } \right)$ are finite and continuous not only in ${\rm{ri}}\left( {{\Delta ^M}} \right)$ but also in ${\mathbb{F}_{{v_w}}}$ (see Figure 7). Notice that $q \in {\mathbb{F}_{{v_w}}}$ , i.e., $w \in {\rm{MSupp}}\left( q \right)$ , says that a probability function represented by $q$ assigns to $w$ a positive value. This means that $w$ is probabilistically possible from the point of view of a probability function represented by $q$ . In this case, we demand that $I\left( {w,q} \right)$ should not receive infinite score and be continuous in $q$ . Note that if $I$ is strictly proper, then we can derive that $I\left( {w, \cdot } \right)$ is finite in ${\mathbb{F}_{{v_w}}}$ , as shown in the following lemma.

Figure 7. The dashed lines including the end points in each polytope represented where a continuous score $I\left( {w, \cdot } \right)$ might be infinite. Their complement is ${\mathbb{F}_{{v_w}}}$ where $I\left( {w, \cdot } \right)$ is finite and continuous.

It is also worth noting that in ${\Delta ^W}$ the condition of $I\left( {w, \cdot } \right)$ being finite in ${\mathbb{F}_{{v_w}}}$ is the same as the notion of regular score in Gneiting and Raftery (Reference Gneiting and Raftery2007).

Invariant expectation under the same input representation

Notice that an expectation value in EUM rules depends not only on the point $p$ in ${\Delta ^M}$ but also on the probability distribution $P$ , in contrast to a divergence in DM rules. Thus, to see the connection between EUM rules and DM rules, we need the following requirement, which is relevant in ${\Delta ^{\mathscr F}}$ , where a point $p$ might represent several probability distributions.

IER has a close relationship with IIR in Definition 3. If a BR $G$ is an EUM rule with $I$ that satisfies IER, then $G$ is invariant under the same representation (IIR). Although IER may seem a strong condition, it is actually a mild restriction because a large class of scores obey IER. Every scoring function defined in ${\Delta ^W}$ satisfies IER. We can generalize this to the case in ${\Delta ^{\mathscr F}}$ as follows.

Note that if ${\mathscr F}$ includes every singleton set $\left\{ w \right\}$ of a world, for example, ${\mathscr P}\left( W \right)$ , then every $I$ is a partition-wise score, and thus it satisfies IER. There are also another ways to satisfy IER. Any additive scores defined in ${\Delta ^{\mathscr F}}$ also enjoy IER, as the following lemma shows. In addition, it shows that strict propriety of an additive score follows from event-wise strict propriety.

Lemma 4 Let $I:W \times {\Delta ^{\mathscr F}} \to \left[ {0,\infty } \right]$ be additive, i.e., for all $w \in W$ and $p \in {\Delta ^{\mathscr F}}$ ,

$$I\left( {w,p} \right) = \sum\limits_{A \in {\mathscr F}} {{I_A}(\left( {{v_w}{)_A},{p_A}} \right)} $$

where ${I_A}:\left\{ {0,1} \right\} \times \left[ {0,1\left] \to \right[0,\infty } \right]$ for all $A \in {\mathscr F}$ .

1. (i) $I$ satisfies IER.

2. (ii) If $I$ is event-wise strictly proper (E-SP), i.e.,

$$\mathop {\rm {argmin} }\limits_{{q_A} \in [0,1]} \left( {{p_A}{I_A}\left( {1,{q_A}} \right) + \left( {1 - {p_A}} \right){I_A}\left( {0,{q_A}} \right)} \right) = \left\{ {{p_A}} \right\}$$

for all $A \in {\mathscr F}$ and ${p_A} \in \left[ {0,1} \right]$ , then $I$ is strictly proper.