Communication by global elites about the strengths and weaknesses of IOs is a common feature of global politics.Footnote 1 Member governments, NGOs, and IOs themselves regularly criticize and endorse the operations and efforts of international organizations. As the US announced its withdrawal from the UN Human Rights Council in June 2019, then Secretary of State Mike Pompeo proclaimed: “The Human Rights Council has become an exercise in shameless hypocrisy, with many of the world’s worst human-rights abuses going ignored and some of the world’s most serious offenders sitting on the council itself” (NPR 2019). As the WHO was challenged by former US President Donald Trump in April 2020, Bill Gates of the Bill & Melinda Gates Foundation tweeted: “Halting funding for the World Health Organization during a world health crisis is as dangerous as it sounds. Their work is slowing the spread of COVID-19 and if that work is stopped no other organization can replace them. The world needs @WHO now more than ever” (Gates Reference Gates2020). And as the IMF faced criticism from a variety of sources in 2019, then IMF Managing Director Christine Lagarde shot back: “I like to say that the IMF brings a wallet, a brain and a heart” (Lagarde Reference Lagarde2019).
Yet, despite the prominence of communication by global elites, we know little about its effects on the popular legitimacy of IOs. A growing body of research explores legitimation and delegitimation by such elites (Zaum Reference Zaum2013; Morse and Keohane Reference Morse and Keohane2014; Binder and Heupel Reference Binder and Heupel2015; Gronau and Schmidtke Reference Gronau and Schmidtke2016; Zürn Reference Zürn2018; Dingwerth et al. Reference Dingwerth, Witt, Lehmann, Reichel and Weise2019; Schmidtke Reference Schmidtke2019; Stephen and Zürn Reference Stephen and Zürn2019; von Billerbeck Reference Von Billerbeck2020; Bexell et al. Reference Bexell, Jönsson and Uhlin2022). However, this literature exclusively maps and explains patterns in the contestation around IOs, without assessing its consequences. Part of the reason is the methodological challenge of isolating the effects of elite communication.
In this chapter, we comparatively study how communication by global elites affects the popular legitimacy of IOs. We use the term “global elites” pragmatically to refer to elite actors who seek to influence politics beyond the nation-state, at global and regional levels. Building on our theory, we develop hypotheses about three conditions that should matter for the effects of global elite communication: the communicating elites (member governments, NGOs or IOs themselves), the IO features invoked in communication (procedures or performances), and the tone of messages (positive or negative).
We test our hypotheses through a population-based survey experiment. In other words, we embed an experiment designed to isolate causal effects of elite communication on citizen legitimacy beliefs toward IOs in a survey. As the survey is sent to nationally representative samples of citizens, we can generalize the experimental findings to the populations of the countries studied. We conducted the experiment in a survey with almost 10,000 respondents representing the populations of three countries: Germany, the UK, and the US. Using “vignettes,” that is, story lines in which we present specific framings of IOs to respondents, we vary the three factors we are interested in: the elite, the IO feature, and the tone. This approach enables us to examine causal effects on legitimacy perceptions comparatively across five prominent regional or global IOs: the EU, IMF, NAFTA, UN, and WTO. Our analysis goes beyond hypothesis testing, as we also explore the contingency of treatment effects among specific citizen subgroups, as well as IO and country contexts.
Our findings underline that communication by global elites matters for legitimacy perceptions. There are five key results. First, we find that communication by member governments and NGOs has stronger effects on legitimacy perceptions than communication by IOs themselves. This suggests that IOs’ increasingly prominent practice of self-legitimation confronts credibility constraints that reduce its effectiveness. Second, the evidence shows that elite communication affects legitimacy perceptions irrespective of whether it invokes IOs’ procedures or performances. While scholars for long have debated the relative importance of procedure and performance for legitimacy, citizens appear equally sensitive to both. Third, we find that negative messages about IOs have stronger effects on legitimacy perceptions than positive messages. This suggests that the opponents of global governance face an easier task than its defenders in shaping public perceptions. Fourth, comparing across IOs indicates that elite communication is more often effective in relation to the IMF, UN, and WTO, than the EU and NAFTA. This finding highlights the benefits of a comparative perspective, since a focus on single IOs may lead to an under- or overestimation of the general capacity of global elites to shape perceptions of IO legitimacy. Fifth, comparing results across countries shows that communication by these global elites is more effective in Germany and the UK than in the US. It may be that US citizens are relatively more susceptible to communication by domestic elites, as we discuss in Chapter 5.
This chapter proceeds in four parts. It begins by outlining our theoretical expectations about how communication by global elites is expected to shape legitimacy beliefs toward IOs. It then elaborates on the survey experimental design and presents the empirical analysis. We end the chapter by summarizing its main conclusions.
Hypotheses
We build on our theory (Chapter 3) to develop expectations about the conditions shaping the extent to which communication by global elites affects citizens’ legitimacy beliefs toward IOs. We assume that communication by trusted elites may offer citizens cognitive shortcuts to opinions about IOs and that citizens may be responsive to such information because it allows them to form opinions about IOs in efficient ways. Our expectations recognize that contextual circumstances of elite communication vary across scales of government. While research on elite communication in comparative politics has focused on political candidates and parties, global governance involves a different set of state and nonstate elites, raising novel questions about the effects of communication under alternative conditions. Specifically, we theorize that the strength of communication effects will depend on the type of global elite engaging in communication about IOs, the institutional features of IOs invoked in the communication, and the tone of the communication.
With regard to the elites engaging in communication, one of the hallmarks of global governance is the multitude of actors that aspire to influence its outcomes. We focus on three types of global elites who are common communicators about IOs: member governments, NGOs, and IOs themselves (see Chapters 2 and 3). There are reasons to expect that these elites are perceived as varyingly credible when communicating about IOs, with implications for the effects on citizens’ legitimacy perceptions. Building on research on expert endorsements and media priming, we expect credibility to be tied to perceptions of impartiality (Miller and Krosnick Reference Miller and Krosnick2000; Druckman Reference Druckman2001; Guisinger and Saunders Reference Guisinger and Saunders2017; Maliniak et al. Reference Maliniak, Parajon, Peterson and Powers2019). Specifically, we expect that citizens consider whether elites in global governance can be expected to hold and reveal accurate information about IOs. Elites that have larger incentives to convey biased information about IOs are less likely to be seen as credible sources. Conversely, elites that stand less to gain from how IOs are perceived can be expected to communicate more honestly about these organizations.
Based on these considerations, we expect NGOs to hold the highest credibility in the eyes of citizens. NGOs are constitutively independent from IOs and therefore more likely to be regarded as autonomous voices (cf. Gourevitch et al. Reference Gourevitch, Lake and Stein2012). Many NGOs, such as Human Rights Watch and Transparency International, have made it their organizational purpose to offer independent assessments of norm conformance and goal achievement among IOs and their member states (Kelley and Simmons Reference Kelley and Simmons2015). Member governments are the principals of IOs (Hawkins et al. Reference Hawkins, Lake, Nielson and Tierney2006). They have played a part in creating IOs, serve on their governing bodies, and carry the main responsibility for implementing their policies. As a result, governments often have particular views on how cooperation should develop, as in conflicts over burden-sharing in the EU, voting weight in the IMF, and dispute settlement in the WTO. Because of these stakes, member governments are likely perceived as less credible communicators than NGOs. IOs, finally, have the most far-reaching vested interests in debates about themselves. The bureaucracies of IOs are committed to advancing the goals of these organizations, but depend on the support of their political environment to achieve them (Barnett and Finnemore Reference Barnett and Finnemore2004). We therefore expect IOs to be the least credible source of information about themselves.
We find support for this gradation of credibility in data on popular confidence in different elites in a global sample of countries. Data from the WVS6 in the years shortly before our survey was fielded (2010–2014) can shed further light on this issue. Using data from fifty-two countries around the world, we assess differences in average citizen confidence in NGOs (proxied by the available indicators “environmental organizations” and “women’s organizations”), governments, and the UN. Using paired t-tests, we find that environmental and women’s NGOs are perceived as more credible than both governments and IOs, and that governments are perceived as more credible than the UN (see Online Appendix D).Footnote 2 This leads to a first hypothesis:
H1: The more credible elites are perceived to be by citizens, the stronger their impact on citizens’ perceptions of IO legitimacy.
Next, we turn to the institutional features of IOs invoked in communication by global elites. We assume that global elites typically attempt to affect individual attitudes by invoking two alternative grounds for endorsement or criticism: the procedures and performances of IOs. An extensive literature shows that favorable attitudes toward a political institution may be shaped by the procedures and performances of that institution – both in the context of domestic politics (Newton and Norris Reference Newton, Norris, Pharr and Putnam2000; Esaiasson et al. Reference Esaiasson, Gilljam and Persson2012, Reference Esaiasson, Persson, Gilljam and Lindholm2019) and global governance (Bernauer and Gampfer Reference Bernauer and Gampfer2013; Dellmuth and Tallberg Reference Dellmuth and Tallberg2015; Bernauer et al. Reference Bernauer, Mohrenberg and Koubi2020). As a consequence, elites interested in shaping the attitudes of citizens toward IOs frequently refer to these features. Procedural standards invoked in elite communication often relate to democratic aspects of IO policy-making, such as inclusiveness and accountability, but can also pertain to expertise and efficiency (Binder and Heupel Reference Binder and Heupel2015). Performance standards include aspects of goal achievement, such as problem-solving effectiveness and collective welfare gains, but can also relate to the fairness of outcomes (Zürn Reference Zürn2018).
Whether citizens’ legitimacy perceptions are most sensitive to the procedures or performances of IOs is a topic of debate. Procedural accounts submit that process criteria are most important for people’s perceptions of legitimacy. Even when institutions generate outcomes to their disadvantage, actors accept their exercise of authority because of how they were set up and operate (Hurd Reference Hurd2007, 71). Procedural accounts have an antecedent in Weber’s (Reference Weber1922/1978) notion of rational-legal legitimacy, emphasizing properly administered rules by properly appointed authorities. In contemporary scholarship, the idea that legitimacy results from features of the decision-making process is prominent in theories of procedural fairness (Tyler Reference Tyler1990) and democratic legitimacy (Held and Koenig-Archibugi Reference Held and Koenig-Archibugi2005).
Performance accounts instead claim that legitimacy perceptions are determined by institutions’ contributions to collective welfare and distributional outcomes. Substantive outcomes are considered more powerful in shaping the perceptions of institutions than the process by which those outcomes were produced (Hurd Reference Hurd2007, 67). This idea features prominently in the study of domestic institutions: “Government institutions that perform well are likely to elicit the confidence of citizens; those that perform badly or ineffectively generate feelings of distrust and low confidence” (Newton and Norris Reference Newton, Norris, Pharr and Putnam2000, 61). In the global setting, it is a common claim that IOs historically have earned their legitimacy through the benefits they have produced for states and societies (Buchanan and Keohane Reference Keohane, Newman, Thakur and Tirman2006).
Our theory gives us no reason to expect that elite communication would be varyingly effective depending on the features of IOs that are invoked. If it is correct that citizens care both about the procedures and performances of IOs when forming legitimacy perceptions (Anderson et al. Reference Anderson, Bernauer and Kachi2019; Dellmuth et al. Reference Dellmuth, Scholte and Tallberg2019; Bernauer et al. Reference Bernauer, Mohrenberg and Koubi2020), then elite communication that invokes these features should be effective in both cases. This leads us to expect:
H2: Elite communication affects citizens’ perceptions of IO legitimacy irrespective of whether it refers to the procedures or performances of IOs.
Finally, we consider how the tone of elite communication may influence the strength of cueing effects. As previously described, elite communication spans the full evaluative spectrum, from endorsing, praising, and defending IOs to challenging, criticizing, and dismissing the same organizations. These positive and negative discursive strategies are performed by all types of elites and frequently referred to as legitimation and delegitimation of IOs (Zaum Reference Zaum2013; Binder and Heupel Reference Binder and Heupel2015; Tallberg et al. Reference Tallberg, Bäckstrand and Scholte2018; Zürn Reference Zürn2018). Empirical examples include former US President Donald Trump calling NAFTA the worst trade deal the US ever signed (New York Times, March 30, 2017), then Greek finance minister Yanis Varoufakis accusing the EU for terrorism during the country’s economic crisis (BBC, July 4, 2015), UN Secretary General António Guterres praising the organization’s peacekeepers for making the world safer (UN Reference Anderson, Bernauer and Kachi2019), and European Commission President Ursula von der Leyen portraying the EU as a global leader on competitiveness and the green economy (Strasbourg, September 15, 2021).
The study and practice of legitimation and delegitimation assume the evaluative component of communication to matter. It is by praising or criticizing IOs that elite messages become potentially powerful in shaping citizen attitudes. If elite messages had been neutral, few would have expected them to be influential. This expectation is borne out in studies showing that positive and negative party cues shape public support for the EU (Maier et al. Reference Maier, Adam and Maier2012) and on international issues generally (Guisinger and Saunders Reference Guisinger and Saunders2017). However, it is an unexplored question whether legitimation or delegitimation is systematically more or less effective in shaping citizen attitudes toward IOs.
As theorized in Chapter 3, we expect that negative elite messages will have stronger effects on legitimacy perceptions than positive messages. We base this expectation on research in comparative politics, economics, and psychology. While identifying slightly different mechanisms, all traditions ground their expectations in general sociopsychological dynamics, and all suggest that negative messages should have a larger impact than positive. Research on voting behavior shows that people respond asymmetrically to positive and negative information about the economy (Bloom and Price Reference Bloom and Price1975; Soroka Reference Soroka2006). Prospect theory submits that individuals tend to be risk averse, leading people to react more strongly to negative information than to positive (Kahnemann and Tversky Reference Kahneman and Tversky1979; Tversky and Kahnemann Reference Tversky and Kahneman1981). Psychological research on impression formation establishes that bad emotions weigh more heavily than good emotions, such that negative information is processed more thoroughly, is stickier, and has greater impact (Baumeister et al. Reference Baumeister, Bratslavsky, Finkenauer and Vohs2001). We expect these general sociopsychological dynamics to be at play also when people respond to communication about IOs. They suggest a third hypothesis:
H3: Negative messages have a stronger impact than positive messages on citizens’ perceptions of IO legitimacy.
Research Design
To examine our hypotheses, we conducted a population-based survey experiment in three countries. While we could have assessed the hypotheses based on a citizen sample from a single country, we wanted to reduce the risk of biases from contextual country factors. By examining communication effects on legitimacy beliefs across different countries, we strengthen our ability to generalize the findings.
Survey Design
The survey experiment was conducted among nationally representative samples in Germany, the UK, and the US. We selected these countries as they are: (a) politically central in the examined IOs, making our findings substantively important for the prospects of global governance; (b) democratic countries, which avoids the issue that legitimacy for political institutions may mean different things to citizens of democratic and autocratic regimes (Jamal and Nooruddin Reference Jamal and Nooruddin2010); and (c) countries with very high levels of Internet penetration (over 80 percent), increasing our confidence in the external validity of the data.
To implement the questionnaire, we relied on online panels from YouGov (see Online Appendix A). A total of 3,270 interviews were conducted in the UK, 3,268 in Germany, and 3,135 in the US during January 2015. Next to the experiment, the survey questionnaire (Online Appendix C1) included several attitudinal and demographic questions, which we use to describe the country-specific samples (Online Appendix C2), and for a series of randomization checks, also known as “balance tests” (see below).
Experimental Design
The experiment was embedded in a survey questionnaire. In the experimental part, respondents were randomly assigned to groups that received different experimental treatments, and a control group that did not receive any treatment. Following the experimental part, all respondents were immediately asked how much confidence they have in an IO. As discussed in Chapter 3, we operationalize legitimacy perceptions using the measure of confidence in IOs: “How much confidence do you personally have in the [IO] on a scale from 0 (no confidence) to 10 (complete confidence)?” In the experiment, the control group only received the question about confidence in a particular IO. The treatment groups (Table 4.1) received a vignette containing the treatment and then the confidence question. Respondents were never allocated to the same treatment group twice. Respondents who were placed in the control group remained in this group throughout the four rounds.
Table 4.1 Vignettes in the global elites experiment
Treatment | Tone | IO feature | Elite | Wording of vignette |
|---|---|---|---|---|
Treatment 1 | + | Procedure | As you may know, most civil society organizations praise the (IO) for being highly democratic. | |
Treatment 2 | – | As you may know, most civil society organizations criticize the (IO) for being highly undemocratic. | ||
Treatment 3 | + | Performance | NGOs | As you may know, most civil society organizations praise the (IO) for doing a very good job in trying to solve the problems it faces. |
Treatment 4 | – | As you may know, most civil society organizations criticize the (IO) for doing a very poor job in trying to solve the problems it faces. | ||
Treatment 5 | + | Procedure | As you may know, the (IO) prides itself for being highly democratic. | |
Treatment 6 | – | As you may know, the (IO) admits to being highly undemocratic. | ||
Treatment 7 | + | Performance | IOs | As you may know, the (IO) prides itself for doing a very good job in trying to solve the problems it faces. |
Treatment 8 | – | As you may know, the (IO) admits to doing a very bad job when trying to solve the problems it faces. | ||
Treatment 9 | + | Procedure | As you may know, the (COUNTRY) government praises the (IO) for being highly democratic. | |
Treatment 10 | – | As you may know, the (COUNTRY) government criticizes the (IO) for being highly undemocratic. | ||
Treatment 11 | + | Performance | Government | As you may know, the (COUNTRY) government praises the (IO) for doing a very good job in trying to solve the problems it faces. |
Treatment 12 | – | As you may know, the (COUNTRY) government criticizes the (IO) for doing a very poor job in trying to solve the problems it faces. |
We used vignettes to present the treatments. A vignette approach is well suited for complex factorial experiments, as different aspects of the presented story line about IOs can be systematically altered in a vignette (Mutz Reference Mutz2011, 54). Here, we manipulated three features of the communicative situation: the elite making the statement (H1), the features of the IO (H2), and the tone of the message (H3). The vignettes were formulated in a way that allowed us to vary the three factors with precision but also express the subject matter in concrete terms so that it would be understandable to respondents (Gibson Reference Gibson2008). Moreover, we sought to formulate vignettes that would work equally well for all IOs and that were short and straightforward, since longer and more complex vignettes make it more difficult to determine what individuals respond to (Mutz Reference Mutz2011, 64–65).Footnote 3
The treatments combined into a 3 by 2 by 2 factorial design, with twelve conditions in total (Table 4.1). We allocated the same number of individuals to each combination of factors, and the number of respondents giving a substantive answer was eventually relatively even across groups (see Online Appendix C2).
To examine H1, we varied the elite making the statement in the vignette: NGOs, member governments, or IOs themselves. While NGOs is our analytical actor category, we use the term CSOs in the vignettes in order for respondents to more easily understand the nature of these actors. To assess H2, we formulated vignettes about the procedures and performances of IOs, where procedural vignettes invoked the democratic character of IOs and performance vignettes the problem-solving effectiveness of IOs. To evaluate H3, we designed the vignettes so that they included positive or negative statements about IOs.
Experimental Rounds
Moving beyond the strict hypothesis test, we also explored the extent to which communication effects vary across different IOs. We therefore conducted the survey experiment in several rounds, with each round performing the same experiment on a different IO. We selected five IOs that are central in their respective policy domains and prominent in public debate: three at the global level (IMF, UN, and WTO) and two at the regional level (EU and NAFTA). While some IOs fly beneath the radar of public awareness, we selected IOs that both are known to citizens at a basic level and regularly subject to positive and negative communication by elites.
Examining available surveys, we observe that, in 2011, 95 percent of the residents in the two EU member states had heard of the EU, and 95 percent of the residents in the three states had heard of the UN, while 85 percent of the residents in the three countries had heard of the WTO (Gallup International Association 2011). Knowledge of the IMF was only asked about in 2005, when about 70 percent of the residents in the three countries had heard of this organization (Gallup International Association 2005). NAFTA was not included in any of these surveys. More recent data from the WVS7 (2017–2020) provide a harder test of knowledge regarding global governance. The data suggest that a majority of citizens has reasonable knowledge about the UN, since about 42 percent of citizens in fifty-one countries could identify the five permanent members of the UNSC correctly (N = 73,294). About 30 percent of citizens in the same countries could identify the location of the IMF headquarters (N = 73,444).Footnote 4
Using relatively well-known IOs in the experiment ensures that treatments expressing elite messages about these IOs are understandable and reasonable to respondents. At the same time, the levels of citizen familiarity and public debate differ across these IOs, suggesting potential explanations of variation in treatment effects, further explored in the comparative analysis. All respondents were asked about all IOs of which their country is a member state. That is, the question about confidence in NAFTA was only asked in the US, and the question about confidence in the EU, only in the UK and Germany. The order of the experimental rounds for all respondents was: UN, EU/NAFTA, IMF, and WTO. We examine potential biases resulting from this design choice in the robustness check section below.
Results
We discuss the results for each hypothesis in turn and then disaggregate the analysis by IO and country. Here and in the ensuing experimental chapters, we follow a standard convention in statistical practice to interpret as statistically significant only treatment effects that have a 95 percent chance of being found in the full population (p < 0.05). We calculate treatment effects using ordinary least squares (OLS) regression with confidence as dependent variable. OLS regression analyses with one predictor are equivalent to t-tests, with the advantage that the clustered and weighted nature of the data can be taken into account. All models are estimated using robust standard errors clustered at the level of individuals.
Communicating Elites
The first hypothesis predicts that elite type matters for the effectiveness of elite communication and yields two observable implications. First, the differences in mean confidence between the treatment groups for the different elite types and the control group should be statistically significant. Second, the differences in mean confidence between the treatment groups for the different elites should be statistically significant. In line with our theory, we expect NGOs to be most effective, national governments less effective, and IOs themselves least effective in communication about IOs. To explore these observable implications, we pooled the data across the four experimental rounds so that the observations on confidence in the different IOs are clustered at the level of individuals. We then collapsed the treatment groups on procedure and performance, enabling us to contrast the effects of negative communication by NGOs, IOs, and governments as well as positive communication by the same elites.Footnote 5 To this end, we created several dummy variables indicating if respondents were exposed to a specific vignette.
Figure 4.1 shows the results. This figure and all ensuing figures reporting treatment effects show the effects with their 95 percent confidence intervals. These intervals indicate a 95 percent certainty that the true treatment effect lies within their range. If the confidence intervals include zero, the treatment effect is not statistically significant. Figure 4.1 demonstrates that the effects on confidence in IOs depend on the type of elite, largely corroborating H1. In line with the first observable implication, the first six treatment effects indicate that communication by all three elite types affects citizens’ confidence in IOs. These effects are potentially substantively important when considering that they result from a one-off exposure to treatment. For example, the first treatment effect (0.356) indicates that citizens who have received positively framed messages from NGOs on average have 0.356 more confidence in IOs on an 11-point scale, compared to those who did not receive such messages. However, the third treatment effect is not statistically significant, suggesting that IOs cannot successfully legitimize themselves in the eyes of citizens through appeals to their procedures or performances.
Figure 4.1 Effects of communication, by elites
Notes: Average treatment effects with their respective 95 percent confidence intervals. Weighted data. See Online Appendix C3 for detailed results.
Most importantly, the second observable implication receives support as well, albeit not in all parts. The last six treatment effects in Figure 4.1 indicate that there are some differences in the strength of communication effects between the three types of elites. The results suggest that NGOs manage to sway confidence in IOs more than IOs themselves, irrespective of whether they seek to legitimize or delegitimize IOs. Similarly, governments appear to shape legitimacy perceptions more than IOs when seeking to enhance confidence in IOs, but not when attempting to weaken it. However, the evidence also suggests that NGOs are not more effective than governments in shaping citizens’ confidence in IOs, contrary to our expectation.
These results suggest that the credibility of elites matters for their capacity to sway public perceptions of IOs. IOs appear unable to increase their own legitimacy by presenting themselves in a positive fashion. They are likely perceived as partial, and therefore noncredible, as a source of positive information about their own merits. The finding in other research that IO endorsements can affect public opinion about a state’s foreign policy is not at odds with this result, as IOs in those cases communicate about other actors (Chapman Reference Chapman2009). While IOs increasingly engage in various forms of self-legitimation, our findings question the effectiveness of that strategy. Instead, they suggest that IOs have to rely on positive communication by NGOs and national governments to increase their legitimacy. The finding that these two latter types of elites are equally effective communicators may be due to citizens not perceiving governments as partial principals of IOs, but as credible voices about the merits of these organizations.
IO Features
The second hypothesis predicts that elite communication is equally effective when invoking the procedures and performances of IOs as grounds for endorsement or criticism. H2 has three observable implications. First, the differences in mean confidence between the procedure group and the control group, as well as between the performance group and the control group, should be statistically significant. Second, the differences in mean confidence in these group comparisons should be similar in size. Third, there should not be a statistically significant difference in mean confidence between the procedure group and the performance group. To test this, we collapsed the treatment groups for the different elites and created a series of dummy variables indicating if respondents received positive or negative procedural or performance treatments.
In line with the first observable implication, the differences in means between the four treatment groups and the control group, respectively, are statistically significant (Figure 4.2). This indicates that positive and negative messages about both the procedures and performances of IOs are effective in swaying citizen confidence. Consistent with the second observable implication, the differences in mean confidence in these group comparisons are also very similar in size, suggesting that procedure and performance have equally strong effects. This finding is ultimately confirmed by the last two treatment effects in Figure 4.2, which show statistically insignificant results for the difference-in-means test between procedural and performance treatments, in keeping with the third observable implication. This is corroborated by a t-test statistic for independent samples (t = 0.002 when comparing positive procedural and performance treatments and t = 0.013 when comparing negative procedural and performance treatments).Footnote 6
Figure 4.2 Effects of communication, by IO features
Notes: Average treatment effects with their respective 95 percent confidence intervals. Weighted data. See Online Appendix C3 for detailed results.
These results suggest that citizens care equally about IOs’ procedures and performances when developing legitimacy perceptions (see also Anderson et al. Reference Anderson, Bernauer and Kachi2019; Dellmuth et al. Reference Dellmuth, Scholte and Tallberg2019). Theoretical accounts that privilege one or the other appear misguided. Contrary to claims that democratic procedure has become the foremost source of legitimacy (Held and Koenig-Archibugi Reference Held and Koenig-Archibugi2005), citizens may value IO performance just as much. Conversely, it would appear imprudent to conclude from findings in recent scholarship that citizens mainly care about IOs’ capacity to deliver, in line with findings from previous observational studies (Dellmuth and Tallberg Reference Dellmuth and Tallberg2015), irrespective of the procedures by which policies are developed. From the perspective of communicating elites, there may be a wide menu of messages for effective legitimation or delegitimation of IOs.
Tone
The third hypothesis predicts that the tone of messages matters for the effectiveness of elite communication. H3 has two observable implications. First, the differences in means between the group receiving negative treatments and the control group, as well as between the group receiving positive treatments and the control group, should be statistically significant. Second, the difference in means between the negative and the positive treatment groups should be statistically significant, and negative messages should have stronger effects than positive.
Figure 4.3 shows that the results are in line with both observable implications. Both positive and negative treatments affect legitimacy perceptions. By implication, the difference in means of −0.517 on the 0–10 confidence scale between negative and positive treatment groups is also statistically significant. Furthermore, negative communication (−0.277) has stronger effects than positive (0.240). The statistically significant difference between negative and positive treatments is corroborated by a t-test statistic for independent samples (t = 5.113, see footnote 5).
Figure 4.3 Effects of communication, by tone
Notes: Average treatment effects with their respective 95 percent confidence intervals. Weighted data. See Online Appendix C3 for detailed results.
Figures 4.1 and 4.2 further show that this pattern largely holds even when we disaggregate by the elites making the statements and by IO features. Figure 4.2 shows the effects of negative messages to be larger than the effects of positive messages, regardless of whether elites refer to procedure or performance. The findings in Figure 4.1 show a more varied pattern. In line with the expectation, they indicate that elite communication by IOs is more effective when negatively expressed. This finding ties in with research showing that communication has stronger effects when it is more costly for the communicating party and therefore more credible (Baum and Groeling Reference Baum and Groeling2009). However, for NGOs and governments, the effects are larger when the communication is positively expressed. If we disaggregate by IOs, as we do below, we find that the statistically significant difference in effects between positive and negative messages holds for all five IOs (Figures 4.5–4.7).
Overall, these results suggest that delegitimation of IOs by their opponents is more successful than legitimation by IOs themselves and their supporters. In line with earlier sociopsychological findings, people appear to be more sensitive to negative information than to positive. Our findings suggest a problematic relationship in the public contestation over IOs. While public criticism against IOs often is intended to push these organizations toward improvements, rather than undermine them (Uhlin and Gregoratti Reference Uhlin, Gregoratti, Bexell, Jönsson and Uhlin2022), such advocacy efforts could have costly negative externalities in terms of reduced public confidence.
Interaction Analysis
As a next step, we go beyond strict hypothesis testing and investigate the extent to which treatment effects could depend on partisanship and awareness, as theorized in Chapter 3. We could expect that a government’s cues work best among citizens identifying with the party in office. In addition, we test if political awareness has a conditioning effect on all treatments, based on the expectation that politically more aware individuals are more likely to comprehend and integrate new information into their opinion formation.
To test the first expectation, we interact the treatment dummy on positive and negative government communication, respectively, with a dummy variable capturing whether a respondent identifies with a party in government (= 1) or not. We find that the effect of the negative government treatment is moderated by partisan identification (Figure 4.4). The effect of the negative government treatment is only statistically significant among those identifying with a party in government.
Figure 4.4 Effect of negative government communication, by partisanship
Notes: Figure shows average treatment effects across two subgroups, with their respective 95 percent confidence intervals. Weighted data. See Online Appendix I1 for detailed results.
To test the second expectation, we examine if treatment effects might be conditional on political awareness. We test these issues by interacting the treatment dummies with two awareness indicators, political knowledge and education (Online Appendices I2–I7). Both political knowledge and education are deemed good measures of political awareness (Zaller Reference Zaller1992; Gabel and Scheve Reference Gabel and Scheve2007). The results suggest that more knowledgeable citizens did not respond differently than less knowledgeable citizens when confronted with our vignettes.
Disaggregating Treatment Effects across IOs
To shed light on how context matters, we undertake an additional analysis of the extent to which communication effects vary across IOs. This analysis allows us to assess if cueing effects occur less often in the context of some IOs compared to others. We are particularly interested in whether the patterns conform to our theoretical expectation, developed in Chapter 3, that prior contestation of an IO conditions the effectiveness of elite communication. When people have developed stronger priors about an IO because it has been intensely debated in the past, they should be less susceptible to new information about this organization. Conversely, if citizens have less developed attitudes toward an IO because it has been less debated in the past, they should be more easily affected by elite communication.
Based on this consideration, we expect elite communication to be more effective in relation to the IMF, WTO, and UN than the EU and NAFTA. The EU has been highly contested in the member states of this IO since the early 1990s, when the conferral of greater political authority to the EU set off a process of growing politicization, manifested in popular rejections of new EU treaties, a rise in EU skeptic parties, and Britain’s vote to leave the EU (Hooghe and Marks Reference Hooghe and Marks2009; Hobolt Reference Hobolt2016; Schmidt Reference Dür, Marshall and Bernhagen2019). We therefore find it likely that European respondents, and especially our UK respondents, have developed more hardened attitudes toward the EU than toward the three global IOs. Similarly, NAFTA has been subject to quite some political contestation in its member countries in recent years, especially the US (Hurrelmann and Schneider Reference Hurrelmann and Schneider2015). The IO has been accused of benefiting some member states more than others, of favoring business interests at the expense of environmental and social standards, and of contributing to exploitation of workers (Bow Reference Bow, Hurrelmann and Schneider2015). We therefore find it likely that our US respondents have adopted more developed attitudes toward NAFTA than toward the three global IOs.
For this analysis, we reexamine the differences-in-means between the treatment groups and the control group presented in Figures 4.1–4.3, but now at the level of individual IOs (Figures 4.5–4.7). The analysis shows that the occurrence of treatment effects varies across IOs broadly consistent with the expectation that prior contestation matters. We exclusively report variation in the occurrence of effects, since differences in the strength of effects across IOs are not statistically significant, as indicated by the overlapping confidence intervals.
We begin by assessing the occurrence of treatment effects by IO for different elite types (Figure 4.5). Positive communication by NGOs is effective in relation to the UN, IMF, and WTO, while negative communication only is effective in relation to the UN and NAFTA. Positive communication by IOs never appears to work, mirroring the general ineffectiveness of IO self-legitimation, while negative communication works in all cases but NAFTA. Positive communication by governments about the UN and WTO seems to influence legitimacy perceptions, while negative communication works in the case of the WTO and EU. In sum, communication by global elites tends to lead to treatment effects in relation to the UN, WTO, and IMF, while we see fewer significant effects for the EU and NAFTA.
Figure 4.5 Effects of communication, by elites and IOs
Notes: Average treatment effects with their respective 95 percent confidence intervals. Weighted data. Sample size is about 3,000 for the global organizations, about 2,000 for the EU and about 800 for NAFTA. See Online Appendix C4 for detailed results.
Figure 4.6 reveals a similar pattern across IOs when comparing communication about IOs’ procedures and performances. We observe statistically significant effects for all or most treatments relating to the UN, WTO, and IMF. Conversely, only one treatment pertaining to the EU yields a statistically significant effect and no treatment at all in the case of NAFTA.
Figure 4.6 Effects of communication, by IO features and IOs
Notes: Average treatment effects with their respective 95 percent confidence intervals. Weighted data. Sample size is about 3,000 for the global organizations, about 2,000 for the EU and about 800 for NAFTA. See Online Appendix C4 for detailed results.
We observe the same pattern with regard to valence. As Figure 4.7 shows, positive communication works in the UN, IMF, and WTO, but not in the EU and NAFTA, while negative communication appears to work in the context of all IOs except NAFTA.
Figure 4.7 Effects of communication, by tone and IOs
Notes: Average treatment effects with their respective 95 percent confidence intervals. Weighted data. Sample size is about 3,000 for the global organizations, about 2,000 for the EU and about 800 for NAFTA. See Online Appendix C4 for detailed results.
Taken together, these results indicate that elite communication is more often effective in the context of some IOs than in others. The pattern is largely consistent with the baseline expectation that prior contestation of an IOs matters for the effectiveness of elite communication: Cueing more often produces significant effects in the context of the IMF, UN, and WTO, and more seldom in the context of the EU and NAFTA.
Disaggregating Treatment Effects across Countries
Next, we disaggregate the analysis at the level of the individual countries (Figures 4.8–4.10). This allows us to examine if cueing effects occur more often in the context of some countries than others. There is considerable variation in the experience and salience of IOs across countries, and this may shape the effects of elite communication.
Figure 4.8 Effects of communication, by elites and countries
Notes: Average treatment effects with their respective 95 percent confidence intervals. Weighted data. See Online Appendix C5 for detailed results.
Figure 4.9 Effects of communication, by IO features and countries
Notes: Average treatment effects with their respective confidence intervals. Weighted data. See Online Appendix C5 for detailed results.
Figure 4.10 Effects of communication, by tone and countries
Notes: Average treatment effects with their respective 95 percent confidence intervals. Weighted data. See Online Appendix C5 for detailed results.
In Germany, communication by all three elite types and regarding both procedure and performance shape citizen confidence toward IOs. However, only negative treatments are statistically significant. This pattern is consistent with the general expectation that negative communication is more effective than positive.
In the UK, similarly, communication by all three elite types and regarding both institutional features is effective. Different from in Germany, both positive and negative treatments are statistically significant, with one exception: negative communication by NGOs. This exception is not driven by dynamics surrounding a particular IO, such as the EU, but applies to all four IOs (Table C.5.8). It suggests that British citizens do not find NGO criticism of IOs credible, possibly because it is common and less surprising.
In the US, there are fewer significant treatment effects. Only positive communication by NGOs and negative communication by IOs appear effective. This pattern indicates that the elites we study in this chapter – NGOs, the national government, and IOs themselves – are less effective communicators in the US compared to other countries. Part of the reason may be that US citizens instead listen to domestic party elites as they make up their minds about IOs, as indicated by previous research on internationalist attitudes in the US (Guisinger and Saunders Reference Guisinger and Saunders2017) and as we also show in Chapter 5.
Validity and Robustness Checks
We perform several robustness checks. First, we replicate all analyses by including country dummies to check whether the aggregate results hold when controlling for potentially unobserved country-specific variables, given the country-specific results shown in this chapter. This change in model specification does not alter the interpretation of our main results (see Online Appendix E). We do not perform this test for the NAFTA subgroup, since it is based on only one country.
Second, we conducted two sets of balance tests. In the first set, we examined whether eight different individual characteristics measured in the survey, including education and age, are evenly distributed across the conditions we aggregated for the analysis. The results increase our confidence in the randomization of the subjects among treatment groups, as we only discover imbalances in six of the ninety-six tests. The second set of balance tests uses the twelve treatment conditions in Table 1 and shows evidence of only four imbalances for ninety-six tests (Online Appendices F1–F2).
Third, we examine whether there are potentially undesired spillover effects because the order of the four experimental rounds was not randomized. For this purpose, we have conducted balance tests for each round separately to assess if the fixed order of the rounds could have given rise to biases resulting from dropouts (Online Appendices F3–F10). We found no pattern indicating a potential systematic bias, as approximately the same very low number of balance tests comes out statistically significant in each round. Still, we examine potential biases further, as the absence of randomization of experimental rounds may give rise to varying distributions of respondents across samples. Indeed, the number of respondents giving substantive answers drops when comparing rounds 1 and 4 (Online Appendix C2). We test whether the experimental effects vary across the four rounds by plotting the predicted marginal effects of the different treatments for different rounds (Online Appendix G). The slopes of the dummy variables for the specific treatments are largely parallel across the four experimental rounds, indicating an absence of systematic differences.
Fourth, we replicated all analyses in order to check whether item nonresponse may have affected the results. If the use of the “don’t know” option reflects lower political knowledge about IOs (cf. Jessee Reference Jessee2017), these values would not be missing at random (cf. Rubin Reference Rubin1976) and average treatment effects may be biased. We therefore examined the causal process behind missingness and found that item nonresponse is unlikely to have affected our results (see Online Appendix H). While the effects of sociodemographic factors become smaller when comparing rounds 1 and 4, possibly as a result of samples having become more homogenous due to increasing item nonresponse, we also find instances of effect sizes becoming larger when comparing across other rounds (e.g., education across rounds 3 and 4, and age and gender across rounds 1 and 2).
Fifth and finally, we examine whether the results for governments as communicating elites are conditional on whether people trust their own government. We find that, while the effect of the positive government treatment is unconditional on government trust, the effect of the negative government treatment is moderated by confidence in national government. More specifically, the negative effect of the negative government treatment becomes stronger the higher the respondent’s trust in government (Online Appendix I8). These findings are in line with previous research suggesting that people distrusting their own government are unlikely to follow government cues (Aaroe Reference Aaroe2012).
Taken together, the evidence from the validity and robustness checks strengthens our confidence in the experimental findings.
Conclusion
This chapter has evaluated the conditions under which communication by global elites affects the popular legitimacy of IOs. In brief, our results indicate that: (a) more credible elites – NGOs and member governments – tend to affect legitimacy perceptions more than less credible elites – IOs themselves, (b) legitimacy perceptions are equally affected by messages about the procedures and performances of IOs, and (c) negative communication has stronger effects on legitimacy perceptions than positive communication. Moreover, a comparative analysis suggests that communication effects are more often effective in the context of the UN, IMF, and WTO, than in the EU or NAFTA, which we attribute to variation in the prior contestation of IOs. We also establish that communication by these global elites more often is effective in Germany and the UK than in the US, which may be reflective of US citizens being relatively more attentive to domestic elites (see Chapter 5).
Our findings suggest three broader implications. First, they speak to the growing literature on legitimation and delegitimation in global governance (Zaum Reference Zaum2013; Binder and Heupel Reference Binder and Heupel2015; Gronau and Schmidtke Reference Gronau and Schmidtke2016; Bäckstrand and Söderbaum Reference Bäckstrand, Söderbaum, Tallberg, Bäckstrand and Scholte2018; Zürn Reference Zürn2018; Dingwerth et al. Reference Dingwerth, Witt, Lehmann, Reichel and Weise2019; Bexell et al. Reference Bexell, Jönsson and Uhlin2022) by demonstrating how such communicative practices impact legitimacy beliefs. While previous research has shown how member governments, NGOs, and IOs themselves make use of legitimation and delegitimation, we identify the consequences of this communication for the popular legitimacy of IOs.
Second, our results suggest that elite communication works slightly differently under the specific circumstances of global governance. The global setting involves another set of elites than those which dominate the study of political communication in the domestic context. Importantly, we found elite credibility based on impartiality, rather than partisanship, to matter in the global setting, different from conventional expectations (Bullock Reference Bullock2011; Druckman et al. Reference Druckman, Peterson and Slothuus2013). With the growing internationalization of politics, it becomes increasingly important for public opinion research to explore how political communication and attitude formation work differently or similarly in the global realm.
Third, our findings suggest why the elites of discontent may have the upper hand in the global contestation over the legitimacy of IOs. While IOs invest considerably in public communication (Ecker-Ehrhardt 2018), citizens do not appear to be convinced by IOs’ attempts to talk up their legitimacy. IOs’ best chance of strengthening their standing with citizens may therefore reside in mobilizing supporters among civil society and national political elites who can speak on their behalf. But such efforts run up against the challenge that positive communication appears less effective than negative in shaping citizen attitudes. Elites who criticize global governance get through to citizens more easily than those who speak to its virtues. These results may help to explain instances of popular backlash against IOs in recent years, as well as difficulties encountered by advocates of global governance.
Open access