Introduction
Personal social networks play an important role in citizens’ political behaviors and attitudes (Pietryka and DeBats, Reference Pietryka and DeBats2017; Reilly, Reference Reilly2017; Ryan, Reference Ryan2011; Santoro and Beck, Reference Santoro and Beck.2017; Settle, Bond, and Levitt Reference Settle, Bond and Levitt.2011; Sinclair, Reference Sinclair2012; Sokhey and McClurg, Reference Sokhey and McClurg.2012; Song and Eveland Jr, Reference Song and Eveland2015). As a result, social network measures have been widely used in surveys to measure respondents’ personal communication contexts. Among scholars who analyze political discussion, however, there is a long-running debate about whether important differences exist between measuring a citizen’s “political” discussion networks and their more general “social” discussion networks. For instance, some scholars suggest that a citizen’s political discussion network will be more politically homogenous than their general social discussion network due to political conflict avoidance on the part of citizens (Mutz, Reference Mutz2002). Others, however, suggest that most citizens do not meaningfully distinguish between political and social discussion partners, discussing both political and nonpolitical matters in a single core discussion network of close acquaintances (Huckfeldt and Sprague, Reference Huckfeldt and Sprague.1995).
This debate yields challenges for the construction of survey items. Should researchers ask respondents about their “political” discussion networks or their general “social” discussion networks? Klofstad, McClurg, and Rolfe (Reference Klofstad, McClurg and Rolfe.2009) argue that the matter is moot: they find that political discussion survey questions prompt largely indistinguishable responses to more general “important matters” social discussion questions on a variety of dimensions, including the size of the discussion network, characteristics of network members, and discussion frequency.
Yet questions remain about the interchangeability of the two discussion measures. Responding to survey questions is a cognitively complex process of memory search, retrieval, consolidation, and finally, mapping judgments to a response option (Krosnick, Reference Krosnick1999; Schwarz and Oyserman, Reference Schwarz and Oyserman.2001). Moreover, many people have different ideas about what constitutes “political” discussion in the first place (Eveland Jr, Morey, and Hutchens Reference Eveland, Morey and Hutchens.2011; Settle, Reference Settle2018). Politically interested respondents, in particular, are not only more enthusiastic about discussing politics with others but also have a significantly broader view of what counts as “political” discussion than do the politically uninterested (Fitzgerald, Reference Fitzgerald2013). Simultaneously, increased partisan animosity in recent years (Mason, Reference Mason2018) has made less interested citizens more inclined to disengage from contentious and rancorous political discussion entirely (Klar and Krupnikov, Reference Klar and Krupnikov.2016; Settle and Carlson, Reference Settle and Carlson.2019).
Accordingly, in the modern political environment, we might expect different kinds of responses from citizens now than in the past. In particular, not only might highly politically interested citizens report larger political discussion networks, when compared to garden-variety social discussion networks, but also the politically uninterested may report smaller political discussion networks. The two survey measures may not be as interchangeable as previously thought.
Study
To evaluate whether political discussion and general social discussion prompts are interchangeable on surveys, we embedded a survey experiment in a module of the 2020 Cooperative Election Study (Reilly, Reference Reilly2022; Schaffner, Ansolabehere, and Luks Reference Schaffner, Ansolabehere and Luks.2021). In the module, we randomly assigned every respondent to receive one of two network prompts:
-
Treatment A (“Politics”): How many friends and family members would you say you regularly talk to about politics? (n = 427)
-
Treatment B (“Social”): How many friends and family members would you say you regularly talk to about important matters in your life? (n = 415)
Questions were phrased to be brief but still closely match other survey questions regularly used in political research, including the choice of “politics” and “important matters” as the two key phrases (see, for instance, Huckfeldt, Johnson, and Sprague (Reference Huckfeldt, Johnson and Sprague.2004); Klofstad, McClurg, and Rolfe (Reference Klofstad, McClurg and Rolfe.2009)). Respondents could indicate that they talked to 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 or 10+ people.Footnote 1
Table 1. Distribution of network size across two measures
On average, “politics” respondents reported 4.3 discussion partners, while “social” respondents reported 4.0 discussion partners. However, while the two resulting response distributions are centered in roughly the same place, they have different shapes. The “politics” group provided more responses on the tails of the distribution (0 or 10+ discussion partners) while the “social” group reported comparably fewer, resulting in a distribution with more central clustering, peakedness, and a smaller overall standard deviation (
$sd_{pol}=3.2$
,
$sd_{soc}=2.5$
;
$kurtosis_{pol}=2.6$
,
$kurtosis_{soc}=4.0$
; see Figure 1). Furthermore, although a t-test yields a negligible and statistically insignificant difference between sample means, a Kolmogorov–Smirnov test reveals a statistically significant difference in the distribution of overall responses (Table 1). Additional analysis confirms that the probability of response is significantly different in tail categories 0 and 10+, where more people from the “politics” condition respond, and in central categories 3 and 4, where more people from the “social” condition respond.Footnote
2
Figure 1. Distribution of network size across two measures.
To evaluate whether these differences were related to political interest, as speculated, we fit a censored poisson regression model, predicting reported discussion network sizes by the experimental condition, the respondent’s level of political interest, and the interaction of the two variables. We operationalized “political interest” using a survey question asking how often a respondent would “follow what’s going on in government and public affairs”, with four response options varying from “hardly at all” to “most of the time”.Footnote 3
Figure 2 plots the predicted network size of respondents based on their experimental condition and level of political interest. Our results reveal that respondents who were most interested in politics responded with higher network sizes when in the “politics” condition than when in the “social” condition. Accordingly, respondents who were least interested in politics were more likely to respond with lower network sizes in the “politics” condition when compared with the “social” condition. Perhaps most tellingly, there was little overall variation in network size across political interest for those in the “social” condition, but significant variation across political interest for those in the “politics” condition.
Figure 2. Predicted discussion network size response varies by political interest for “politics” respondents but not for “social” respondents (95% confidence intervals).
Discussion
Our results suggest that “politics” and general “social” prompts are not fully interchangeable when measuring the size of respondent-reported interpersonal discussion networks in surveys. A “politics” generator results in more responses with smaller or larger networks than does an “important matters” social network generator. Furthermore, respondent levels of political interest are strongly correlated with discussion network size in the “politics” generator, but not in the “social” generator, indicating these distributions are generated by different survey response processes in our respondents.
These findings have implications for how we measure social and political discussion networks and for how we think of discussion network formation generally. Methodologically, researchers should consider whether they want to generate estimates of personal networks that are more political in nature or more general in nature. Substantively, our results suggest that political discussion networks are, in fact, different in nature from general social discussion networks, in contrast to earlier findings (Huckfeldt and Sprague, Reference Huckfeldt and Sprague.1995; Klofstad, McClurg, and Rolfe Reference Klofstad, McClurg and Rolfe.2009). Accordingly, some changes may be required in the way we think about social communication in politics in the modern polarized age.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/XPS.2022.34
Data availability statement
Support for the 2020 Cooperative Election Study data used in this research was provided by the National Science Foundation (Award #1948863).
The data, code, and any additional materials required to replicate all analyses in this article are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at: https://doi.org/10.7910/DVN/O96MDX. (Reilly and Belk, Reference Reilly and Belk2022).
Acknowledgments
We thank Matthew Pietryka, Debra Leiter, and members of the New College of Florida Political Behavior Lab for comments and advice on earlier versions of this manuscript, as well as the comments of three anonymous reviewers. We also thank the National Science Foundation (Award #1948863) and the New College of Florida for funding support.
Conflicts of interest
The authors declare no conflicts of interest.
Ethics statement
The data collection for this research study was approved by the New College of Florida’s Institutional Review Board (IRB ID: 2016/08/1) and adheres to all relevant aspects of APSA’s Principles and Guidance for Human Subjects Research.
Open access
