Gathering Legislative Speeches in the Uruguayan Parliament

Uruguay is one of the oldest and stablest democratic regimes in Latin America, leading, along with Costa Rica, most rankings measuring the quality of democracy in the region (Coppedge et al. Reference Coppedge, Gerring and Lindberg2017; Polity I 2012; Freedom House 2014). Its highly institutionalized party system (Mainwaring Reference Mainwaring2018), established by Nationalists and Colorados (PN and PC) during the nineteenth century and lately the Frente Amplio (FA, born in 1971), is considered the backbone of Uruguayan democracy.

To a large extent, such persistence of parties in Uruguayan politics is rooted in the ability of partisan elites to preserve clear programmatic and ideological stances in party competition. For this reason, Uruguay is an interesting case to reconstruct the preferences of parties and legislators over time. Given this opportunity, the main concern of this research note is whether the positions of Uruguayan parties are identifiable and differentiable across legislatures using text analysis. And more important: are different scaling methods used to estimate the preferences of parties convergent or divergent over time? Because these questions have received little attention in the research, particularly among Latin American countries (for an exception see Kim et al. Reference Kim, Londregan and Ratkovic2018; and Izumi and Medeiros Reference Izumi and Medeiros2021 for the Brazilian case), the goal here is to contribute to current and future research on these topics, comparing two scaling methods that make use of a different type of data to address our research questions.

Traditionally, studies of political behavior have used legislative surveys, expert surveys, and party manifestos to reconstruct the preferences of political actors (Benoit and Laver Reference Benoit and Laver2006, Reference Benoit and Laver2007; Knutsen Reference Knutsen1998; Laver and Hunt Reference Laver and Ben Hunt1992). However, despite their widespread use and undeniable virtues, these sources have their own shortcomings (Slapin and Proksch Reference Slapin and Proksch2014). On the one hand, there are temporal limitations to rebuilding the preferences of actors, due to generations of deceased voters, extinct political parties, or disappeared economic elites, among many other limitations, especially in third wave democracies. On the other hand, spatial variation is also challenging, as a great number of countries and other subnational institutions have made only limited advances in data availability.

More recently, scholars have developed new techniques to complement attitudinal studies, with data coming from roll-call votes in legislatures (Poole and Rosenthal Reference Poole and Rosenthal2000; Carroll and Poole Reference Carroll and Poole2014), social networks (Bond and Messing Reference Bond and Messing2015), archival data, and public speeches (Kim et al. Reference Kim, Londregan and Ratkovic2018; Laver and Garry Reference Laver and Garry2000; Huber and Inglehart Reference Huber and Inglehart1995; Dinas and Gemenis Reference Dinas and Gemenis2010; Laver et al. Reference Laver, Benoit and Garry2003; Welbers et al. Reference Welbers, Van Atteveldt and Benoit2017; Ramey et al. Reference Ramey, Klingler and Hollibaugh2019; Slapin and Proksch Reference Slapin and Proksch2014). In all cases, the goal has been to collect and systematize data to infer the preferences of different individual and collective actors over time and under different political and institutional contexts. Among these alternatives, W-nominate scores (Poole et al. Reference Poole, Lewis, Lo and Carroll2011) and other ideal point estimation techniques (Jackman Reference Jackman2001; Martin and Quinn Reference Martin and Quinn2002; Kellermann Reference Kellermann2012) based on roll calls have become one of the most reliable pieces of evidence to capture the preferences of political actors. Yet roll-call data availability is very limited beyond most advanced democracies.

In this context, text analysis has emerged as a reasonable option to bypass the limitations of roll-call records in most democracies around the world. As a matter of fact, text analysis has been considered a viable alternative to solve the limitations of time and space usually attributed to roll-call data (Curini et al. Reference Curini, Hino and Osaka2018; Lauderdale and Herzog Reference Lauderdale and Herzog2016). However, the content analysis techniques (of examining speeches, documents, or written records of events) also presents some weaknesses that have captured the attention of experts (Gabel and Huber Reference Gabel and Huber2000; Klemmensen et al. Reference Klemmensen, Hobolt and Hansen2007). On the one hand, the method requires a considerable amount of resources, particularly when it is applied to important volumes of documents or speeches. The need for reliable and systematic coding leads to significant demand for manual work. On the other hand, the ability to extend this technique to analyze content across time and space has important technical limitations. Coding and systematizing large volumes of text from manual work is onerous for one or even several encoders. Due to these limitations, extant applications have mainly focused on industrialized democracies. Our new dataset aims to expand those limits.

We built on extant techniques to develop an original dataset implemented on R (R Core Team 2017). The sequence starts with web scraping to obtain the Actas de Sesión; that is, the formal registry of all the activities involved in an individual legislative session. Some of these available documents on the Uruguayan legislative website are in HTML and some are in PDF format. The remaining documents that were not available on the web were manually scanned and transformed into PDF format.

The data generation process is flexible, giving researchers different options to compile the resulting dataset according to their interests. Depending on the research goal, scholars could be interested in different portions or contents of legislative speeches. The package includes a function allowing users to compile and recompile the dataset with different structures. For example, if someone is interested in scaling the position of parties, the dataset can be compiled using this unit of analysis. Also, the dataset can be recompiled with individual legislators as the unit of analysis, which enables adding new variables like factions, electoral districts, and district size, among many others that might interest students of legislative bodies and other research fields. Additionally, it is possible to track specific debates within and among legislatures (e.g., abortion, human rights, budgeting) using functions included in packages like quanteda (Benoit et al. Reference Benoit, Watanabe and Wang2018).

Figure 1 displays the building process of our dataset. First, the resulting structure of the text-mining process considers that some or several legislators speak at different times during the same session. Notice that this is a dataset of speeches performed by legislators who invariably belong to a certain political party. This rule excludes some individuals who participate in legislative sessions, like cabinet ministers, and staff members who are frequently invited to participate in committee and general sessions by request. The main function of the package has a search algorithm based on regular expressions that eliminate these cases.

Figure 1. Sequence of the Data Generation Process

Formally, figure 2 shows the resulting structure of the dataset. In order to work with individual speeches, we extract and cluster all the interventions for each legislator who speaks in a single session. Then we check whether the legislators’ names are correctly written and correct the multiple typing errors in the original document to cluster the interventions for each legislator. Compiling the dataset by individual legislators, the first variable is the legislator’s name, followed by other variables like the legislature number, the date and content of the speech, the legislator’s party, and the document type.Footnote ² Depending on the research interest, the dataset can be recompiled using the legislator´s party as the unit of analysis or any other added variable of interest, such as gender, district magnitude, or others.

Figure 2. Resulting Structure of the DGP

The resulting dataset contains more than 57,000 observations, covering more than 1,000 unique legislators from 7 political parties across 6 legislative periods for 3 different chambers: the Chamber of Representatives, the Senate, and the Asamblea General (General Assembly) (as the fusion of the upper and lower chambers). One of the main features of the dataset and the speech package is the easy integration with available packages and tools for computational text analysis, such as quanteda (Benoit et al. Reference Benoit, Watanabe and Wang2018) and tm (Feinerer and Hornik Reference Feinerer and Hornik2018; Feinerer et al. Reference Feinerer, Hornik and Meyer2008). The structure of the dataset was conceived for that purpose, allowing researchers to compute scores, obtain summary statistics and frequency tables, and generate word clouds and other descriptive measures of interest. Also, it is possible to estimate scores for an individual legislator across different legislative periods, or for a specific set of legislators, parties, or any other combination in which the researcher could be interested. Table 1 shows the variables included in the dataset. In addition to the content of legislative speeches, several variables allow working with auxiliary information, like the legislator’s party, gender, or electoral district.

Table 1. Variables Included in the Dataset

Table 2 and figure 3 present descriptive statistics related to the speeches and the number of legislators in the dataset by legislative period. For example, in the first legislature after the transition to democracy (1985–90), a total of 236 legislators (between incumbents and alternates) spoke on the floor, delivering 7,645 speeches in 424 sessions. The average length of those speeches was 1,156 words. Figure 3 shows the distribution of legislative speeches among parties and across legislative periods. There is some systematic variation, suggesting that government parties tend to deliver shorter speeches on the floor while opposition parties tend to do the opposite.

Table 2. Descriptive Statistics of Legislative Speeches in the Uruguayan Parliament, 1985–2015

Figure 3. Number of Words in Individual Legislators’ Speeches in Uruguayan Parliament, 1985–2015, by party

During the first legislative period (1985–90), the legislators of the governing party (PC) delivered speeches with a lower median compared to the opposition parties (PN and FA). During the following three legislative periods, the PC and the PN governed together as a coalition, and both parties’ legislators made shorter speeches compared to the main opposition party, the FA. This was reversed in 2005, when the FA became the governing party. During the following two legislative periods (2005–15), the FA’s legislators delivered shorter legislative speeches than legislators of both opposition parties, the PN and the PC. This is a reasonable finding; time is important for legislative work, so governing parties are mainly concerned with approving legislation, while opposition parties may be more interested in delivering speeches for position-taking purposes.

Estimating Policy Positions of Uruguayan Parties Using Roll-Call Votes and Legislative Speeches

Given the importance of its political parties, Uruguayan democracy has been characterized as a partidocracia (Caetano et al. Reference Caetano, Rilla and Pérez1987). The continuity of the main party labels, as well as the salience of their ideological profiles, is a propitious context to address the main goal of this research note. That is, to estimate their policy positions through different scaling methods based on individual legislative speeches and roll-call voting, and compare them to find whether systematic differences emerge.

Although estimating ideological or policy positions from text and roll calls raises some concern because of the limitations that individual legislators might face when voting, both sources of data continue to be the central piece for estimating parties’ and individuals’ preferences (Carroll and Poole Reference Carroll and Poole2014, Slapin and Proksch Reference Slapin and Proksch2014). For example, previous studies focused on the Uruguayan Senate and the Brazilian Congress have shown that ideal point estimates may be reflecting latent dimensions different from ideology (Izumi Reference Izumi2016; Zucco Reference Zucco2009, Reference Zucco2013; Izumi and Medeiros Reference Izumi and Medeiros2021). Also, according to some studies, legislative speeches can be restricted for strategic reasons in a way similar to voting (Proksch and Slapin Reference Proksch and Slapin2012). However, other studies argue that legislative speeches are less constrained than voting, as long as legislators vote according to the party line (Schwarz et al. Reference Schwarz, Traber and Benoit2017).

Therefore, we ask whether Uruguayan legislators and parties are more or less constrained by these factors. We are aware that under some circumstances the government-opposition cleavage or party discipline can influence legislators’ behavior, and that is one additional reason to perform the comparison between speech and vote. If there were no restrictions on the behavior of individual legislators, one would expect to find congruence between what they say on the floor and how they vote. But precisely because it is expected that those restrictions are operative, we seek to systematically compare parties’ positions estimated from legislative speeches and roll calls.

To address our main concern, we first estimated parties’ positions from their legislators’ floor speeches. One of the most widespread methods to infer the parties’ policy positions from political texts is Wordscores (Laver et al. Reference Laver, Benoit and Garry2003; Lowe Reference Lowe2008; Lowe et al. Reference Lowe, Benoit, Mikhaylov and Laver2011; Klemmensen et al. Reference Klemmensen, Hobolt and Hansen2007). Based on all available floor speeches during the period 1985–2005, figure 4 shows the Wordscores for the three main political parties in the General Assembly.Footnote ³ Since Wordscores requires setting reference texts with known scores in the relevant dimension (Slapin and Proksch Reference Slapin and Proksch2014), we used the speeches of individual legislators known for their ideological positions.Footnote ⁴ As the figure shows, the comparison among parties shows little ideological differentiation at the party level. Nevertheless, figure 4 also reveals an important intraparty variation in policy preferences. This variation reflects the well-known factional nature of Uruguayan parties, showing the differences that may be associated with the policy positions of factions (Buquet et al. Reference Buquet, Chasquetti and Moraes1998; Moraes Reference Moraes, Siavelis and Morgenstern2008; Magar and Moraes Reference Magar and Moraes2012). Thus, based on text scaling, Uruguayan parties display a very similar ideological outlook, with an internal differentiation that we attribute to their traditionally factionalized structure, in line with the literature on Uruguayan parties.

Figure 4. Wordscores in Uruguayan General Assembly, 1985–2005

Second, we proceeded to estimate parties’ positions from roll-call data. Nominal votes are exceptional in Latin American politics, and Uruguay is not the exception (Carey Reference Carey2009; Zucco Reference Zucco2013; Ainsley et al. Reference Ainsley, Carrubba and Crisp2020). Because the dataset allows for a specific search of words and text patterns, we searched for all the available roll calls during the period. In line with Ainsley et al. (Reference Ainsley, Carrubba and Crisp2020), we identified roll-call votes from an automatized search for regular expressions.Footnote ⁵ However, unlike Ainsley et al. (Reference Ainsley, Carrubba and Crisp2020) or Zucco (Reference Zucco2013), we decided to keep only the roll calls associated with the treatment of executive vetoes that take place in the Asamblea General, since the vast majority of nominal votes in the lower or upper chambers refer to specific issues on administrative decisions of the chamber (e.g., the election of authorities and other administrative personnel) that cannot be coerced into yes or no, which is crucial to running the W-nominates algorithm.

Additionally, the treatment of executive vetoes is the only legislative instance in which nominal voting is mandatory, and thus it constitutes the most reliable and comparable source of roll calls in the Uruguayan Parliament. Therefore, limiting the analysis of roll calls to the executive vetoes allows us to deal with a more homogeneous corpus of nominal votes, restricted to substantive and divisive issues that emerge from potentially conflicting interests between legislative and executive powers.Footnote ⁶ Thus, we collected 77 roll-call votes during the 4 legislative periods under consideration, which were used to calculate another standard measure of ideal point estimation: W-nominate scores.Footnote ⁷

Figure 5 displays the W-nominate scores for the three main Uruguayan parties for the period 1985–2005, using all available roll calls.Footnote ⁸ Unlike Wordscores, W-nominate scores display a substantive level of party differentiation. Such differentiation at the party level is largely consistent with the programmatic or ideological differences observed by domestic and comparative students of parties who rely on elite survey data or public opinion polls. Furthermore, our W-nominate data also show a great deal of within-party differentiation, which resembles our estimations in Wordscores. Therefore, parties differentiate not only from each other but also among themselves, as students have observed using different sources of empirical evidence (Buquet et al. Reference Buquet, Chasquetti and Moraes1998; Moraes Reference Moraes, Siavelis and Morgenstern2008; Magar and Moraes Reference Magar and Moraes2012; Chasquetti Reference Chasquetti2014). Overall, W-nominate scores are closer to the findings observed in the specialized literature.

Figure 5. W-nominate Scores in Uruguayan General Assembly, 1985–2005

We went on to compare the results of W-nominate scores with Wordscores. Interestingly, a large body of literature in recent years has pointed out that political discourse is an appropriate way to capture the policy preferences of different political agents (Benoit and Herzog Reference Benoit, Herzog, Bachner, Wagner Hill and Ginsberg2017). This means that if there is an essential difference among parties (and alternatively among factions), it should be captured by analyzing political speeches, as well as other political expressions using documents, newspapers, and other sources. Yet our results using Wordscores do not reveal such differentiation at the party level. Instead, the only observed differentiation occurs within parties, as can be seen in figure 5.

Regarding the systematic differences among scaling methods, figure 6 shows that there is no clear association. The overall correlation is low (r = 0.22), and it is statistically significant at 99 percent confidence. When we look at partial correlations across legislative periods, some differences emerge. It can be seen that for the periods 1985–90 and 2000–2005, there is no clear association between speech and roll-call votes. But for the periods 1990–95 and 1995–2000. there is a moderate to high correlation (0.64 and 0.76, respectively). Not surprisingly, these two legislative periods were characterized by the functioning of governing coalitions between the PC and the PN that reconfigured Uruguayan politics and laid the foundations for the electoral reform of 1997, motivated by the electoral rise of the FA.

Figure 6. Comparing Parties’ Policy Positions Through Wordscores and W-nominate in Uruguayan General Assembly, 1985–2005

According to our estimations using speech, legislators seem to agree on a similar set of terms, using a highly homogeneous vocabulary that does not allow us to make a clear differentiation among parties with legislative representation. Such lack of differentiation contrasts with the voting patterns of parties in the legislature, where they are distinct from each other. Arguably, legislators may speak with a very similar jargon or set of words and expressions, from which we cannot identify different policy orientations. However, differentiation appears when parties and factions step up to vote.

These results are somewhat surprising, given scholars’ observations on the nature of programmatic parties in Uruguay, which would lead us to expect more congruence between speech and voting patterns. The divergence we found between the two scaling methods, based on a different type of data, does not let us confirm this expectation. While estimating the parties’ positions through roll-call data leads to differentiation among parties (as well as inside parties), the estimation using floor speeches does not confirm that differentiating pattern. These results are similar to those of Schwarz et al. (Reference Schwarz, Traber and Benoit2017) for the Swiss legislature, who also find that text scaling from legislative speeches reveals intraparty differentiation more clearly than roll calls.

The divergence among scaling methods may be due to the methods themselves, or to the fact that both data sources are the product of different goals on the part of individual legislators. In other words, it is possible that when an individual legislator speaks on the floor, he or she intends to maximize the expressive function of legislative work (Andeweg Reference Andeweg2014). But when the same legislator votes, he or she may be maximizing party goals, especially among disciplined parties like those in Uruguay. In this case, we expect to find some divergence among the policy positions based on legislative speeches compared with those coming from roll-call data, in line with what Izumi and Medeiros (Reference Izumi and Medeiros2021) found for the Brazilian Senate.

To check the robustness of our results, we replicated the analysis using two alternative, unsupervised scaling methods based on both speech and roll calls. We estimated the parties’ policy positions through Wordfish (Slapin and Proksch Reference Slapin and Proksch2008) to overcome the issues that may arise by selecting the reference texts. Wordfish is an unsupervised scaling algorithm designed to estimate policy positions based on word frequencies that, unlike Wordscores, does not require the identification of reference texts (Slapin and Proksch Reference Slapin and Proksch2008). Our results using Wordfish show a very similar picture in terms of differentiation within and among parties. We also estimated parties’ positions from roll-call votes through IDEAL (Jackman Reference Jackman2001; Clinton et al. Reference Clinton, Jackman and Rivers2004), a Bayesian scaling method that allows us to overcome the problems that may arise as a consequence of the low number of roll calls, and perhaps more important, to avoid identifying the “polar” legislators, as in W-nominates. As with Wordfish, the estimations from IDEAL show very similar results to those obtained using W-nominates.Footnote ⁹

Both robustness tests allow us to be confident that our results are not heavily dependent on the decisions we made by selecting the reference texts and the “polar” legislators when scaling parties’ positions through Wordscores and W-nominates. In turn, and most important, they also suggest that the divergence we found between speech and vote is not dependent on the scaling method but on the fact that they are reflecting different behavior by individual legislators, factions, and parties. It seems that floor speeches in the General Assembly do not allow for capturing the variation in parties’ policy positions. Unlike roll calls, legislative discourse does not seem to be enough to differentiate among parties, but it does allow for intraparty differentiation, which is a key factor for the study of Uruguayan parties, given their high levels of fractionalization.

In sum, legislative speeches are not necessarily a good proxy to substitute for legislative voting using roll-call data, if we are to identify the ideological basis of political parties (Kim et al. Reference Kim, Londregan and Ratkovic2018). Instead, the fact that speech and votes are not highly correlated indicates that partisan structures are strong enough to impose certain decisions beyond the preferences or political discourse of individual legislators. If this is the case, the role of leadership within parties and factions must be subject to further research.

Concluding Remarks

This research note introduces a novel dataset and its related R package, resulting from a data generation process that involved web scraping, cleaning, and organizing more than 57,000 floor speeches in Uruguayan Parliament between 1985 and 2015. Taken as a whole, the dataset constitutes an effort to address the difficulties involved in analyzing a large amount of data coming from text and discourse, especially when researchers have to deal with multiple speakers in deliberative bodies. The first contribution this research aims to make for researchers interested in these issues is the accessibility of the data, as well as the replicability of the entire data generation process. A set of new and preexisting techniques were used to build our dataset, which we expect will create incentives to explore new applications for a wide variety of research fields, based on specific needs.

Regarding the main concern of this research note, our results indicate that quantitative text analysis reveals diverging outcomes compared to scaling methods using roll-call data. More specifically, voting patterns in the Uruguayan case seem to be a more effective method to capture the ideological preferences of political parties, compared to legislative speeches that do not reveal differentiation among parties. The divergence between scaling parties’ positions from legislative speech and vote could be due to strategic reasons, as suggested by previous works. As Zucco (Reference Zucco2013) and Izumi (Reference Izumi2016) have demonstrated, the government-opposition dimension can alter the way legislators vote regardless of their ideological or policy preferences.

Additionally, as some studies have shown for advanced and developing democracies, legislative speeches may be a more reliable source for determining the sincere policy positions of individual legislators and parties, since they are less constrained by party leaders, compared to voting behavior (Proksch and Slapin Reference Proksch and Slapin2012; Schwarz et al. Reference Schwarz, Traber and Benoit2017; Izumi and Medeiros Reference Izumi and Medeiros2021). This research note has shown that for the Uruguayan case, legislative speech shows intraparty differentiation but does not allow for differentiation between parties. Roll-call voting, in turn, shows greater potential to capture differences in party positions while also revealing important intraparty differences. Whether these differences are due to strategic reasons or to the government-opposition dimension is a relevant question that could be addressed in detail in future research.

From a methodological standpoint, our results show that scaling techniques based on text, like Wordscores or Wordfish, may not substitute for those based on roll calls, like W-nominate or other ideal point estimation techniques, and vice versa; instead, they complement each other in finding relevant differences between what legislators say and how they vote. This can be seen as our second contribution to the research on these topics, to the extent that the gap may be explained by the different goals that individual legislators face during the legislative work. These partially conflicting goals may lead legislators to balance speech and vote in a way that maximizes the benefits of party unity (e.g., belonging to a united party that constitutes a useful label for electoral purposes) while they signal differential positions to specific constituencies that are important for the legislator individually (e.g., they can help to define a personal profile for legislators seeking to send a signal to specific social groups that form the legislator’s basis of electoral support). Indeed, the divergence among scaling methods has to be seen as a puzzle open to future research, as it may indicate the role played by partisan structures, as well as leadership, in determining party discipline.