MEASURING DEMOCRACY

Evaluating the relative strength and weakness of democracy indices seems to come into vogue every decade or so among political scientists (Boese Reference Boese2019; Coppedge et al. Reference Coppedge, Gerring, Altman, Bernhard, Fish, Hicken, Kroeig, Lindberg, McMann, Paxton, Semetko, Skaaning, Staton and Teorell2011; Gerring et al. Reference Gerring, Knutsen, Maguire, Skaaning, Teorell and Coppedge2021; Levitsky and Way Reference Levitsky and Way2015; Munck and Verkuilen Reference Munck and Verkuilen2002). The critiques are well intentioned and made in the pursuit of a laudable goal: that is, to clearly explain the global state of democracy currently and over time. Freedom House shares this goal.

The first step in measuring democracy is to establish a conceptual definition. Little and Meng (Reference Little and Meng2023) adopt a quasi-minimalist definition, arguing that democracy—although multidimensional—primarily is characterized by leaders and political parties losing elections. Other components of democracy that may matter (e.g., civil liberties), they suggest, are implicitly included in measures of electoral competitiveness because restrictions on these associated rights eventually should affect electoral outcomes.

Freedom House, conversely, uses a definition of democracy based on fair, contested elections in which universally enfranchised voters can exercise their right to choose who governs free from improper influence and in which the judiciary, media, and civil society can reasonably act as a check on the actions of elected officials (Repucci and Slipowitz Reference Repucci and Slipowitz2022). Although it is different from that used by Little and Meng, this definition of democracy also falls well within established scholarship (Collier and Levitsky Reference Collier and Levitsky1997; Coppedge et al. Reference Coppedge, Gerring, Altman, Bernhard, Fish, Hicken, Kroeig, Lindberg, McMann, Paxton, Semetko, Skaaning, Staton and Teorell2011; Dahl Reference Dahl1971; Diamond Reference Diamond1999; Karl and Schmitter Reference Karl and Schmitter1991; Linz and Stepan Reference Linz and Stepan1996; Platter Reference Platter2002).

To measure the level of global democracy and freedom, Freedom House uses a set of indicators to evaluate political rights and civil liberties. The result is Freedom in the World, a survey of countries and territories issued annually since 1973. The report’s methodology is derived from the Universal Declaration of Human Rights and includes 25 indicators. In political rights, the indicators are grouped into three subcategories: (1) the electoral process, (2) political pluralism and participation, and (3) the functioning of government. There also is an additional discretionary question measuring efforts, if they exist, to change the ethnic composition of a country or territory. The civil liberties indicators are grouped into four subcategories: (1) freedom of expression and belief; (2) associational and organizational rights; (3) the rule of law; and (4) personal autonomy and individual rights. Each indicator is scored between 0 and 4, and each country and territory receives an aggregate score from a total of 100. Equally weighing and combining scores on political rights and civil liberties determines each country and territory’s status as Free, Partly Free, or Not Free. Freedom House assessments, which incorporate both elections and other components of democracy, reveal a gradual but steady decline in global freedom for the past 17 years.

Could the difference in the empirical trend that Little and Meng identify and the one reported by Freedom House be explained by the bias of expert coders? In their article, Little and Meng (Reference Little and Meng2023) suggest that a media echo chamber that is sounding alarms about the threat of democratic decline may be influencing coders because “many expert coders and commentators about democratic backsliding are based in the United States.”

However, the majority of the Freedom House analysts are based outside of the United States. The cohort of country analysts and expert advisors changes slightly every year, but typically numbers between 150 and 170 individuals. Freedom House staff make a concerted effort to find individuals who are living in, working in, studying in, or hailing from the countries and territories that they are assessing for the survey. In 2022, 72% of analysts were located outside of the United States.

More important, using a broad array of indicators that require the input of expert analysts does not mean that Freedom House’s assessments are not factual or based on empirical evidence. Expert evaluations are not, as Little and Meng (Reference Little and Meng2023) suggest, “subjective impressions.” To ensure this, Freedom House uses a transparent process to gather evidence of changes in global freedom and to review draft evaluations produced by expert analysts.

Freedom in the World reports and the aggregate trends that are summarized in them are produced from individual country and territory assessments. These assessments, in the form of draft narratives and score-change proposals, are updated and extensively reviewed annually by analysts. In addition to individual analysts and regional advisors—both external to Freedom House—the review process includes in-house researchers and staff from the organization’s international programs teams who work on the ground around the world. Proposed score changes for each country and territory are discussed in annual rating-review meetings. To improve or decrease any indicator and therefore change the score of a country or territory, the following conditions must be met: (1) the change is triggered by a discrete event or series of events during the coverage period or the final culmination of a more gradual change; (2) it is a real-world change rather than a policy, legislative, or legal proposal or the stated intention of state or nonstate actors; and (3) there is empirical evidence of the effect that the change has produced. This threshold for evidence means that to result in score changes, proposals require empirical proof of real-world developments. To ensure transparency about this decision-making process, Freedom House has included an explanation for an indicator’s score change at the end of the indicator’s description in every country and territory report since the 2018 edition (Freedom in the World Methodology 2018).

To summarize, Freedom House defines democracy broadly to include both political rights and civil liberties, and it evaluates countries and territories based on a methodology that assesses the real-world experience of people living around the globe. To produce scores on all 25 indicators, country analysts are recruited, and their proposed assessments are reviewed through an iterative process that includes Freedom House’s research and programmatic staff and external regional advisors. To result in actual score changes, assessments proposed by analysts must include empirical evidence of changes to political rights and civil liberties. Explanations for score changes are a part of all country reports, which are published on the Freedom House website.

Translating the social world into a set of indicators is neither simple nor straightforward. However, this does not mean that researchers must abandon trying to appraise aspects of democracy and freedom that may be especially difficult to measure quantitatively. Assessing civil liberties contributes important and much-needed context to our understanding of political institutions and the nature of democratic backsliding.

EVIDENCE OF BACKSLIDING

According to Freedom in the World data, the high point for global freedom was between 2002 and 2008, when approximately 46% of the countries in the world were rated Free (figure 1) and the average aggregate score hovered around 62 (Gorokhovskaia Reference Gorokhovskaia2023). Since then, both the percentage of Free countries and the average aggregate score has gradually but steadily fallen: in 2022, 43% of countries were rated Free and the average aggregate score was 57.9.

Figure 1 Percentages of Free, Partly Free, and Not Free Countries, 1972–2022

Backsliding has been uneven across political rights and civil liberties. Whereas the indicators for electoral process, which include both the administration of elections and measures of electoral competitiveness, have generally remained unchanged—even improving slightly between 2006 and 2015—freedom of expression and belief, the rule of law, and associational and organizational rights have significantly deteriorated (figures 2 and 3). Disaggregating evidence for global backsliding in this way brings Freedom House’s data into closer alignment with Little and Meng’s findings that show little evidence of backsliding on electoral indicators but some evidence of backsliding on media freedom.

Figure 2 Average Scores on Political Rights Indicators, 2005–2022

For ease of comparison, the average scores on each indicator category have been rescaled to have the same upper and lower limits.

Figure 3 Average Scores on Civil Liberties Indicators, 2005–2022

For ease of comparison, the average scores on each indicator category have been rescaled to have the same upper and lower limits.

It is important to consider the role of backsliding outside of elections for several reasons. First, deterioration in civil liberties that results from attempts made by leaders to stay in power can be harmful and long-lasting even when it does not actually result in would-be autocrats winning elections. We only need to look at the aftermath of President Jair Bolsonaro’s defeat in Brazil in 2022 to observe the violent consequences of attacks on rights in a country where elections continue to result in the alternation of those in power. Although elections are foundational pillars of democracy, associated rights and freedoms are crucial to their continued functioning as democratic instruments.

Second, as Meng and Little (Reference Little and Meng2023) acknowledge, a decline in civil liberties eventually can impact the quality of elections. Attacks on freedom of expression, for example, can degrade the information available to voters, and restrictions on the freedom of association may make it less likely that political opponents to incumbent leaders emerge or can generate social support. Levitsky and Way (Reference Levitsky and Way2010) argued more than a decade ago that despite the fact that the “formal architecture of democracy” in the form of elections had diffused across the globe, not all countries with multiparty elections were democracies. Respect for civil liberties is one of the three conditions that differentiate democracies from other polities with elections, such as competitive authoritarian regimes: “By raising the cost of opposition activity (thereby convincing all but the boldest activists to remain on the sidelines) and critical media coverage (thereby encouraging self-censorship), even intermittent civil-liberties violations can seriously hinder the opposition’s capacity to organize and challenge the government” (Levitsky and Way Reference Levitsky and Way2010, 9). Degradation of rights outside of elections often takes place in advance of the decline of electoral competitiveness, and it is important to monitor and evaluate.

The third reason that backsliding in the arena of civil liberties is important to track is because it impinges on people’s lived experience. Censorship, discrimination, attacks on culture and religion, prohibitions on the right to protest, and arbitrary criminal prosecutions—to name only a few problems faced by people around the world—matter not only because they may make polities less democratic but also because they diminish people’s human rights. In Freedom House parlance: they make people less free. Concern about the impact of rights restrictions on people was a founding feature of the Freedom in the World survey as far back as the earliest reports, which describe these rights as “themselves essential aspects of freedom” (Gastil Reference Gastil1978, 119). Tracking these attacks in real time and raising the alarm about them is another way to combat further degradation. Freedom House is an organization that provides analysis but also seeks to empower individuals to exercise their fundamental rights through advocacy and direct support to frontline defenders of freedom.

It is important to note that the fact that Freedom House—and other like-minded organizations—views attacks on civil liberties as an issue of advocacy does not mean that the measurement of these freedoms is guided by advocacy concerns or is impervious to critical feedback. Neither does it mean that Freedom House conducts research entirely apart from academic scholarship. Periodically, the research team invites academic researchers, with both qualitative and quantitative backgrounds, to review the methodology used by Freedom in the World. One collaboration resulted in Freedom House establishing a formula for designating a country as an “electoral democracy.” In 2016, a methodology review of Freedom in the World, conducted by a team of 20 external experts, resulted in the refinement of existing indicator definitions and the inclusion of gender-related guidance in questions for relevant indicators. More can be done, of course. One challenging aspect for Freedom House is ensuring that data and the annual evaluation process are understood and can be accessed by those outside of the organization. Limits on available resources mean that aspects of the process can be less transparent than is desirable. Another challenge, for both Freedom House and other researchers working to globally assess political rights and civil liberties, is efforts by governments to deliberately limit access to information through censorship, digital controls, and the arrest and intimidation of members of civil society. Concerns about data availability and reliability emerge during the report cycle every year.

Finally, it is important to understand how backsliding on civil liberties looks in practice. The only civil liberty that Little and Meng (Reference Little and Meng2023) examine is freedom of the media. Guided by a focus on finding objective measures, they examine only punitive actions against journalists. They use data on the number of journalists murdered or imprisoned as documented by the Committee to Protect Journalists (CPJ). Based on these two indicators, Little and Meng find that it is difficult to conclude whether freedom of expression is facing a serious global threat because the number of journalists murdered is declining and the number of those imprisoned is increasing.

There are obvious problems with relying solely on this type of data. Based only on the murders of journalists, for example, we could state that media freedom in Russia has improved during the past decade because many more journalists were killed before 2010 (i.e., 31, according to CPJ) than after 2010 (i.e., six, according to CPJ). Of course, media freedom in Russia actually deteriorated severely during the past decade, and independent media now exists only outside of the country. However, there are other more fundamental issues with limiting the data for indicators in this way.

Many civil liberties, including media freedom, are challenging to measure. Phenomena that are difficult to count, however, are not necessarily less objective or factual and neither are they less important as causes of democratic backsliding. Media freedom is a subcategory of civil liberties assessed by Freedom in the World. The indicator includes violence against journalists, but it also considers other factors such as censorship, surveillance, criminal prosecutions and civil suits against journalists and media outlets, discriminatory control of broadcasting and publication licenses, and nonfatal threats and violence against journalists. These other factors are included because they present a more complete picture. Based on this holistic approach, Freedom House’s data show that freedom of expression—which includes not only media freedom but also freedom of private expression, academic freedom, and freedom of religion—has declined more than all other civil liberties since 2005.

The case of Nicaragua is an illustrative example of the difference between Freedom House’s approach and that of Little and Meng. Since 1993, two journalists have been murdered in Nicaragua, according to data from CPJ, and two journalists currently are imprisoned. These datapoints no doubt are concerning. However, there is more happening in the country, which in 2022 received a Freedom in the World score of 0 out of 4 on media freedom.

Authorities have cracked down on the media since President Daniel Ortega returned to power in 2007. In 2021, the government oversaw the closure of 20 independent news outlets; suspended the transmission of two local radio shows that criticized officials; and even seized ink and paper from newspapers, thereby preventing them from printing their circulations. Like other authoritarian regimes, Nicaragua’s government also adopted laws that make it possible to prosecute journalists for spreading “fake news,” which resulted in an escalation of reported self-censorship among independent journalists. All of these measures, which fall short of killing and imprisoning journalists, should concern researchers, civil society, and policy makers. Including them in the analysis of democracy and freedom helps us to understand democratic decline in Nicaragua but also in other countries, where these tactics already may be happening or are on the horizon.