The availability of surveys on governor approval by state and time period

Collecting data on governor approval has two simple logistical issues. The first is to identify and then access governor approval data – both academic and commercial – across states and over time. In this dataset, I focus on polls with state-specific samples instead of compiling data from polls that have national-level samples which can then be reweighted to reflect state-specific population characteristics (e.g., Lax and Phillips Reference Lax and Phillips2009) because the former generally relies on larger samples of respondents for each state and does not require assumptions about correlations between attitudes and demographics being constant across states.Footnote ⁴ Some states have academic pollsters who collect and archive data on public websites. Other states have newspapers that sponsor polls and either archive them or publish them in archives that can be accessed online. Commercial survey houses then often meet market demands for timely readings of executive approval, but they are under no obligation to share their data; they may even have to honor client wishes that the data remain private or have the data available only at high prices.Footnote ⁵ Compiling these data thus requires identifying these various actors and recording the information on publicly available polls.

A second key challenge of data collection stems from the divergent conceptualizations and measurements of approval. I focus on attitudinal measures of support for the incumbent executive and do not collect data on vote intentions that reflect the popularity of the governor and also the popularity/viability of their opponent. But I cast a wide net of measures asking about the “approval,” “favorability,” or “ratings” of the executive’s “management,” “job,” “performance,” or “image.” These questions are roughly comparable to the question wordings commonly used to measure presidential approval. I also restrict our data collection to questions asking about general job performance and not their work on a specific issue (e.g., managing the economy, responding to a natural disaster, or fighting the COVID-19 pandemic) that may not be correlated with other aspects of their performance. By focusing on this narrow set of questions I hope the approval series are likely to reflect a unidimensional construct of public support for the executive.

The starting point for understanding governor approval is the JAR database compiled by Beyle, Niemi, and Sigelman (Reference Beyle, Niemi and Sigelman2002).Footnote ⁶ I extracted from the dataset measures of governor approval that are not domain specific (e.g., questions on how the governor is managing the economy) and are based on adults/voters/registered voters/likely voters and not subsamples such as just Democrats or Republicans or businesspeople. The resulting database has 5,115 unique measures of governor approval going from 1947 to 2009, but has not been updated in the last 12 years.

To extend this dataset to the present, I used internet and social media searches and communications with scholars and pollsters to identify governor approval polls conducted since the JAR was released. I have collected additional 5,200-plus measurements of governor approval from polls with exclusively, state-specific samples through 2020,Footnote ⁷ for a total of 10,328 unique observations when combined with the JAR data.Footnote ⁸ The combined dataset has data from 1,281 unique data series (series where the pollster and response options are constant).Footnote ⁹ For each survey I collected, where possible, the percentage of respondents giving the incumbent a positive rating and a negative rating. For variables with even numbers of response options, the scale is divided such that those above the midpoint are positive and those below are negative. For those with an odd number of response options, the middle category is excluded. From this, I also calculate the net approval rating of the governor (% positive–% negative) and the relative approval rating (the %positive/(% positive + % negative) that gave either a positive or a negative rating) to take into account differences generated by the presence of middle categories or nonresponse/do not know rates. If only the positive percentage or the net approval rating was reported, only that number is recorded. Thus using different measures will generate different numbers of observations.

The availability of data by state is outlined in Table 1. While all 50 states are included in the dataset, not all states have equal numbers of surveys available about them. Idaho, for example, has only had 50 publicly published measures of governor approval from 1985 to 2020. In addition to Idaho, Mississippi, North and South Dakota, Vermont, and Wyoming each average fewer than 2 surveys a year in the period since surveys first became available. Then there are other states like Michigan, New Jersey, and New York where an average year will have more than a dozen published surveys in it.Footnote ¹⁰ Then surveys are not distributed equally across years either (Figure 1); governor approval data exploded in availability after 2004 with the increase in web-based polls and online archiving and while levels of polling have remained high since they have diminished in recent years.

Table 1. Governor approval rating observations by state, SEAD 1.0

Figure 1. Distribution of survey marginals over time.

To understand some of the sources of these variations in the frequency of polling across states and within them over time, in Table 2, I model the number of measurements released in each state by quarter, starting in 1980. I focus on four sets of patterns. The first is whether polls on governors are shaped by the election calendar for governors, such that interest in the public’s evaluation of governors is limited to the lead-up to elections. Specifically, I look at whether the number of polls measuring governor approval is higher in each of the 4 quarters of a gubernatorial election year or the two quarters before a special election (e.g., a recall). I interact this timer measure with the level of competitiveness of the race when the governor was last elected and with the competitiveness of the upcoming election, each measured as the difference between the two leading candidates and with higher values representing less-competitive races because the press and the public are more likely to pay attention to competitive elections. I look at both the last election and the next election (albeit in separate models) because I do not know which temporal perspective is more likely to generate polling. For states where the next election will be held after 2020, the lead variable is coded as missing and those observations are excluded. The second explanation for polling frequency, in contrast, has nothing to do with the governor but evaluates if state-level polling infrastructure and poll timing reflect the presidential election calendar, with governor approval being added to surveys that are focused on presidential elections. Thus I look at whether there are more polls about governors in the four quarters of the presidential election year and whether this is especially true in years where the margin between the top two candidates in the previous election or the upcoming election was smaller. Third, I look at whether larger states are more likely to have more polls by controlling for the logged population of the state. Finally, I control for the temporal expansion of polls over time documented in Figure 1 by adding dummy variables for five year periods between 1980 and 2020. I model this as a negative binomial model with state-level random effects.Footnote ¹¹

Table 2. Correlates of the number of surveys on governor approval conducted in a quarter

Note. Negative binomial model with state-level random effects, standard errors are in parentheses.

* p < 0.05 (two-tailed).

The results in Table 2 show that surveys about governors are driven in part by the gubernatorial electoral calendar. Compared to an average quarter in a nongubernatorial election year, the second, third, and fourth quarters of an election year see higher levels of polling about the governor. Interestingly, it is the quarter before the election that sees the greatest increase in polling and not the election quarter itself-this reflects a decrease in polling in November and December after the election.Footnote ¹² The larger interaction terms in the second set of models suggest that the increase in polling is particularly pronounced in states where the upcoming election is expected to be tight and decreases as the margin between the two leading candidates in the next election increases. States where the previous election was tight also tend to see more polling on average, but the previous race does not shape polling dynamics in the lead-up to the next gubernatorial election. There is also an increase in polling the quarter before special elections. Thus polling about governors is often helping to inform observers about the governor’s standing in the upcoming election.

Yet Table 2 also shows that surveys about governors are shaped by electoral dynamics at the presidential level that generate polling in a state. States that are expected to be competitive electorally tend to have more polls conducted in them. There is also a distinct electoral cycle in polling, although it differs somewhat in specification across models. Model 1 suggests that as the presidential election nears, polling about governors increases in states that were competitive last time, with polling especially high in both the quarter of the presidential election and the quarter before it occurs. In contrast, the second model looking at the competitiveness of the upcoming presidential race suggests that competitive states tend to have more polls on average but that noncompetitive states are especially less likely to have polls in the quarters right before the election. Thus data availability on governor support is often a function of national-level factors that are not connected to the governor at all.

Finally, the distribution of polls is shaped by general structural factors. In particular, polling is more common in larger states than in smaller ones. Finally, the period dummies show that the frequency of polls has generally increased over time, with a particularly large increase in the period since the late 2000s.

These dynamics across states have important implications for what one can do with data over time. States with large populations, competitive presidential and gubernatorial elections, and with presidential and gubernatorial elections on nonconcurrent cycles will have richer series than will small states without competitive elections. Moreover, there is less data on governor approval the further back in time one goes. These limitations of the data are important for working with the data – in some cases, they make it impossible to estimate the measure of latent leader popularity discussed below.

Using the data to extract measurements of the latent level of governor approval over time

Despite their breadth, the data on governor approval compiled and described have limitations in their use. For example, if multiple measures of governor approval have been conducted at the same point in time and take slightly divergent values, it is unclear which is the one that scholars should use as the measure of governor approval at that period. Moreover, if one is interested in changes in approval over time instead of just levels or to see how long an event affects approval for, then one needs to not just have contemporaneous measures of approval but a way to connect them to other measures that come before or after them.Footnote ¹³ Dealing with these challenges requires a way to combine the data from various surveys into a single, ongoing measure of government approval.

To solve these issues, I treat governor approval as a latent variable that is not directly observable but is approximated through questions in public opinion surveys. Divergence across surveys can occur due to sampling variation, but surveys also have a range of methodological issues – including variations in survey question wording, survey house effects, item effects, and response effects – that can cause measures to systematically diverge. The various reported measurements of governor approval thus partially tap into the underlying concept but none of which perfectly measures it.

When estimating the value of a latent variable, one strategy is to combine data from multiple indicators overtime to separate the latent variable from measurement noise and systematic biases. Repeated observations improve the precision of estimates. Temporal variation within series helps mitigate the measurement problem inherent in survey research questions because temporal changes in attitudes measured in the same way over time reflect changes in attitudes, not particular aspects of the measurement instrument such as question wording or individuals’ interpretations of survey items (Mueller Reference Mueller1970). By focusing on how series from the same firm with a constant measurement vary over time, I can isolate the latent attitude from its measurement. Then covariation across multiple times series improves measurement reliability. Any given survey or series can be an outlier. Moreover, we should expect systematic errors owing to survey house effects (nonrandom differences in measures of approval unique to each polling firm) as well as item effects (variations in question wording biasing results in one direction or another). Aggregating among several closely related items across different surveys, however, allows the signal to stand out from the noise. If multiple series measure the same trend then it is unlikely to be a function of house effects, survey response effects, and other issues of measurement. Whether the various time series indeed tap the same underlying construct becomes an empirical question. If so, I can use the commonality in their dynamics to estimate a weighted average of executive popularity.

Combining data from multiple series also allows us to solve an additional problem that emerges from changes in the frequency of different survey series from the same state over time. Some firms poll only at election times, some periodicals that used to sponsor polls have stopped doing so, new players have entered the market but do not have historical data, and then firms occasionally change question wording. By using surveys from multiple firms and noting how they covary with each other, I can use that combination to adjust for gaps in any single series.

To combine the various series and to extract their common variance, I use the dyads-ratio algorithm, developed by Stimson (Reference Stimson1991) to extract latent variables from multiple survey questions. The algorithm, implemented in various interactions of his WCALC software package,Footnote ¹⁴ is built on the idea that if a given time series is a valid measure of approval, then the ratio between any two values in the series is a relative measure of approval over time. The approach converts all series in ratios between each observation in each series and then the algorithm assesses the common variance among the resultant ratio series while taking into account their sample size as a measure of survey precision. From this, reliability measures for each series are produced and serve two purposes: (1) they are used to weight each series in generating the latent measure of approval, and (2) they indicate whether a unidimensional construct is supported and how strongly the input series correlate with it. Then since the input series are converted to unit-less ratios, the resulting output lacks units and the final step of the algorithm uses the weighted values of the original series to recode the ratios back into levels of approval.Footnote ¹⁵

I can illustrate the need for these methods as well as the utility of the solution with data from New York. New York’s large population makes it a frequently polled state (Table 1); 19 pollsters either currently publish measures of governor approval or have regularly done so in the past. Yet, these series differ in their temporal coverage and continuity. Some began polling regularly in the 1980s while others have only begun in the last year or two. Then the survey questions being used vary substantially across surveys. Polling questions tapping governor approval have included questions such as: “Do you have a favorable or unfavorable opinion about [Governor Name]?”, “How would you rate the job that [Name] is doing as Governor? Would you rate it excellent, good, fair, or poor?”, “Do you approve or disapprove of the way Andrew Cuomo is handling his job as governor?”, and “Is your opinion of [Name] favorable, unfavorable, or haven’t you heard enough about him?” The specific response options (approve/disapprove, strongly approve/somewhat approve/somewhat disapprove/strongly disapprove, etc.) also vary by firm and/or time period. Some firms use a variety of question wordings; Siena College’s research institute, for example, has asked questions asking respondents whether they “approve or disapprove” of the governor, whether they would rate his performance as “excellent, good, only fair, or poor,” and whether they have a “favorable or unfavorable” opinion of the governor.

The variety of series available for New York governors is graphed in the first panel of Figure 2. Series differ in their length, with some being very short. Yet even if we look at the long-running series that would potentially allow for time-series analysis, they are not identical. Figure 2b, for example, combines the long-running Quinnipiac University Poll’s questions on governor approval, the Siena College Research Poll’s questions about whether the governor is viewed favorably, and Marist College’s poll asking whether the governor can be rated as doing an excellent job. Each series has certain advantages: the Marist College poll has the longest history but is asked less frequently than the other polls are, which means it misses variation, especially in recent years. The Sienna poll, in contrast, is much higher frequency but has a much shorter history and this specific question was not asked for several years in the middle of the 2000s. Finally, the Quinnipiac poll has a longer history than the Sienna poll and a higher frequency than the Marist poll but a lower temporal frequency than the Siena poll and so it, like the Marist poll, misses some changes. Then the surveys all have different point estimates of how much the public approves of the leader, which are evidence of house effects or question wording effects. So Yet the series seem to rise and fall together (with occasional outliers), suggesting that there is a common latent series.

Figure 2. Input series and the latent measure of governor approval in New York.

Figure 2. (Continued)

Using all the series in Figure 2a, I used the dyads-ratio method to generate a latent series combining them for each quarter.Footnote ¹⁶ Figure 2c graphs the latent measure which covers the entire period for which data are available and has controlled for the house and measurement effects while still capturing the variation in approval that exists over time. It captures the rise and fall of governor support due to new leaders taking office, scandals, and the recent COVID pandemic that generated a rally effect for Cuomo. The raw data points that surround the line show how the latent series reduces the impact of outliers and is less volatile than individual surveys are (in part due to aggregating to quarters instead of months) but still follows the overall ebbs and flows of the data.

I have repeated the above process for all states in the dataset and these estimated approval series are released as part of the SEAD v.1.0.Footnote ¹⁷ While I focus here on quarterly measures, the public release contains estimates at the annual level as well for scholars interested in policy outcomes that occur annually. I have estimated separate measures of the approval rating, the disapproval rating, the net approval rating, and the relative approval rating as described above. In some cases, there are too few surveys (or there is too little overlap across series) to estimate one of these series for a state; relative approval, for example, is only available for 46 states. Then even in states where the algorithm can generate a combined series, there can be temporal gaps where no polls were conducted (e.g., nonelection years) or have series that do not overlap with other series. For example, there are five states where there have been no published polls of the governor’s popularity (that I have identified) since December 2019.Footnote ¹⁸ The algorithm interpolates the old value until a new value is provided for that state. Users should consult the variable measuring the number of survey series available in a quarter in making their modeling choices; in the models below I exclude any periods where three quarters in a row were interpolated. Models looking at the effect of an event should exclude any events that occurred in quarters where opinion was interpolated to avoid downward biased estimates.

While the full set of series is available online, Figure 3 illustrates how quarterly series look in four different states to illustrate the divergence of dynamics across states. The solid lines represent the series and the hashes at the bottom denote quarters where at least one valid poll was conducted. In each state, periods without surveys where executive approval is estimated to be constant are visible. Connecticut and Florida (and New York in Figure 2) are examples of states where governor approval ebbs and flows significantly over time and, in the case of Florida, displays significant volatility from month to month. Yet the flows are quite different in each state, reflecting state-level dynamics more than national-level processes. Louisiana is also a state where governor approval has varied quite a bit (e.g., increasing massively with the transition from Jindal to Bel Edwards) but where a lack of data makes the series quite short. North Carolina, in contrast, is a state where polling is frequent but where governor approval has tended to be fairly steady. Exploring the roots of these differences across states is one potential use of these data.

Figure 3. Relative governor approval by state, 1976–2020.

An application: Do governors have honeymoon effects

To illustrate how these data can be used to explore questions of approval dynamics, I examine here whether governors have consistent honeymoons. Previous work on American presidents suggests that there is a popularity cycle whereby executives start with higher support and also often get a bump at the end of their term (Mueller Reference Mueller1973; Stimson Reference Stimson1976), a pattern that has also been documented in presidential systems outside of the United States (Carlin et al. Reference Carlin, Hartlyn, Hellwig, Love, Martinez-GallardoCarlin and Singer2018; Cuzán and Bundrick Reference Cuzán and Bundrick1997). That initial wave of warm feelings might give the executive leverage to enact their agenda early in their term. Yet there have been few systematic studies on whether governors experience a similar honeymoon and they reach divergent conclusions. Crew and Weiher (Reference Crew and Weiher1996), for example, only find a honeymoon in one of the three states they study. Beyle et al. (Reference Beyle, Niemi and Sigelman2002) document a small honeymoon effect but argue that governors’ honeymoons are weaker than presidents’ are and find substantial variation across states. Finally, King and Cohen (Reference King and Cohen2005) find that, on average, governor popularity is falling over time and then rebounds at the end of the term. Thus, the degree to which governors get a honeymoon is an open empirical question.

The mixed results regarding gubernatorial honeymoons are somewhat surprising given the robust literature on presidential honeymoons. The divergence between governors and the president may reflect important differences in the opportunities facing executives at the national and subnational levels. Governors are not as well-known as presidents, and their inauguration and early policy announcements are unlikely to be covered outside local media. Thus, governors are not launched into office with the same level of media attention that is expected to generate presidential honeymoon effects. Yet governors should gain support coming from their electoral victories and even local media coverage should generate some positive attention. Thus, I expect gubernatorial honeymoons to emerge in the data.

To answer this question, I use data from the EAD database as it is available from the period 1976 to 2019. I start in 1976 (for the states where data are available for this period) because this is when state-level unemployment estimates are first available and I stop in 2019 because the COVID pandemic dramatically changes the basis on which governors are held accountable (Singer Reference Singer2021). I measure approval in three ways. I start with the measure of relative approval that previous work (e.g., Beyle et al. Reference Beyle, Niemi and Sigelman2002; King and Cohen Reference King and Cohen2005) has suggested is the best way to take into account differences across surveys: the measure records the percentage that has a favorable opinion of the governor among all those that have an opinion. Specifically, it divides the percentage of the public who approved of the governor by the percentage that gave a positive or negative evaluation of the governor (i.e., excluding those that gave a neutral response or did not have an opinion). This measure of relative approval allows for greater comparability within a governor’s term as levels of awareness of the governor change. After modeling this series, I then model the raw approval numbers and the raw disapproval numbers separately to see if any honeymoon comes from governors receiving a bump in their approval that then fades or from governors not receiving disapproval early in their term.

To test for the presence of honeymoon effects, I follow Carlin et al. (Reference Carlin, Hartlyn, Hellwig, Love, Martinez-GallardoCarlin and Singer2018) and Chappell and Keech (Reference Chappell and Keech1985) and add dummy variables for the first, second, and third terms the governor is in office. If the governor has a honeymoon and is more popular than they would be in an average quarter, then each of these variables should be positively signed. Because Carlin et al. (Reference Carlin, Hartlyn, Hellwig, Love, Martinez-GallardoCarlin and Singer2018), Cohen and King (Reference Cohen and King2004), and Stimson (Reference Stimson1976) find that leaders also receive a slight bump at the end of their term as the outgoing executive either runs for reelection or is able to stand above the election fray, I also add a dummy variable for the last quarter of their term. Because the honeymoon is likely to be larger for new governors than for ones the public is familiar with, I differentiate the starts of first terms from the start or second/third terms. I also do not interpolate any missing values in the first three quarters because the interpolated values are likely to miss honeymoon bumps that deviate from previous/subsequent quarters and so periods are excluded if a survey was not conducted in that quarter. To ensure that results are comparable across models and do not differ due to sample composition effects I only include the 44 states where reliable series could be estimated for all three outcome variables.Footnote ¹⁹

While I focus on honeymoon effects, I control for other factors that may drive governor approval. In particular, I control for the state of the economy at the time of the survey. Previous work shows that governors are held accountable for the level of unemployment in their state, with a debate over whether the public is more strongly attuned to the absolute level of unemployment or the degree to which state unemployment levels deviate from national trends (Cohen and King Reference Cohen and King2004; Crew et al. Reference Crew, Branham, Weiher and Bernick2002; King and Cohen Reference King and Cohen2005). My analysis (not shown here) suggests that national unemployment trends do not have a significant negative effect on governor approval when state-level trends are controlled for and that results are similar when the raw unemployment levels are used instead of relative unemployment levels. I thus focus on raw unemployment levels in the analysis below. The average for the 3 months of the quarter is used to generate the quarterly measure and I include both the unemployment rate in the quarter of the survey and two lags to capture a potential delay in how the public learns about and responds to the economy. I then control for the demographic traits of the governor. While there has been no systematic study of whether governor approval differs for male and female governors, female presidents see different levels of support than male presidents (Carlin, Carreras, and Love Reference Carlin, Carreras and Love2020) and so I include a dummy variable for female governors. To see if unelected officials get a rally when they take office and also are held to different standards, I include a dummy variable for nonelected governors who took over after a governor was impeached, resigned, or died and also a dummy variable that designates the quarter that the governor was impeached/resigned or died along with two lags to see how long that effect lasts. Finally, I include a measure of the governor’s partisanship.

I estimate the basic economy models using a generalized least-squares estimator. An Im–Pesaran–ShinFootnote ²⁰ test indicates that all panels do not have a unit root and so I do not first difference the data. The Woolridge test confirms, however, that there is substantial autocorrelation in these series that needs to be adjusted for.Footnote ²¹ To account for heterogeneity in the autocorrelation structure across states, I use a panel-specific AR(1) correction (Beck and Katz Reference Beck and Katz1995). The estimated autocorrelation correction varies substantially across states, suggesting that a single pooled time correction such as a lagged dependent variable or pooled AR(1) term would not capture the serial correlation in these data.

The results in Table 3 confirm that the average governor does indeed get a honeymoon in their relative approval. In their first quarter, their relative approval is roughly 5 percentage points higher than it would be at any similar point of their term. That bump is relatively short-lived; their approval then tends to fall in their second quarter but it remains 1.8 points above its long-term trend while falling to just over 1.3 above their long-term trend in their third quarter. Any honeymoon is also limited to governors taking office for the first time; the approval bump in the first quarter of the second term is not significantly different from 0, although reelected governors do tend to have slightly lower levels of disapproval. The typical gubernatorial honeymoon is also smaller and shorter than the one received by the average newly elected president in the Americas (Carlin et al. Reference Carlin, Hartlyn, Hellwig, Love, Martinez-GallardoCarlin and Singer2018), suggesting that the media attention mechanisms drive presidential honeymoons are more limited for governors.

Table 3. Dynamics of governor approval

Note. Generalized least squares estimates with panel-specific AR(1) corrections and heteroskedastic consistent standard errors in parentheses.

* p < 0.05 (two-tailed).

The results in Model 2, however, suggest that the bump in relative approval that governors receive at the beginning of their term is not primarily driven by high levels of approval. Raw approval ratings tend to be only about 1 percentage point above their average in their first three quarters and are not significant at any point in later terms. Instead, the results in Model 3 suggest that the key driver of initial honeymoons is a substantial decrease in disapproval levels at the beginning of the term.Footnote ²² The combination of low levels of disapproval and flat approval levels suggests that those who are not disapproving are not approving of the governor either but instead are undecided, waiting to form judgments until the governor has had a few quarters to start governing.

A similar type of honeymoon emerges for governors who take office unexpectedly. Nonelected governors tend to have levels of relative approval that are roughly 2 percentage points more popular than their elected counterparts are. Governors who take office following the governor’s resignation get an additional 2+ percentage point bump in their relative approval each of their first two quarters, although that gain does not continue into the third quarter.Footnote ²³ That change is also driven by low levels of disapproval; governors who take office following a resignation have lower levels of disapproval than did their predecessor and that disapproval remains particularly low for two quarters before settling in. Governors who replace a dead governor do not receive a similar initial honeymoon effect, although that is based on a very small number (n = 6, none since 2003) of cases in this period.

The other results in Table 3 are largely consistent with the extant literature. Governor popularity tends to fall with higher unemployment, and that effect lingers for at least two quarters.Footnote ²⁴ Increasing unemployment results in almost equal levels of falling approval and increasing disapproval. There is no evidence of a gender gap among governors in these data; holding the unemployment rate constant, female governors and male governors tend to be equally popular. Then Republican governors in this period tend to be less popular than Democratic governors, although the difference is less than a percentage point on average and becomes insignificant in the period since 2010. Further work using these data can add events that are not tied to the electoral calendar (e.g., scandals, disasters) to model the magnitude and length of their effects.

Taken together, the results in Table 3 show that the public does not disapprove of governors at the beginning of their term but instead withholds judgment about them until they have acted. That high relative approval can be potentially leveraged as a mandate to enact their policy priorities but will fade if the public does not like the results. Because any bump is relatively small and short-lived, governors looking to maintain their popularity have to deliver solid policy performance to have their popularity continue.