“To the extent possible—even if it requires a bit of imagination—members will picture themselves as effective users of inside power [when meeting with constituents].”—Fenno (Reference Fenno1978, 137)
“The Center for Effective Lawmaking named Joe Donnelly the least effective Democrat lawmaker in the United States Senate…. You can send a really incredible swamp-person back to the Senate, like Joe Donnelly, or you can send us Republicans like Mike Braun to drain the swamp.”—President Trump (May 2018)
When seeking reelection, legislators point to their legislative votes and other forms of position taking (Mayhew, Reference Mayhew1974), their constituency service, and their successes as effective lawmakers to garner favor with their constituents. However, despite extensive evidence that members of the US Congress differ substantially from one another in their ability to advance policy proposals (e.g., Volden and Wiseman, Reference Volden and Wiseman2014), voters do not seem to be particularly active in holding their representatives accountable based on lawmaking effectiveness.
Indeed, over the past four decades, the average reelection vote share for a member of the US House of Representatives with a Legislative Effectiveness Score (LES) above her party's median is 70.3 percent, compared to a nearly identical 69.6 percent vote share for those below their party's median in effectiveness.Footnote 1 Alternatively, if we divide members of the House into three groups—highly effective lawmakers, average in effectiveness, or ineffective (relative to comparable lawmakers)Footnote 2—we find that the reelection vote shares are 68.0, 68.9, 69.5 percent, respectively. Put simply, effectiveness in lawmaking is not rewarded in the general election; voters do not seem to hold ineffective lawmakers accountable for their poor performances in advancing bills.
Such patterns could arise from voters not knowing how effective their lawmakers are, or from voters not caring about effectiveness in lawmaking. In this paper, we engage with these two possibilities by analyzing survey data from Amazon's Mechanical Turk (MTurk) and from the 2016 Cooperative Congressional Election Study (CCES). We demonstrate that most constituents are unable to discern whether their own representative has been highly effective or ineffective as a lawmaker.
When constituents are presented with information about their representative's legislative effectiveness, however, we find that they become more likely to approve of the effective lawmakers, and less likely to approve of ineffective lawmakers. Moreover, perhaps surprisingly, partisans are willing to punish ineffective representatives who are copartisans, and they are also willing to reward highly effective representatives from the other party. That said, ideological moderates are much more responsive to evidence of lawmaking effectiveness than are extremists; and the size of this effect approaches the typical boost in approval arising from legislators being from the same party as the constituent.
Our results suggest that, although most Americans do not have much information about representatives' lawmaking effectiveness, constituents do employ credible information about effectiveness when it is presented to them, and that informed voters' evaluations of incumbents are positively correlated with their representatives' effectiveness.
1. Legislative effectiveness and constituent approval
To explore how legislators' successes at moving their bills through the legislative process affect voters' evaluations of their performances, we use Volden and Wiseman's (Reference Volden and Wiseman2014) LES. The LES is a comprehensive metric based on all bills a lawmaker sponsors, how far they progress toward law, and how substantial each bill is. We then bin each lawmaker as ineffective, average, or highly effective, as described above. This approach maps onto the sorts of considerations voters would likely weigh if they were informed and interested in lawmaking effectiveness. For example, representatives who are more active in lawmaking score higher; and the LES also places less weight on commemorative measures, in comparison to more substantive or high-profile legislation.
Despite this likely alignment between a lawmaker's LES and her constituents' evaluations of her performance, in many cases voters might not have credible information about who is an effective lawmaker, perhaps due to them lacking motivation to investigate lawmaking effectiveness (i.e., Lupia, Reference Lupia2015). Moreover, legislators themselves are often the main source of information about their own relative effectiveness, offering biased information that constituents likely discount. Hence, one might expect that there is often no meaningful relationship between a legislator's actual lawmaking effectiveness and her constituents' beliefs about her lawmaking effectiveness.
Extant research would suggest, however, that credible sources could provide information in an accessible manner that may influence constituents' views (Lupia, Reference Lupia2015). In the current context, we seek to identify whether constituents who are exposed to information from independent and non-partisan researchers will support legislators who are described as “highly effective,” while providing less support to ineffective lawmakers.
Because partisanship serves as a perceptual lens (i.e., Lodge and Taber, Reference Lodge and Taber2013) that attenuates (or exacerbates) the effects of information provision (e.g., Sulkin et al., Reference Sulkin, Testa and Usry2015), we also seek to identify whether information that casts a copartisan in a negative light is less likely to be accepted by strong partisans, while negative information about legislators from the other party might be especially persuasive (i.e., Anderson and Harbridge, Reference Anderson and Harbridge2014; Ashton and Munis, Reference Ashton and Munis2021). Relatedly, we also seek to assess whether ideological preferences influence constituents' responses to congressional behavior (e.g., Harbridge and Malhotra, Reference Harbridge and Malhotra2011), such that extreme conservatives, for example, may not approve of effective liberal lawmakers. More generally, we might expect that the effect of credible information about lawmaking effectiveness will be greatest among ideologically moderate constituents, who are open to supporting effective lawmakers across partisan and ideological lines.
2. What does the public know about the lawmaking effectiveness of their representatives?
To explore these conjectures, we recruited a sample of 1035 respondents from Amazon's Mechanical Turk in December 2015 (see Supplemental Appendix 1 for details about the sample). We also replicated the same experiment with a national sample of 1049 voters that was part of the 2016 CCES (Ansolabehere and Rivers, Reference Ansolabehere and Rivers2013), administered just before the November 2016 election. For both samples, we asked the subjects questions about their own perceptions of their representative's lawmaking effectiveness. To facilitate this analysis, we identified who a subject's US House Representative was, using address and zipcode information. After identifying the appropriate Representative, subjects were asked the following question:
How effective do you think [REPRESENTATIVE] is, in terms of passing laws?
Highly effective
About average in effectiveness
Not at all effective
The language about “effectiveness” has a clear normative element to it. One could instead imagine selecting more neutral terms, such as “productivity” or “legislative activity.” However, given that accountability discussions are most likely to play out in electoral campaigns, during which clearly normative labels are used repeatedly (as the introductory quote from President Trump illustrates),Footnote 3 our approach better mimics the nature of public discussions than would a more value-neutral approach. The three response options (highly effective versus average versus not at all) mimic the treatment given below.
Figure 1 presents the analysis from respondents in the control group, who did not receive any information in the survey about their representative's lawmaking effectiveness. The figure shows that constituents' perceptions of effectiveness are not correlated with existing measures of legislative effectiveness. Rather, most respondents thought that their representative was “average” in lawmaking effectiveness, regardless of her actual effectiveness.Footnote 4 Taken together, the results are quite consistent with our first conjecture, as there is no clear relationship (p > 0.50) between a respondent's perception of his representative's lawmaking effectiveness and the legislator's actual effectiveness.
Figure 1. Knowledge about lawmaker effectiveness (control group).
Note: Data from two national survey samples show that most constituents label their representative as “Average” in lawmaking effectiveness regardless of their actual effectiveness (based on the bills they sponsored). Correlations between perceived and actual effectiveness were not statistically significant (p > 0.50 in both surveys).
3. Are Americans' opinions about their representatives responsive to new information?
Building on this baseline, we conducted the following survey experiment on the MTurk and CCES samples. Before a subject was asked about approval of his or her representative, we randomized whether the respondent received information about the representative's actual level of lawmaking effectiveness (based on the representative's LES).Footnote 5 More specifically, all subjects were exposed to the following information, where the $-sign represents the inclusion of lawmaker-specific information and where the text in brackets was varied across treatment conditions:
As you may know, your representative, $Representative, is a $Party, who has served for $Years years in Congress.
[CONTROL/PLACEBO/TREATMENT TEXT AND FIGURES]
For the experiment, a quarter of the sample received no information about effectiveness. Another quarter of the sample was assigned to a placebo condition that discussed legislative effectiveness but did not give subjects any information about their own representative's lawmaking effectiveness. Instead, these respondents were given the following additional prompt: “Members of Congress differ in how effective they are at moving their proposed bills into law”; and they were presented with the image shown in Figure 2.
Figure 2. Image for placebo condition in survey experiment.
Because we found no differences in relevant responses between subjects assigned to the placebo and the pure control conditions, we combine these two conditions in the analyses reported here, using them together as the baseline control group. The remaining subjects were presented with the following prompt:Footnote 6
Members of Congress differ in how effective they are at moving their proposed bills into law.
A team of independent and non-partisan researchers recently found that your representative, $Representative, is $Effectiveness compared to others in Congress.
In addition to the prompt, subjects were also presented with one of the images in Figure 3 that illustrated the representative's actual relative lawmaking effectiveness.
Figure 3. Images for the informed (treatment) condition.
The information treatments were designed with the intention of maximizing their credibility. First, we emphasized that the data came from a “team of independent and non-partisan researchers.” Second, we employed bar charts when presenting the data because individuals often perceive bar charts to be more scientific than text alone (Tal and Wansink, Reference Tal and Wansink2016). Taken together, these choices were made to increase the credibility of the information beyond what might be inferred by subjects if it had been provided by a more biased or less sophisticated source (Lupia, Reference Lupia2015). Offering this strong treatment also established a firm baseline from which to explore the conditional or moderating effects of partisanship and/or ideology.
Figure 4 displays the treatment effect (along with 95 percent confidence intervals) of receiving information about effectiveness on approval.Footnote 7 Each model was run separately for each subset of lawmaker type (highly effective, average, ineffective); and each included as the sole independent variable whether the treatment information was provided.Footnote 8
Figure 4. Informational effects on approval of lawmaker.
Note: The figure shows the marginal effects (along with 95 percent confidence intervals) of information provision in two national survey experiments, analyzed in OLS regression models.
Figure 4 shows that the information affected subjects' approval levels in expected ways. First, it is clear that the approval levels for representatives who were revealed to be “average” in effectiveness do not differ between treatment and control groups (p = 0.71 and p = 0.85 in the MTurk and CCES samples, respectively).Footnote 9 Such a finding is unsurprising, given that the vast majority of control group subjects already thought that their legislator was average in terms of lawmaking effectiveness (see Figure 1).
In contrast, information provision has a notable effect on approval for representatives who are either “highly effective” or “ineffective.” More specifically, when a constituent learns that his or her representative is “highly” effective, the constituent is somewhat more likely to approve of her legislator (p < 0.01 for the MTurk sample, but p = 0.11 for the CCES sample). The effect of information for ineffective legislators is even larger in magnitude, consistent with research on individuals' heightened attention to negative information (Soroka, Reference Soroka2014; Boydstun et al., Reference Boydstun, Ledgerwood and Sparks2019; Ashton and Munis, Reference Ashton and Munis2021). Constituents who learn that their representatives are ineffective are about 25–30 percentage points less likely to approve of the job they are doing (p < 0.01 in both studies).
Taken together, we see that across treated subjects, an approval gap of nearly 40 percentage points emerged between ineffective and highly effective lawmakers (illustrated further in Figure A1 in the Supplemental Appendix). Moreover, as shown in the Supplemental Appendix (see Figure A4), a sizable gap also emerges in the percent of informed voters who say that they intend to vote for the lawmaker for reelection (with voters much more likely to support effective over ineffective representatives). That gap is 30 percentage points in the MTurk study and nearly 15 percentage points in the CCES.
3.1 Does partisanship or ideology moderate the effect of information?
To assess whether these informational effects depend on voters' partisanship or ideology, we begin by presenting the relationship between job approval and information provision (Figure 5), broken down by whether the respondent and her representative are copartisans. For these analyses, we only included the subjects who self-identified as being either Republicans or Democrats (including those who “lean” towards one of the parties). Taken together, the results suggest that partisanship moderates the effect of information provision on representatives' job approval, but in an unexpected way.
Figure 5. Treatment effects on approval, by partisanship.
Note: The figure shows the marginal effects (along with 95 percent confidence intervals) of information provision to constituents who are copartisan (or outpartisan) with their representative in Congress. Analyses are based on two national survey experiments, analyzed in OLS regression models. The results show a drop in approval of ineffective copartisan lawmakers and an increase in approval of highly effective representatives from the opposing party.
As seen in both panels of Figure 5, when copartisans learn that their representative is ineffective, they are about 40 percentage points less likely to approve of their job performance (p < 0.01). As Figure A2 in the Appendix shows, 85–90 percent of uninformed copartisans approve of their legislator, dropping down to about 45–50 percent for those who learn they have an ineffective representative. Similarly, constituents from the other party are somewhat more likely to approve of a legislator if they learn that the legislator is effective (p = 0.02 for the MTurk sample, but p = 0.63 for the CCES sample). In other words, partisans are not turning a blind eye to the information that does not comport with their partisan worldview. Rather, in most cases, copartisans respond to the new information and incorporate it into their views about their representatives.
While the results in Figure 5 might be interpreted to mean that copartisans are not meaningfully rewarding effective lawmakers, it is important to realize that these results likely suffer from a ceiling effect on the expected magnitude of copartisans' attitude change. Because 80–90 percent of copartisans already approve of their representatives, any positive information can only produce a limited systematic shift in constituents' attitudes.Footnote 10 That said, interactions explored in the Supplemental Appendix show no evidence of partisanship limiting constituent responsiveness to the new information. In sum, these results lend further evidence to the claim that providing constituents with credible information about their representatives' lawmaking effectiveness produces sizable attitude shifts, even accounting for party attachments. Indeed, upon giving the information treatment, we find no statistically significant difference in approval between an ineffective representative of one's own party and a highly effective lawmaker in the opposing party.
Turning to questions regarding voters' ideology, Figure 6 presents treatment effects of information on job approval, now broken down by whether the respondent is ideologically extreme or moderate. To bin respondents into these two groups, we rely on self-reported ideology on a seven-point scale. Those reporting that they are Very Liberal, Liberal, Conservative, or Very Conservative are characterized as ideologically extreme, whereas those self-identified as Somewhat Liberal, Middle of the Road, or Somewhat Conservative are here characterized as moderate.Footnote 11 Consistent with our conjecture above, the results point to a greater informational effect among moderates.
Figure 6. Treatment effects on approval, by ideological extremism.
Note: The figure shows the marginal effects (along with 95 percent confidence intervals) of information provision to moderate and ideologically extreme constituents. Analyses are based on two national survey experiments, analyzed in OLS regression models. The results show somewhat larger informational effects for moderates than for extremists.
Specifically, providing information about a highly effective lawmaker is associated with about a 22-percentage-point rise in approval for moderates, but a smaller or nonexistent (for the CCES sample) rise for extremists. Information about an ineffective representative results in more than a 30-percentage-point drop in approval among moderates, compared to about 20 percentage points among extremists. A joint test of these coefficients shows a significantly larger informational effect for moderates than for extremists on the whole (p < 0.05).
4. Conclusions and future directions
Americans hold strong partisan views, with much higher approval ratings of copartisans than of representatives of an opposing party. In contrast, there is little evidence that constituents base their approval or voting behavior on how well their representatives do their jobs as effective lawmakers. In this paper, we establish that such a disconnect comes from lack of information, rather than a lack of concern about effective lawmaking.
Absent information about lawmaking effectiveness, we find about a 60 percentage-point gap in constituent approval between copartisan and out-partisan representatives. In some cases, upon providing constituents with credible information, nearly as large of a gap opens up in respondents' approval between ineffective and highly effective lawmakers. Among ideologically moderate constituents, that gap is 50 percentage points. It is about 40 percentage points among all responding citizens and voters, including among those represented by a copartisan. And it is 30 percentage points among ideologically extreme voters and among those represented by a member of the opposing political party. In contrast, those who are not given credible information about the lawmaking effectiveness of their representatives display no difference in their approval of effective and ineffective lawmakers.
While the findings reported here appear to be strong and robust across different settings and samples, they do raise a variety of questions for future research. First, how are the treatment effect sizes influenced by the extent to which subjects value lawmaking effectiveness over other qualities that they consider when evaluating their representatives? On a related note, what components of lawmaking effectiveness are deemed to be most valuable to voters? Future work, exploring effectiveness in policy areas of greatest interest to specific constituents, and exploring success in specific stages of lawmaking, especially in contrast to other elements of effective representation, would be welcome.
Second, we worked hard to offer credible information from a trustworthy source. Yet many informational sources are questioned and diminished in current polarized political discussions. Additional work exploring effectiveness information provided by different sources—such as interest groups or newspaper endorsements—may be enlightening.
Finally, although additional survey experiments can help to identify these relative effect sizes, field experiments may be needed to establish the external validity of the results explored here (especially overall effect sizes). As objective information on the lawmaking effectiveness of members of Congress becomes increasingly available, interest in exploring the role of information provision in electoral campaigns regarding this important aspect of representation will surely grow.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/psrm.2021.66. To obtain replication material for this article, please visit https://doi.org/10.7910/DVN/5EO10I.
Acknowledgement
The authors thank the Bankard Fund at the University of Virginia for financial support. Volden and Wiseman thank the US Democracy Program of the Hewlett Foundation and the Democracy Fund for ongoing support of the Center for Effective Lawmaking. The 2016 Cooperative Congressional Election Study (CCES) was supported by the National Science Foundation, Award #1559125. Thanks to seminar and conference participants at American Political Science Association Conference (San Francisco), American University, Columbia University, European Political Science Association Conference (Milan), University of North Carolina, Pew Research Center, University of Virginia, and Yale University (CSAP Summer Conference), as well as Nicholas J. G. Winter, Laurel Harbridge Yong, Christopher Kam, and Rob Van Houweling for thoughtful and constructive comments. Please direct all comments and questions to [email protected].
