Understanding the role that racial prejudice plays in American politics is a major challenge for political scientists. Until recently, research relied primarily on explicit measures of racial prejudice. However, these measures are vulnerable to social desirability bias. As norms of egalitarianism have grown increasingly entrenched in American society, some prejudiced people have become less willing to openly express discriminatory attitudes against members of other racial or ethnic groups (Huddy and Feldman, Reference Huddy and Feldman2009).

Facing this challenge, the introduction of measures of implicit prejudice promised a breakthrough. The Implicit Association Test (IAT) developed by Reference Greenwald, McGhee and SchwartzGreenwald et al., for example, was designed to reveal “race-related stereotypes and attitudes that are consciously disavowed by the subjects who display them” (Greenwald et al., Reference Greenwald, McGhee and Schwartz1998, p. 1473). Similarly, the Affect Misattribution Procedure (AMP) claimed “to circumvent responses strategies $\ldots$ among individuals or in situations where motivational pressures are high” (Payne et al., Reference Payne, Cheng, Govorun and Stewart2005, p. 289). The IAT, the BIAT (a shorter version of the IAT developed by Sriram and Greenwald (Reference Sriram and Greenwald2009)), and the AMP were quickly embraced by political scientists as “alternative survey techniques for measuring racial prejudice in ways that circumvent social desirability pressures” (Segura and Valenzuela, Reference Segura and Valenzuela2010, p. 501).

Implicit measures of prejudice gained substantial traction in political science when Barack Obama ran for president in 2008, with scholars launching innovative studies designed to uncover the impact of hidden or subconscious “anti-African American racism” on voting behavior (Pasek et al., Reference Pasek, Tahk, Lelkes, Krosnick, Payne, Akhtar and Tompson2009; see also Greenwald et al., Reference Greenwald, Smith, Sriram, B.-A. and Nosek2009; Finn and Glaser, Reference Finn and Glaser2010; Payne et al., Reference Payne, Krosnick, Pasek, Lelkes, Akhtar and Tompson2010; Segura and Valenzuela, Reference Segura and Valenzuela2010; Messing et al., Reference Messing, Jabon and Plaut2016). Subsequent research used implicit measures to understand prejudice toward other marginalized groups including immigrants (e.g., Malhotra et al., Reference Malhotra, Margalit and Mo2013), women (e.g., Mo, Reference Mo2015), and Hispanics (e.g., Kam, Reference Kam2007; Pérez, Reference Pérez2010, Reference Pérez2016). Across these studies, implicit measures were seen as “very useful in studying politically relevant attitudes that individuals will often want to misrepresent” (Malhotra et al., Reference Malhotra, Margalit and Mo2013).

In social psychology, however, the inevitable cycle of critical assessment following innovation had begun. Early on, scholars waged that the IAT may capture stereotype awareness rather than endorsement (Arkes and Tetlock, Reference Arkes and Tetlock2004). Now, both proponents and critics of implicit measures in social psychology agree that scores on implicit prejudice measures should not be interpreted as capturing racial prejudice that is unaffected by social desirability pressures (e.g., Fazio and Olson, Reference Fazio and Olson2003; De Houwer et al., Reference De Houwer, Beckers and Moors2007; Ito et al., Reference Ito, Friedman, Bartholow, Correll, Loersch, Altamirano and Miyake2015; Gawronski et al., Reference Gawronski, Morrison, Phills and Galdi2017). Rather, implicit prejudice may reflect just one side of a dual-process model (e.g., Gawronski and Bodenhausen, Reference Gawronski and Bodenhausen2006; Gregg et al., Reference Gregg, Seibt and Banaji2006)—a separate cognitive system, with fundamentally different consequences for behavior than explicit prejudice (e.g., Dovidio et al., Reference Dovidio, Kawakami and Gaertner2002), especially in the context of sensitive topics like race and discrimination (e.g., Greenwald et al., Reference Greenwald, Poehlman, Uhlmann and Banaji2009). Many social psychologists, including the developers of the two measures of implicit prejudice most widely used in political science, the IAT/BIAT and the AMP, also agree that the concept of “implicit prejudice” should be discarded because it is systematically ambiguous and misleading (e.g., Greenwald and Banaji, Reference Greenwald and Banaji2017; Payne et al., Reference Payne, Vuletich and Lundberg2017; Corneille and Hütter, Reference Corneille and Hütter2020).

For their part, political scientists have demonstrated that when explicit and implicit measures of prejudice are pitted against one another, explicit measures better predict outcomes of most interest to political scientists, such as vote choice or policy views (Ditonto et al., Reference Ditonto, Lau and Sears2013; Kalmoe and Piston, Reference Kalmoe and Piston2013; Kinder and Ryan, Reference Kinder and Ryan2017). Nevertheless, studies of racial attitudes and their political effects—including those published in the discipline's top journals—have continued to incorporate the IAT, the BIAT, or the AMP as prejudice measures on the grounds they that circumvent the underreporting of racial bias caused by “normative pressures facing respondents asked explicit questions about race relations” (Iyengar and Westwood, Reference Iyengar and Westwood2015, p. 696; see also Valentino et al, Reference Valentino, Neuner and Vandenbroek2018; Chudy, Reference Chudy2021; Engelhardt, Reference Engelhardt2021). The objective of this study is to assess the credibility of claims that the measures of implicit prejudice most commonly used by political scientists, the BIAT and the AMP, provide valid measures of hidden racial prejudice.

We take a different approach from prior work in three main ways. First, we conduct the first analysis of implicit measures of prejudice in a high-quality, nationally representative sample of respondents—the only such sample to our knowledge in which respondents completed both the BIAT and the AMP. Prior work in social psychology has examined the correlation between the IAT/BIAT and the AMP on small samples and/or anomalous samples of respondents coming forward to be interviewed because of their interest in racism (e.g., Payne et al., Reference Payne, Govorun and Arbuckle2008; Greenwald et al., Reference Greenwald, Smith, Sriram, B.-A. and Nosek2009; Bar-Anan and Nosek, Reference Bar-Anan and Nosek2014).Footnote ¹ In contrast, our sample comes from the American National Election Study—the gold standard in political science for nationally representative surveys.

Second, we take a different approach from work that reports levels of implicit anti-Black prejudice by evaluating whether the measures overstate favorability toward Black people among white Americans. Finally, we build on previous political science research that compares the relative explanatory power of implicit and explicit measures to examine how well implicit measures of prejudice capture anti-Black sentiments among the most explicitly prejudiced white respondents in our samples.

Our results bring out key limitations of both the BIAT and the AMP as measures of hidden or unconscious racial prejudice. First, we demonstrate that the relationship between the Black/white BIAT and the Black/white AMP, although statistically significant, is substantively indistinguishable from zero. Given that political scientists have used both to measure the same construct—anti-Black prejudice—one or both measures cannot be valid. Second, we show that both the BIAT and the AMP classify one in every three white respondents as not merely free of racial prejudice, but as preferring Black people to white people. Decades of research in political science and public opinion have made clear that the claim that a third of white people are biased in favor of Black people is not credible. Third, our analyses bring to light that implicit racial prejudice measures frequently fail to identify even the most racially prejudiced whites. Substantial numbers of white Americans who explicitly declare that Black Americans are less intelligent and lazier than white Americans are classified as free of anti-Black prejudice by the BIAT and/or the AMP.

The paper proceeds as follows. First, we summarize how the BIAT and the AMP were administered in our data sources. Next, we present the main empirical results. Then, we highlight and evaluate a number of methodological concerns. Finally, we call attention to some broader implications of our findings.

2. Results

Political scientists routinely rely on the BIAT and the AMP as measures of covert racial prejudice that circumvent social desirability biases. If they both measure hidden racial prejudice, scores on one will predict scores on the other well. Figure 1 plots white respondents’ BIAT-D scores on the x-axis and their AMP scores on the y-axis ($N = 1352$) in the 2008–2009 ANES panel study. For each implicit measure, higher values indicate a positive preference for white people over Black people; conversely, negative values indicate a positive preference for Black people over white people, and zero indicates indifference. The solid line is an OLS regression line and the gray shaded area around the line is the 95 percent confidence interval. The quantity of interest is the magnitude of the relationship between the BIAT and the AMP.

Figure 1. Relationship between BIAT-D scores and AMP scores is virtually non-existent. Note $N = 1352$ white respondents in the 2008–2009 ANES panel study. BIAT-D scores are measured on a –2 to 2 scale, and AMP scores on a –1 to 1 scale (higher values indicating higher anti-Black prejudice).

As Figure 1 shows, there is virtually no connection between them ($b = 0.07$, $R^2$ $= 0.016$). True enough, a one-unit movement on the BIAT produces a 0.07-unit movement on the AMP and that relationship is statistically significant at conventional levels—but the substantive effect is essentially zero. An individual's score on the BIAT tells us virtually nothing about her score on the AMP, and her score on the AMP tells us virtually nothing about her score on the BIAT. This means that either the BIAT or the AMP is not capturing hidden racial prejudice, or both measures are not capturing hidden racial prejudice.

To examine whether either is a credible measure of covert racial prejudice, we look at the BIAT and the AMP individually. Figure 2 shows density plots of white respondents’ BIAT-D scores (left) and AMP scores (middle) in the ANES panel and, for cross-validation, the results from the AMP administered in the 2008 ANES time series study (right).

Figure 2. Both the BIAT and the AMP overstate white respondents’ favorability toward Black people. Note $N = 1352$ white respondents in the 2008–2009 ANES panel study; $N = 894$ white respondents in the 2008 ANES time series study. Each plot shows the density of scores on implicit measures of prejudice across all respondents. The dashed line is the zero line and the shaded area to the left of the line shows the fraction of whites who appear to be more favorable toward Black people than whites on each implicit measure.

Consistent with previous research (e.g., Greenwald and Banaji, Reference Greenwald and Banaji2017; Greenwald and Lai, Reference Greenwald and Lai2020), the unshaded area in Figure 2 to the right of the zero point of indifference (dashed line) shows that most white Americans “demonstrate automatic preference for whites relative to Blacks” (Greenwald and Lai, Reference Greenwald and Lai2020, p. 426). This is the result that has provided a legal predicate for anti-discrimination suits and propelled the widespread use of measures of implicit prejudice in racial sensitivity training. The problem is that the same measurement procedure classifies implausibly large numbers of white Americans as having an automatic preference for Black people relative to white people. The shaded area to the left of the zero line in Figure 2 shows the percentage of respondents who are classified as more favorable toward Black people than white people. For both the BIAT and the AMP in the panel study, it is 34 percent. For the AMP in the ANES time series study, it is 30 percent, indicating that respondents who took the AMP in the time series behaved similarly to those in the panel.

It is important to note that this result follows from the standard calculation of a zero point of indifference. Greenwald et al. (Reference Greenwald, Nosek and Sriram2006) demonstrate that the zero point on the IAT maps onto the zero point of self-reported, explicit measures of preference. Moreover, the calculation of a zero point of indifference continues to be standard operating procedure in research on implicit measures (see Greenwald and Lai, Reference Greenwald and Lai2020). In fact, among respondents classified as more favorable to Black people than white people on the BIAT, the mean BIAT-D score is –0.31 (on a –2 to 2 scale). Among those classified as pro-Black on the AMP in both the panel and the time series, the mean AMP score is –0.13 (on a –1 to 1 scale). The ANES data thus indicate that apparent implicit preferences for Blacks over whites are fairly strong among many white respondents.

The claim that one out of every three white Americans has an automatic preference for Black Americans relative to whites is improbable, given decades of political science research on white racial attitudes (Huddy and Feldman, Reference Huddy and Feldman2009 provide a useful review). In the 2008 ANES time series study, for example, only 9.7 percent of white respondents rate their feelings toward Black people as warmer than their feelings toward white people on a standard feeling thermometer.Footnote ⁴ Likewise, only 3.6 percent of white respondents rate Black people as harder working, and just 2.1 percent as more intelligent, than white people. It is not easy to square the finding that, according to the BIAT and the AMP, one in every three whites prefers Black people relative to white people with the claim that implicit measure of racial prejudice is a satisfactory method for circumventing social desirability bias.Footnote ⁵

Our final analysis builds on previous work that has evaluated the relative explanatory power of implicit and explicit measures of prejudice (e.g., Ditonto et al., Reference Ditonto, Lau and Sears2013; Kalmoe and Piston, Reference Kalmoe and Piston2013; Kinder and Ryan, Reference Kinder and Ryan2017). These studies usually include both implicit and explicit measures of prejudice in regression analyses to evaluate their effect on outcomes like policy preferences or vote choice, and find that the explanatory power of the implicit measures declines substantially when explicit measures are accounted for. We take a different approach, subsetting our ANES data to include just those respondents who are willing to openly endorse explicitly prejudiced statements.

Specifically, the first row of Table 1 under the column headers subsets our data to include respondents who rate Black people as “lazier” than white people. The second row includes those who rate Black people as “less intelligent at school” than white people (“less intelligent” in the time series). The third row comprises the those who feel “cooler” toward Black people than whites. As Table 1 shows, among those who describe Black people as lazier than whites, 23 percent are classified as free of anti-Black prejudice on the BIAT in the 2008–2009 panel, 24 percent are classified as free of prejudice on the AMP in the panel, and 20 percent are classified as free of prejudice on the AMP in the 2008 time series. The comparable numbers for white respondents who say Black people are less intelligent than whites are 26, 28, and 19 percent. Finally, among those who feel “cooler” toward Black people than whites, 26, 28, and 23 percent in the BIAT (panel), AMP (panel), and AMP (time series), respectively, are classified as free of implicit anti-Black prejudice. Again, it is not obvious how to square the claim that the measures of implicit prejudice used in political science circumvent social desirability biases when they classify between a fifth and third of whites who are willing to tell a stranger, the ANES interviewer, that Black people are inferior to white people as free of anti-Black racial prejudice.

Table 1. Significant proportions of white respondents who openly express prejudice toward Black people are classified as free of implicit anti-Black prejudice

Explicitly prejudiced white respondents only are included. In the first row, $N = 228$ white respondents in the 2008–2009 ANES panel study and $N = 410$ in the time series study rate Black people as lazier than white people; in the second row, $N = 347$ in the panel and $N = 361$ in the time series rate Black people as less intelligent than white people; in the third row, $N = 267$ in the panel and $N = 297$ in the time series feel cooler toward Black people than white people. The values in each cell are the percentages of these explicitly prejudiced respondents who are coded as not prejudiced against Black people (either neutral or more favorable toward Black people than white people) based on their BIAT-D-score (second column) or AMP score (panel, third column; time series, fourth column).