2 Objections to SIR and an overview of Reference Royzman, Cusimano and LeemanRoyzman et al. (2017)

SIR reported an association between subjects’ responses to the Disgust-Sensitivity Scale – Revised (DS-R, Reference Olatunji, Williams, Tolin, Abramowitz, Sawchuk, Lohr and ElwoodOlatunji et al., 2007), a refined variant of Reference Haidt, McCauley and RozinHaidt et al.’s (1994) trait disgust sensitivity scale, and their opposition toward GMF. This scale has been criticized for its content as well as its response options (e.g., Landy & Piazza, 2017; Reference Fessler, Arguello, Mekdara and MaciasTybur et al., 2009). Regarding the former, the majority of vignettes describe situations that could be interpreted as physically disgusting, but also creepy, irritating, frightening, surprising, or upsetting. Regarding the latter, only a subset of items (vignettes) ask participants to report how “disgusting” they find a situation or event, with the remaining items’ response options covering a wide range of negative states, including “bothered”, “upset”, and other non-specific expressions of avoidance. Exacerbating these issues, the meaning of “disgusting” itself has been has been shown to be broad enough to encompass a gamut of affective reactions (e.g., Landy & Piazza, 2017; Reference NabiNabi, 2002; Reference Costello and OsborneRoyzman et al., 2017), making it the denotative equivalent of “bothered”. Given the multi-faceted nature of the DS-R, one would be hard pressed to see it as anything more specific than a measure of negative affect (with emphasis on creepiness, horror, disapproval, and disgust).

Our study was, in part, designed to ask if disgust sensitivity would still predict attitudes toward GMF when we could be more certain that we were measuring actual feelings of disgust. To address the limitations of the DS-R, we used a set of vignettes that are more pathogen-specific (TDDS-P, Reference Tybur, Lieberman and GriskeviciusTybur et al., 2009) as well as a 3-item measure of disgust as oral inhibition (OI), a measure derived from the theoretical meaning of disgust (Reference NabiNabi, 2002; Reference Royzman, Leeman and SabiniRoyzman, Leeman & Sabini, 2008; see below). Prior studies have shown OI (unlike “disgust”Footnote ²) to be specific to pathogen-linked content (Reference Royzman, Atanasov, Landy, Parks and GeptyRoyzman, Atanasov, Landy, Parks & Gepty, 2014; Reference Blake, Yih, Zhao, Sung and Harmon-JonesBlake, Yih, Zhao, Sung & Harmon-Jones, 2016), unrelated to social desirability concerns (Reference Royzman, Kim and LeemanRoyzman, Kim & Leeman, 2015), and resistant to figurative use (Reference Royzman, Leeman and SabiniRoyzman et al., 2008). We also included a set of concretely phrased non-disgust options that made it possible to isolate OI from some pertinent alternative reactions (e.g., social disapproval or unpleasant non-gustatory sensations). The inclusion of specific non-disgust options is a significant improvement for two reasons. First, including distinct alternative response options undercuts any motivation for a subject to use OI to express some other feeling that they may have or misreport what they feel in the interest of signaling that they have had some feelings at all. Second, it allows one to test whether any of these other affective reactions may have driven the original reported association between “disgust” and GMF attitudes. Our study was also an improvement in terms of scale administration (see Reference Royzman, Cusimano and LeemanRoyzman et al., 2017, p. 467, for a summary).

With these refinements in place, the correlation between oral inhibition and people’s attitudes to GM-food was close to zero. A subsequent analysis showed that OI was not predictive of GMF attitudes either as a composite or at the level of its constituent responses (all ps> 0.4). Thus, contrary to the claims made by SIR, disgust-the-feeling (OI) appears to be unrelated to a desire to restrict GM-food (harms and benefits not withstanding). Notably, disgust-the-word (“disgusting”) still did rather well, serving as a significant predictor of opposition to GM-food (r = .25, p = .004), just as it did in SIR’s original study.

In spite of these deflationary results, Inbar and Scott make no attempt to defend their approach. Thus, it appears that what ultimately divides us is not the question of whether DS-R can be retained or salvaged as a measure of trait disgust but whether our approach offered enough of an improvement over SIR’s to make our findings more trustworthy.

IS advance four major reasons why they do not view our study as an improvement. First, they express a concern about our lack of proper psychometric analyses, thereby calling into question whether our novel high-granularity response options did in fact carve out distinct affective categories. Second, and most fundamentally, they assert that OI is incidental to the modern construal of disgust and therefore that we were unjustified using OI to operationalize disgust. Third, they argue that it is particularly inappropriate to pair OI with TDDS-P since it is characterized by lack of “oral” items. And fourth, they question the lay meaning of “creeped out” in the context of the TDDS-P vignettes. Finally, in their response, IS offer new evidence purporting to bolster their claim that disgust is the predominant feeling in reaction to thinking about GMF. We address each of these issues in turn.

3 Have we measured distinct affective categories?

One of Inbar and Scott’s central critiques is that we are not justified to assume that our scale of concrete, high-granularity options (e.g., “feel like gagging”) in fact measured separate affective categories (e.g., distinguishing oral inhibition from non-gustatory aversive sensations or moral disapproval).

In developing this critique, Inbar and Scott state early on that confirmatory factor analysis (CFA) is not appropriate for the three single-item factors in the measure because CFA requires a minimum of two (but preferably three) indicators for each latent factor (Reference BentlerBrown, 2006). This is a valid point, but the issue is broader than that. Factor analysis in general (not only CFA but including exploratory factor analysis), is considered inappropriate for factors with less than two items (Reference Costello and OsborneCostello and Osborne, 2005). As a result, the full 11-item measure utilized by Reference Royzman, Cusimano and LeemanRoyzman et al. (2017) is not appropriate for inclusion in any type of factor analysis since it was designed to include multiple single-item factors. Thus, the exploratory factor analysis conducted and reported by Inbar and Scott was inappropriate on statistical grounds.

CFA is relevant to our measure because it was based on established scales, and items were grouped a priori according to presupposed factors based on theory, prior findings, and face validity (Reference BentlerBrown 2006). Thus, for each of the seven vignettes we conducted CFA for the three putative factors that included at least two indicator variables. This yielded a total of 8 items per vignette: the three oral inhibition items, the three disapproval items, and the two epidermal discomfort items.

The model fit indices included (among others): chi-square (Reference HatcherHatcher, 1994), the SRMR (Reference Hu and BentlerHu & Bentler, 1999), the Comparative Fit Index (CFI; Reference BentlerBentler, 1990), and the Root Mean Square Error of Approximation (RMSEA; Reference Brown, Cudeck, Bollen and LongBrown & Cudeck, 1993).Footnote ³ These indices indicated excellent fit of the data to the proposed model structure. For example, chi-square was statistically significant for 6 of the 7 vignettes with the “poop” vignette being the only one with a significant chi-square value. (Notably, chi-square is often significant even in cases when a model provides an acceptable fit to the data [Reference Floyd and WidamanFloyd & Widaman, 1995; Reference HatcherHatcher, 1994]). CFI values were greater than .95 for all seven vignettes and SRMR values were less than .08 for all seven vignettes. RMSEA was less than .08 for six vignettes with the other coming in under the .10 value that is considered acceptable. Additional indices were equally consistent with the model (contact authors for further details).

In summary, the single item factors included in the measure are not appropriate for any form of factor analysis. This is indeed an inherent limitation to the use of single item factors, however these items included in Reference Royzman, Cusimano and LeemanRoyzman et al. (2017) were highly face valid. CFAs conducted with the remaining response options indicated excellent model fit for each of the seven vignettes measured. These results support a conclusion that the measure utilized in Reference Royzman, Cusimano and LeemanRoyzman et al. (2017) was not psychometrically unsound as argued.

4 Is oral inhibition an appropriate measure of disgust?

IS’s most pivotal critique is that there is a clear scientific consensus that oral inhibition is not a theoretically appropriate conceptualization of disgust. If they are correct, then our failure to observe an association between oral inhibition and GM attitudes has no significant bearing on their claim that feelings of disgust affect GM attitudes. However, it appears that IS are mistaken about the current consensus among disgust scholars.

From Darwin on, scholars viewed disgust as a category of food rejection (e.g., Reference Rozin, Haidt, McCauley, Lewis and HavilandRozin, Haidt & McCauley, 1993; Reference Olatunji and SawchukOluatunji & Sawchuck, 2005), whose origin is the “mammalian bitter taste rejection system” (Reference Rozin, Haidt and FincherRozin, Haidt & Fincher, 2009, p. 1180), and whose facial emblem is the “gape” (Reference Rozin, Lowery and EbertRozin, Lowery & Ebert, 1994) or the “sick face” (Reference Widen, Pochedly, Pieloch and RussellWiden, Pochedly, Pieloch & Russell, 2013). Rozin views disgust as “a food-related emotion” defined as “revulsion at the prospect of oral incorporation of offensive objects” (Reference Rozin and FallonRozin & Fallon, 1987, p. 23; Reference Rozin, Haidt, McCauley, Levinson, Ponzetti and JorgensonRozin, Haidt & McCauley, 2000) and affirms that all the classic disgust “scholars, as well as most others, posit a special relation between disgust and the mouth, and disgust and food rejection” (Reference Rozin, Haidt, McCauley, Levinson, Ponzetti and JorgensonRozin et al., 2000, p. 188). Reference Haidt, McCauley and RozinHaidt et al. (1994), whose trait disgust scale is the precursor to the DS-R (Reference Olatunji, Williams, Tolin, Abramowitz, Sawchuk, Lohr and ElwoodOlatunji et al., 2007), stipulate that disgust is “at its core, an oral defense” (p. 702) while Olatunji & Sawchuck describe it as a manner of food rejection and a “guardian of the mouth” (Reference Olatunji and SawchukOluatunji & Sawchuck, 2005, p. 935). Reference EkmanEkman (2003) explicitly linked disgust with gagging (see also Reference Ekman, Matsumoto, Friesen, Ekman and RosenbergEkman, Matsumoto & Friesen, 2005). Thus, in an influential research report designed to unknot the more pointed theoretical meaning of disgust from the rougher lay meaning of the term, Reference NabiNabi (2002) affirms that, though disgust’s broad “common usage … appears to reflect that which is not only repellent but also irritating or annoying” (Reference NabiNabi, 2002, p. 695), “theoretically, disgust refers to the offence taken to noxious objects or ideas that evoke…nausea” (Reference NabiNabi, 2002, p. 695) (see also Reference Kayyal, Pochedly, McCarthy and RussellKayyal, Pochedly, McCarthy & Russell, 2015; Reference Royzman and SabiniRoyzman & Sabini, 2001).

Most importantly, oral inhibition emerges as the criterial feature of disgust that contemporary scholars reach for when in need of a clear touchstone to separate disgust from non-disgust. Reference NabiNabi (2002) used a desire to vomit (“feel like throwing up”) as a key criterion for establishing if the lay terms “disgusted/“disgust” brought to mind what was mainly disgust or some mix of annoyance and anger. In an analysis seeking to differentiate disgust from a related parasite avoidance adaptation, Reference Kupfer and FesslerKupfer and Fessler (2018) delineate the experience of being disgusted as involving “feelings of revulsion, nausea, gagging, the urge to vomit”. In arguing that Nazis are genuinely disgusting, Reference Sherman, Haidt and CoanSherman, Haidt and Coan (2007) aimed to show that people watching a video about the neo-Nazi culture reported the tightening and clenching of their throats. In claiming that fraud and dishonesty provoke actual disgust, Reference Chan, Boven, Andrade and ArielyChan, Boven, Andrade and Ariely (2014) sought to demonstrate that being exposed to instances of deceit reduces oral consumption of water or milk. And, as part of the project aiming to produce a highly discriminant facial display of disgust, one that would clearly differentiate it from anger, sadness, or fear, Widen et al. (2014) instructed an actress to imagine that she was feeling sick and was about to vomit (with the resultant “sick face” being highly similar to the “gape” face used by SIR).

Some scholars have also forewarned of the dangers of linking disgust with something as non-criterial as avoidance since, unlike OI, it would result in difficulties demarcating it from states such as “fear” (e.g., at spiders and gore; Reference Cisler, Olatunji and LohrCisler, Olatunji & Lohr, 2009; Reference Woody and TeachmanWoody & Teachman, 2000), which people also experience when exposed to pathogen vectors. In the case of a self-report measure such as the TDDS-P, operationalizing disgust as avoidance (e.g. “rate the degree to which this [bleeding cut] makes you want to move away”) would make it nearly impossible to separate it from a medley of other affective states (fear, creepiness, epidermal discomfort, disapproval) which TDDS-P is well capable of invoking (Landy & Piazza, 2017; Reference Costello and OsborneRoyzman et al., 2017).

The current scientific consensus is hard to miss. Oral inhibition is what scholars uniformly appeal to when the goal is isolate the criterial feature(s) of disgust, be it for the purpose of segregating the specific theoretical meaning from a broader lay one, differentiating disgust from a closely related non-disgust response, establishing that a non-prototypical disgust stimulus (e.g., Nazis, fraudsters) is genuinely disgusting, developing a discriminant non-verbal operationalization of disgust, or ascertaining cases of disgust-specific avoidance in a clinical setting. Any non-OI delineation of disgust is thus likely to be a minority position.

What about the authors that IS cite? Could they be the dissenting minority in question? The answer appears to be no. To the degree that these authors take a stab at delineating the disgust experience at all, all appear to endorse OI as the key component of that experience. Reference Tybur, Lieberman and GriskeviciusTybur et al. (2009) list “sensation of disgust” and “disgust facial expression” as the two primary features of the output of the “disgust system” (p. 69) and suggest that disgust is characteristically or “often accompanied by nausea, a desire to vomit, and a loss of appetite” (p. 70). Reference Cisler, Olatunji and LohrMurray and Schaller (2016) limit their comment on the “emotional experience” of disgust to an observation that said experience has “evolved from a more ancient and functionally specific distaste response to oral stimuli (Reference Rozin, Haidt, McCauley, Levinson, Ponzetti and JorgensonRozin et. al, 2000), and many contemporary conceptual accounts identify infectious diseases as the primary selective pressure underlying its evolution” (p. 83). Finally, according to Reference Curtis, de Barra and AungerCurtis, de Barra, and Aunger (2011):

The quote merits two observations. First, Curtis et al.’s goal is clearly not to define or specify the criterial feature(s) of disgust, but rather to describe some of its observable identifiers or “associates”, none of which are at odds with the conception of disgust as OI. But more importantly, there is little doubt that, in the context of self-report measures, the identifier or associate of greatest relevance is (iv), which specifies what disgust is as an affective state and features nausea front and center.

None of this is obviously to dispute that disgust is linked to pathogen avoidance. Reference Royzman and SabiniRoyzman and Sabini (2001) were among the earlier authors to characterize disgust as a system for minimizing toxin or pathogen exposure at the level of adaptive task description. However, they also noted that the aversive experience of OI is a good candidate response for motivating physical withdrawal. All in all, there is little doubt that the prevailing scientific consensus is squarely on the side of OI being the criterial feature of disgust. The authors that IS have appealed to are either a part of (or, at the very least, do not detract from) this broad consensus.

5 Is oral inhibition ill-suited for TDDS-P?

Even if OI comprised a defensible measure of disgust in general, it may not have been defensible in the context of TDDS-P. In IS’s view, OI is “a priori … a poor fit for these [TDDS-P] non-oral items” (Reference Inbar and ScottInbar & Scott, 2018, p. 5). They are not explicit about why this is an issue, but we have a hunch. When it comes to items featuring red sores, bleeding cuts, or scurrying insects, the key adaptive task seems to be that of avoiding a potential pathogen exposure by eschewing physical contact with its source. One could argue that gagging and pursing one’s lips would have little adaptive value; what one wants first and foremost is to put some distance between oneself and a potentially diseased entity. Thus, the argument goes, people may not have felt true oral inhibition in response to these items at all.

But if the major concern is a lack of variance due to a floor effect, this is a worry easy enough to dispel by looking at our data. We found that oral inhibition (tied with creeped out) was the strongest high-granularity response reported in reaction to the TDDS-P items (with OI’s coefficient of variance [0.42] being 20% higher than that of “disgusted” [0.23], which did predict resistance to GMF, in line with Reference Scott, Inbar and RozinScott et al. [2016]). We see no reason why this should be otherwise. Other than the “moldy leftovers” item and the scurrying cockroach item (see Reference van Huisvan Huis, 2017 on the evidence of insect consumption among early humans and through evolutionary history), at least three items in the TDDS-P feature explicit (body odor) or implied (sweaty palms, dog poop) malodorousness, one of the key inputs to the food rejection system (Reference Rozin, Lowery and EbertRozin et al., 1994)Footnote ⁴. This issue to the side, it is unclear why the “orality” of an item should matter in the first place. Congruent with the theory of preadaptation (i.e., the co-option of existing structures/systems for new use; Reference Rozin and FallonRozin & Fallon, 1987), there is plentiful evidence that oral inhibition is elicited by a range of “non-oral” items ranging from bodily fluids (Reference Royzman, Leeman and SabiniRoyzman et al., 2008, Pilot) and injuries (Reference Chapman, Kim, Susskind and AndersonChapman, Kim, Susskind & Anderson, 2009; Reference Royzman, Leeman and SabiniRoyzman et al., 2008, Pilot) to anomalous sex (Reference Royzman, Leeman and SabiniRoyzman et al., 2008; Studies 1 and 2; Reference Royzman, Atanasov, Landy, Parks and GeptyRoyzman et al., 2014). In sum, our use of the TDDS-P scale for eliciting oral inhibition is as well justified as the use of OI itself.

6 Taking stock

Let’s collect our results. IS critiqued OI as a theoretically appropriate way to conceptualize and operationalize disgust while also contending that it was a particularly poor match for the TDDS-P vignettes. Having reviewed IS’s arguments, we have concluded that (1) according to the prevailing scientific consensus, oral inhibition is the criterial feature of disgust, (2) has no clear alternative, and (3) is the only reliable way to demarcate disgust from other pathogen avoidance mechanisms (creepiness, epidermal discomfort, fear), at least in the context of a self-report measure à la TDDS-P. Furthermore, (4) the combined weight of theory and evidence strongly indicates that our use of the TDDS-P scale was appropriate for eliciting oral inhibition. Along with the psychometric evidence in the section above, our decision to use a wide range of additional affective response options (including some face valid single-factor response options) further engenders confidence that subjects used our measure as intended.

What can be said about the overall validity of our measure? In modern validity theory (Reference Newton and ShawNewton & Shaw, 2013), validity is not considered a property of the test, but of a particular interpretation of that test adopted for a given purpose and whether this interpretation is justified in light of the totality of evidence before us. The key interpretative move that undergirds all conclusions in Reference Royzman, Cusimano and LeemanRoyzman et al. (2017) is that, imperfect as our assessment might have been, it is a considerably better proxy for the feelings of disgust (as understood by the scientific community at large) and their relationship to GMF attitudes than SIR’s. Everything that has been said above — OI’s hard-to-argue status as the criterial feature of disgust, its apparent specificity, its psychometric differentiation from Disapproval and Epidermal Discomfort, its selective tracking of pathogen-linked content — makes the move well justified.

7 The meaning of “creeped out”

If disgust sensitivity, properly measured, does not predict GMF attitudes, why might disgust sensitivity, improperly measured, do so? We hypothesized that sensitivity to potential threats might have, among other things, confounded the relationship between improperly-measured disgust sensitivity and GMF attitudes. This hypothesis was supported by a correlation between GMF attitudes and feeling “creeped out” in response to the TDDS-P.

IS claim that, while “creeped out” may be associated with unease about a potential threat or riskFootnote ⁵, it is an open question if this is the meaning that subjects had in mind when rating the items of TDDS-P. Implied in this worry is the claim that (a) the canonical elicitors of creepiness (those that one would expect to loom large in a subject’s minds when queried about the meaning of “creepiness” as such) are categorically different from (b) the content of the TDDS-P vignettes. But is this borne out by the evidence? Reference McAndrew and KoehnkeMcAndrew and Koehnke (2016) conclude from their study that non-normative health and hygiene (e.g., being extremely thin or obese, having greasy hair or pale skin) are typical triggers of creepiness (see also Reference Watt, Maitland and GallagherWatt, Maitland, & Gallagher, 2017, for a convergent analysis). In light of this, it appears that the characteristic triggers of feeling creeped out are well represented in TDDS-P. Indeed, half of the TDDS-P items portray individuals with non-normative, unhealthy physical characteristics or poor hygiene, including “Sitting next to someone who has red sores on their arm.”, “Standing close to a person who has body odor.” “Shaking hands with a stranger with sweaty palms”, and “Accidentally touching a person’s bloody cut”.

Whatever “creeped out” may or may not mean in the context of TDDS-P, it clearly not a stand-in for oral inhibition, the construct that comprises the theoretical meaning of disgust. It is this construct that SIR and IS presumably would want to implicate, not merely as a trait predictor, but also as a state antecedent, of people’s avowed opposition to genetically engineering animals and plants. Thus, the strong correlation between “creeped out” and GMF attitudes should not be taken as evidence in favor of IS’s thesis.

8 State disgust

In the penultimate section of their commentary, Inbar and Scott present an empirical critique of what they perceive to be our theoretical counterpoint to their “disgust fuels the moral opposition” proposal, i.e., that “fear, not disgust, is the primary emotion associated with GM food opposition” (Reference Inbar and ScottInbar & Scott, 2018, p. 3). They offer no textual evidence for this claim. This is not surprising since we never made such a claim. Our paper was concerned with the experience of “creepiness” (which we explicitly and repeatedly set apart from that of “fear” both in the principal study and in the supplementary survey) as a proxy for trait sensitivity to a potential hazard or risk. Our claim that sensitivity or vigilance towards risks may underlie GMF opposition is not equivalent to the claim that fear is the dominant reaction to thinking about GMFs, nor that fear (or a related feeling) is a causal antecedent of GMF oppositionFootnote ⁶. Therefore, IS are largely correct that we do not show evidence in favor of the claim that state fear is the dominant reaction to GMFs, but this is because no such evidence has ever been searched for in the first place.

This misunderstanding to the side, we have no principled opposition to the view that disgust is the dominant affective reaction towards GMF. That said, we were skeptical of the evidence that IS and SIR presented defending this claim, and conducted a study to re-assess the prevalence of disgust as a reaction to GMF.

9 Study re-assessing the prevalence of state-disgust

We conducted a study (described in detail in the Appendix) geared to re-reassess the absolute and relative prevalence of state disgust when thinking about GMF. Both SIR and IS presented some studies to this effect. Our study departed from theirs in several ways. First, rather than presenting subjects with GMF-related vignettes, (e.g., unintentionally eating genetically modified salmon), subjects were asked simply to report what they felt while thinking about GMF. This affords us a more ecologically valid measure of the types of stimuli/vignettes that might naturally come to mind when thinking about whether GMF should be opposed or prohibited.

We also modified the way we probed subjects’ affective states. SIR asked their subjects to report which of the two response options (disgust vs. anger) best reflects their reaction to different GM scenarios. IS used a similar method with three response options (disgust, anger, and fear) instead. However, this method comes with some strong assumptions. First, it forces people to report one feeling or another (leaving no room for the possibility that no emotion is being felt). Second, it operationalizes disgust as the word “disgust” (assuming, once again, a close correspondence between the theoretical and lay meanings of the term, an assumption disconfirmed some 15 years ago; see also our arguments above). Third, it limits the range of non-disgust options to one (“anger” in SIR) or two (“fear” and “anger” in IS), assuming that these states would adequately and fairly cover anything and everything else one could feel in this case. These assumptions are difficult to defend, and therefore call into question the validity of subjects responsesFootnote ⁷. To address this, in addition to running a conceptual replication of SIR’s study, we designed two additional conditions that do not make these assumptions and so furnish a truer test of what feelings, if any, people experience.

At the beginning of the study, all subjects (N = 1048 recruited, N = 922 retained following attention check) were queried about whether they opposed or did not oppose “genetically engineering plants and animals for food production” (thus separating opponents from non-opponents) and whether they thought it “should be prohibited no matter how great the benefits and minor the risks of allowing it” (with an affirmative answer leading to the subject being classified as an opposing absolutist). Then, after completing a brief attention check, subjects were randomly assigned to one of three affective report conditions: (1) a replication of IS’s study in which subjects were forced to choose between two feeling states, Angry or Disgusted (2-item FC), (2) a 12-item forced choice condition (12-item FC) in which subjects were forced to choose between feeling Afraid, Angry, Bothered, Confused, Creeped out, Excited, Frustrated, Grossed out [a slang term shown to correspond more precisely than “disgusted” to the theoretical meaning of disgust, Nabi, 2002], Neutral, Sad, Suspicious, or Worried, or (3) an open-ended response mode condition (OERM).

The OERM condition was unique in that it left open the possibility that people do not feel or perceive themselves as having affective reactions at all. Rather, in this condition, subjects were prompted to “use the space below to describe (as you would in a diary) what is going though your mind and what you are experiencing as you are thinking about the practice of genetically engineering plants and animals for food production”. Subjects were further enjoined to note that there are “no right or wrong answers” and not to “hold back or self-censor in any way”.

To diagnose the prevalence of affective language in OERM, we carried out a series of content analyses using a total of 81 affective word stems corresponding to the feeling concepts tapped in the 12-item FC condition of the study. 80 of these were organized into 8-item clusters detailed in Table 2 (Appendix). Due to its precise classification being the present point of contention, “creeped out” was retained as a “free-standing” term, though, in our view, the evidence from prior research (Reference Costello and OsborneRoyzman et al., 2017; Reference McAndrew and KoehnkeMcAndrew & Koehnke, 2016), clearly substantiates its inclusion in a Fear (Afraid + Worry + Suspicion) super-cluster.

As we predicted, SIR’s procedure for measuring state disgust overestimates the absolute prevalence of disgust in relation to other, more assumption-free methods (see Table 1 above). When subjects’ choices were limited to disgust and anger, disgust substantially and significantly dominated non-disgust for absolute opponents (73.3 percent) as well as for the sample as a whole (62.2 percent). However, only 2.8% of the sample chose “grossed out” in the 12-item FC condition (Table 3, Appendix) and only 1.7% of the sample mentioned one of the target disgust-related words in the OERM condition (Table 4, Appendix), both significant drops from the 62.2 percent, χ²(1, 633) = 256.55, p< .0001 and χ²(1, 598) = 246.5, p< .0001, respectively (based on the “N-1” Chi-squared test as recommended by Reference CampbellCampbell, 2007, and Reference RichardsonRichardson, 2011), with no significant difference between the 12-item condition and OERM (χ²[1, 633] = 0.82, p = 0.36).

Table 1: Proportion of subjects choosing/using disgust-related terms as a function of study condition and evaluative orientation (absolute opposition, non-absolute opposition, non-opposition).

Note. FC = forced choice; OERM = open-ended response mode. Superscripts indicate whether disgust is more dominant than

^a is dominated by

^b or is equivalent to

^c non-disgust by Chi-square goodness-of-fit at α = 0.05.

The open-ended response format (Table 4, Appendix), which provides the most assumption-free approach to the problem, suggests that very few people spontaneously perceived themselves as experiencing any emotion at all (41 out of 288, 14.2%). For those apt to experience any feeling at all, feeling worried appears to be three times more common than feeling disgusted (see Appendix for details). Finally, it is worth noting that 1.7 percent (the absolute prevalence of disgust in the open-ended condition) is likely to be an overestimate and that a truly conservative estimate of spontaneously experienced and causally relevant state disgust is likely to be closer to zero (see Appendix for discussion).

These results strongly disqualify disgust as the driver of absolutist opposition in the face of consequentialist reasons. If Reference Scott, Inbar and RozinScott et al.’s (2016) sentimentalist viewpoint was correct, and absolute opposition to GM food was in fact “disgust-based” (p. 316) or “disgust-backed” (p. 322), the last thing one would expect to find is that the majority of GMF’s absolutist opponents feel no disgust at all.