Background for the project
In the summer of 2018, we decided to write a registered report studying disgust sensitivity in children for the Special Issue on Disgust and Political Attitudes that was published by Politics and the Life Sciences in November 2020, with Michael Bang Petersen, Joshua M. Tybur, and Patrick A. Stewart as coeditors. As we describe in greater detail here, we had grand ambitions. Not much had been written on this topic—research on disgust sensitivity and the influence of disgust sensitivity on political attitudes had focused almost exclusively on adult samples. We wanted to fill this gap by studying the association between disgust sensitivity and the nascent political attitudes of children. Moreover, we saw an opportunity to develop alternative measures of disgust sensitivity. In the study of political attitudes, scholars have relied largely on self-reported measures of disgust sensitivity. We sought to broaden scholars’ measurement toolkit by adding a sensory measure that captures individual differences in olfactory reactions to potentially disgusting smells along with a behavioral measure. This would help eliminate some social desirability effects that permeate this literature (Banaji et al., Reference Banaji, Roediger, Nairne, Neath, Surprenant, Roediger, Nairne, Neath and Suprenant2001), as olfactory reports are accurately reflective of disgust (Adolph & Pause, Reference Adolph and Pause2012; Cavazzana et al., Reference Cavazzana, Wesarg, Parish‐Morris, Lundström and Parma2018; Croy et al., Reference Croy, Olgun and Joraschky2011), but they are unlikely to be linked directly to political attitudes in the minds of most people.
We were delighted that our registered report, after a round of revisions informed by anonymous referee reports, received an in-principle acceptance for inclusion in the special issue, and we got to work on the study we had proposed at the end of 2019. The study was conducted with the help of the Medialab within the International School for Advanced Studies (SISSA) located in Trieste, Italy. The choice of this location was strategic: we wanted to capitalize on the heterogeneity of the local cultural environment outside the American context, where racial biases reflect, in part, that nation’s long and peculiar history of slavery and the racialized caste-like system that followed in its wake. In our registered report, we proposed collecting a sample of 100 children along with one or both of their parents. While we anticipated that we might fall short of this goal, we could not have foreseen two events that almost completely sidelined our research. First, after eight days of data collection, extensive flooding in Trieste required us to pause the study.Footnote 1 We planned to restart data collection in 2020 after the winter break. The local team slated March 2020 for the re-launch of the project. Unfortunately, the COVID-19 pandemic hit Italy particularly hard in late February 2020, and all in-person research was halted.
At the time of this writing, it still remains inadvisable to conduct a study that asks unvaccinated children to smell jars and touch items that others may be in touch with. Furthermore, the research team trained to do our project has since departed and retraining is constrained by travel restrictions. In short, it is now impossible for us to complete the research that we began. Therefore, we are left with data from 29 parents and 32 children, which is far fewer than we had planned for.
In the interest of pushing the field forward, albeit marginally, we describe our motivation, research plan, and preliminary findings in the pages the follow, and we make our materials freely available at OSF.Footnote 2 While we were disappointed to not ultimately be included in the excellent special issue published by Professors Petersen, Tybur, and Stewart, we hope that what is an example of “science interrupted” for us can provide the seeds for scientific advance for another research team, as well as illustrate the benefits of the registered report format for rewarding researchers who take on high-risk, high-reward projects.
Our motivation
Our starting point for this project was the accumulation of evidence showing that the behavioral immune system, or the collection of cognitive processes selected through human evolution to guard against infections by using disgust, shapes political attitudes (cf. Terrizzi et al., Reference Terrizzi, Shook and McDaniel2013). This body of research contends that the behavioral immune system, operating largely outside conscious awareness, motivates individuals who are more prone to experience disgust to adopt policy preferences that minimize (real or perceived) threats linked to objects and individuals threats potentially carrying pathogens. Individual variance in the susceptibility to feel disgusted, called disgust sensitivity, correlates with a preference for political policies that enable the avoidance of out-group members, such as opposition to immigration and leeriness of homosexuals (Aarøe et al., Reference Aarøe, Petersen and Arceneaux2017; Faulkner et al., Reference Faulkner, Schaller, Park and Duncan2004; Inbar et al., Reference Inbar, Pizarro, Knobe and Bloom2009; Smith et al., Reference Smith, Oxley, Hibbing, Alford and Hibbing2011; Terrizzi et al., Reference Terrizzi, Shook and McDaniel2013).
We were interested in studying whether there is a link between disgust sensitivity and the emerging political attitudes of children related to immigration—in particular, prejudice toward members of different ethnic groups. Among adults, many factors influence attitudes toward immigrants, from economic threat to clashing cultural values (Brader et al., Reference Brader, Valentino and Suhay2008; Hainmueller & Hiscox, Reference Hainmueller and Hiscox2010; Sniderman et al., Reference Sniderman, Hagendoorn and Prior2004). For many adults, prejudice against immigrants lies at the heart of these justifications (Allport, Reference Allport1954; Wright et al., Reference Wright, Citrin and Wand2012). Where does prejudice among adults come from? The standard explanation of adults’ attitudes (including prejudiced attitudes that underlie particular political attitudes) presumes that they arise from a complex interaction of the childhood environment (including the in utero environment) and biological predispositions (Funk et al., Reference Funk, Smith, Alford, Hibbing, Eaton, Krueger and Hibbing2013). Childhood experiences, including interactions with parents and peers, are thought to structure the political attitudes to which adults gravitate later in life (Holbein Reference Holbein2017; Jennings et al., Reference Jennings, Stoker and Bowers2009; Searing et al., Reference Searing, Schwartz and Lind1973). Nonetheless, there is little research on political attitude formation that directly studies children, and the few studies that exist examine the attitudes and behaviors of adolescents (Holbein, Reference Holbein2017). In contrast, we sought to explore the sensory, cognitive, and behavioral correlates of disgust sensitivity in school-age children (6–10 years old) along with their parents.
Research that has examined children shows that their political attitudes correspond strongly with those of their parents, especially before children reach adolescence, but this correspondence declines in adulthood when the influence of biologically instantiated psychological motivations becomes more apparent (Hatemi et al., Reference Hatemi, Funk, Medland, Maes, Silberg, Martin and Eaves2009). Similarly, previous research reports strong correlations between parents and their adult offspring on food-related disgust sensitivity (Davey et al., Reference Davey, Forster and Mayhew1993; Rozin et al., Reference Rozin, Fallon and Mandell1984). Twin studies of adults show that genetic factors (which are inherited from parents) explain more variance in disgust sensitivity than does the shared childhood environment (Sherlock et al., Reference Sherlock, Zietsch, Tybur and Jern2016). Yet because the influence of the shared environment on political attitudes tends to be stronger in childhood and adolescence and declines in adulthood (Hatemi et al., Reference Hatemi, Funk, Medland, Maes, Silberg, Martin and Eaves2009), it is still possible that the link between disgust sensitivity and children’s emerging political attitudes surrounding immigration could reflect both biological predispositions and parental inputs.
Although children are capable of expressing opinions on political topics (Hatemi et al., Reference Hatemi, Funk, Medland, Maes, Silberg, Martin and Eaves2009), we anticipated that, unlike adults, children would not be able to connect disgust sensitivity to their nascent political attitudes about immigration in the abstract. In contrast, we expected both children and adults to be able to connect disgust sensitivity to concrete scenarios about interacting with immigrant children. This hypothesis was based on two rationales. First, concrete scenarios should draw on more intuitive mental modules devoted to social categorization that emerged early in the evolution of human ancestors navigating social coalitions in small scale societies (Petersen, Reference Petersen2015). Second, children do exhibit prejudice toward individuals of out-groups by five years old (Raabe & Beelmann, Reference Raabe and Beelmann2011). They are capable of learning about abstract political issues (Hatemi et al., Reference Hatemi, Funk, Medland, Maes, Silberg, Martin and Eaves2009), as well as prejudice toward social groups (Degner & Dalege, Reference Degner and Dalege2013), from their parents. Therefore, we expected to find the political attitudes of parents (along with the parents’ level of disgust sensitivity) to be more closely linked to children’s expression of political attitudes on more abstract questions about immigration.
A multifaceted approach to measuring disgust sensitivity
Most studies measure disgust sensitivity with self-administered survey questions about how disgusted one feels in different scenarios (e.g., stepping in dog poop) (see, e.g., Tybur et al., Reference Tybur, Lieberman and Griskevicius2009). Simplified versions of the scales developed for adults have been applied to youth samples. The 30-item Disgust Emotion Scale (Kleinknecht et al., Reference Kleinknecht, Thorndike and Walls1996) has been developed as a reliable index of the multifaceted construct of disgust sensitivity and has been validated in children (Muris et al., Reference Muris, Huijding, Mayer, Langkamp, Reyhan and Olatunji2012). However, work remains to be done on systematically assessing whether cognitive responses demonstrate convergence validity with other measures of disgust (e.g., sensory and behavioral) by tapping the same underlying construct of disgust sensitivity, particularly during childhood development.
In contrast to self-reports used to measure attitudes, sensory self-reports are deemed to be more reliable and to more accurately reflect participants’ sensory experience. This is particularly true for olfaction in relation to affective states (Doty et al., Reference Doty, McKeown, Lee and Shaman1995). As Yeshurun and Sobel (Reference Yeshurun and Sobel2010) claim, the main dimension of the olfactory space—pleasantness—is affective in nature, and children as young as five are able to report on this olfactory dimension (Cavazzana et al., Reference Cavazzana, Wesarg, Parish‐Morris, Lundström and Parma2018). Specifically for disgust, adults can almost invariably evoke disgust via the olfactory channel, particularly for objects that carry microbial threat (Croy et al., Reference Croy, Olgun and Joraschky2011), and disgusting odors elicit stronger physiological reactions than their visual counterparts (Adolph & Pause, Reference Adolph and Pause2012). In other words, behavioral and physiological responses related to defensive motivations may be more effectively triggered by odors than by comparable visual stimuli, and they are also less well modulated through cognitive downregulation.
To validate whether disgust has direct behavioral consequences, researchers have devised behavioral tasks that should be affected by disgust sensitivity. In the child versions of such tasks, children are asked to perform different consecutive steps in which they are invited to approach disgusting materials, such as a sticky candy fallen on the ground, a used dirty sponge, and a used cotton swab, and to rate how disgusting each situation was (Muris et al., Reference Muris, Huijding, Mayer, Langkamp, Reyhan and Olatunji2012). This test allows researchers to quantify disgust-induced avoidance (i.e., how many steps the child was able to complete) in relation to the disgust they report experiencing.
A multifaceted approach that includes sensory indicators (e.g., olfactory responses of disgust), cognitive indicators (e.g., survey measures), and behavioral responses provides an appealing way to investigate the effects of disgust sensitivity in association with prejudice, because it taps into psychological processes that are conscious as well as into those that lie outside conscious awareness and control. Moreover, given the ease with which people of all ages can report the pleasantness of odors, including sensory measures in our measurement toolkit not only helps circumvent social desirability bias in adults (Banaji et al., Reference Banaji, Roediger, Nairne, Neath, Surprenant, Roediger, Nairne, Neath and Suprenant2001), but it also provides an ideal approach for studying attitude formation in children.
Hypotheses
In our registered report (preregistered at https://osf.io/9we84/), our main set of hypotheses centered on the thesis that in concrete scenarios, disgust sensitivity and attitudes toward immigrants (e.g., playing with a child from another country) will correlate both in children and in parents, while in abstract scenarios (e.g., do you support immigration in your country?), such correlation will only emerge for adults. We measured disgust sensitivity across multiple modalities: (1) sensory (olfactory disgust), (2) cognitive (self-reported disgust sensitivity), and (3) behavioral (avoidance of disgusting items). Specifically, we hypothesized that:
H1: Olfactory disgust, self-reported disgust sensitivity, and behavioral avoidance will equally influence children and parents’ answers to the concrete scenarios about immigrants.
H2: Olfactory disgust, self-reported disgust sensitivity, and behavioral avoidance will influence parents’ answers to abstract questions about immigration more than children’s answers.
In addition, we proposed running omnibus linear mixed-model analyses to evaluate whether the olfactory disgust ratings and behavioral avoidance correlate with self-reported disgust sensitivity in children, even after controlling for their parents’ disgust sensitivity. However, because we lacked a sample size that would provide anywhere near the statistical power needed to interpret these results, we did not run this analysis.
Methods
Participants
The investigation was conducted at the International School for Advanced Studies in Trieste. SISSA Medialab, an in-house company working on all forms of science communication, aided with the data collection as part of its interactive science-related activities with the community. Both children and parents were sampled from the general local population and invited to participate in the study in a research space at the main train station of the city as well as in the SISSA laboratories. As previously tested by one of the authors (Parma), this space and the citizen science research format allowed for quicker yet reliable data collection (comparable to lab-based evaluations) and maximized the possibility of simultaneously testing parents and children.
In November 2019, we recruited Italian parents living near the research site in Trieste who reported that they were nonsmokers and that they and their children were in good health. Children aged 6 to 10 years old participated in the study. Parents provided consent for themselves and their children, and the children provided verbal assent to participation. As we explained earlier, we had to abort the study after collecting responses from 29 parents and 32 children.
Stimuli
We describe the stimuli and questionnaires in English. Because participants were native Italian speakers, all materials were translated into Italian by Parma and the local team, who were all native Italian speakers.
Political scenarios
The political scenarios presented on immigration were posed in a concrete and a more abstract form. The presentation of concrete scenarios was preceded by the presentation of two pictures, one with the face of a white child and one with the face of a black child, followed by concrete questions regarding the ways in which the participants would be willing to interact with the child (see Table 1). The faces of the white and black children were placed on a neutral background. We use the skin color of the child depicted in the picture as a marker of otherness. However, we kept the sex of the participant constant to that of the child shown in the picture (e.g., girls saw a picture of a girl). The order of the picture and the related questions was counterbalanced across participants. Participants were asked to read the scenarios and complete the survey alone, but younger children were helped by an experimenter who read the instructions and the test material aloud. Participants provided their answers using a visual analogue scale (VAS) anchored to a happy and to a sad face. Children offer more reliable responses on this answer format (Cavazzana et al., Reference Cavazzana, Wesarg, Parish‐Morris, Lundström and Parma2018; Mellor & Moore, Reference Mellor and Moore2014).
Table 1. Concrete questions on immigration.
We coded the 4-point response to range from 0 to 100 (i.e., 0, 33, 67, 100). We took the average response across the five items separately for the in-group child and the out-group child. We then calculated the standard measure of in-group bias by subtracting the mean for the out-group child from the mean for the in-group child. Higher values indicate a greater willingness to interact with the in-group child relative to the out-group child.
To ease participants into the tasks, the concrete scenarios always preceded the abstract ones. The abstract scenarios used the prompt employed in previous survey research studying political attitudes in young children (see Table 2) (Hatemi et al., Reference Hatemi, Funk, Medland, Maes, Silberg, Martin and Eaves2009). The questions were worded so that both children and adults could easily understand them, and we used the VAS (happy/sad faces) for children. We coded the 4-point response to range from 0 to 100 (i.e., 0, 33, 67, 100).
Table 2. Abstract questions on immigration.
By comparing responses to the questions about African immigration with responses to the questions about Ukrainian immigration, we are able to test the hypothesis that individuals from less familiar cultures (Africa in the Italian context) could be more likely to trigger the behavioral immune system (Aarøe et al., Reference Aarøe, Petersen and Arceneaux2017; Faulkner et al., Reference Faulkner, Schaller, Park and Duncan2004).
Disgusting odors
We asked participants to smell and rate three disgusting odors: fish sauce (100% solution), isovaleric acid (1% solution, smells like rancid cheese), and civet (10% solution, a fecal smell).Footnote 3 The experimenters placed 2 ml of the odor solutions in 125 ml wide-mouth glass jars and positioned them 2 cm below the nostrils of each participant for 3 seconds. After exposure, participants were asked to complete a paper-and-pencil VAS rating how pleasant they found the odor (0–100, with 0 being extremely unpleasant and 100 being extremely pleasant).
Disgusting materials
Three behavioral disgust tasks, inspired by Muris et al. (Reference Muris, Huijding, Mayer, Langkamp, Reyhan and Olatunji2012), were presented to each participant: (1) touch a sticky candy found on the ground; (2) touch a used dirty sponge; (3) touch a used cotton swab. Each material was presented in a randomized order and followed by the question “Would you touch this?” (touch probability). If the participant said yes, a VAS (0–100) was presented to record the answer to the question “How pleasant would it be to touch this?” (touch pleasantness). Taking the average of the answers to the first question generates a probability that the participant was willing to touch the items; we registered interpreting this as an indicator of behavioral avoidance (i.e., the higher the number, the lower the avoidance). The average of the answers to the second question provides the behavioral disgust rating among items that participants were willing to touch.
Disgust sensitivity questions
To measure expressed disgust sensitivity, we selected seven questions proposed by Tybur, Lieberman, and Griskevicius (Reference Tybur, Lieberman and Griskevicius2009) as a specific measure of pathogen disgust. The response scale was changed from a 7-point Likert scale into a simpler VAS with four options—0, not at all disgusting; 33, a bit disgusting; 67, disgusting; and 100, very disgusting—in which the verbal labels were matched with emoticons reflecting the different degrees of disgust (see Figure 1 for scale emoticons and Table 3 for the English translation of the scale and questions). In line with our coding scheme for all of the VAS response sets, we coded the options to range from 0 to 100. To clarify the concept of disgust for younger children (6 years old), the experimenter included an example of a “yucky” situation (e.g., a slimy animal) that was not included in the testing material. We chose the Pathogen Disgust Scale over the 30-item Disgust Emotion Scale for children (Kleinknecht et al., Reference Kleinknecht, Thorndike and Walls1996), because it better taps individual differences in the behavioral immune system than the older and less reliable disgust sensitivity scale on which the Disgust Emotion Scale was based.
Figure 1. Emoticons used in the visual analogue scale for disgust sensitivity questions.
Table 3. Disgust sensitivity questions.
Because the meaning of some words may be unclear to some of the children, the experimenter provided synonyms and concrete examples (e.g., “An odor is a smell. A flower smells and gives an odor”).
The descriptive statistics for the variables are shown in Table 4 (children) and Table 5 (parents). On average, the children displayed in-group biases in both the concrete and abstract scenarios, while parents only displayed in-group bias, on average, on the abstract question. Children and parents displayed, on average, similar levels of disgust sensitivity with respect to the odor, touch rating, and self-reported disgust sensitivity. Children showed higher levels of behavioral disgust avoidance than parents. The average age of children was 8 and the average age of parents was 42. Note that most child participants identified as boys (nearly 60%), while most parents identified as women (69%).
Table 4. Summary statistics for children.
Table 5. Summary statistics for parents.
Procedures
We achieved task compliance and maintained data quality by implementing a 1:1 researcher-to-participant ratio. The total research time for each child-parent dyad was 15 to 20 minutes. Two different experimenters simultaneously tested the child and their parent to avoid cross-influences. Participants were asked to answer questions about the concrete and abstract scenarios before they were asked questions about disgust in order to avoid creating a testing effect in which the disgust questions induced disgust and then affected answers to the politically relevant questions (Inbar et al., Reference Inbar, Pizarro, Knobe and Bloom2009; Inbar et al., Reference Inbar, Pizarro and Bloom2012).
Data analysis plan
To test our main hypotheses, H1 and H2, we preregistered the following regressions:
$$ {y}_c\hskip0.35em =\hskip0.35em {\unicode{x03B2}}_0+{\unicode{x03B2}}_1D{S}_c+\Gamma {\mathbf{X}}_{\mathbf{c}}+\unicode{x03B5} $$
$$ {y}_p\hskip0.35em =\hskip0.35em {\unicode{x03BB}}_0+{\unicode{x03BB}}_1D{S}_p+\Lambda {\mathbf{X}}_{\mathbf{p}}+\unicode{x03C5} $$
where y represents immigration attitudes, DS represents disgust sensitivity (measured in four ways: olfactory disgust, self-reported disgust sensitivity, behavioral avoidance, and behavioral disgust), X represents a matrix of controls for demographic variables (age and gender), the subscript c denotes children, and the subscript p denotes parents.Footnote 4 We run separate regressions for each our dependent measures of immigration attitudes (concrete and abstract questions). The main test of our hypotheses comes from t-statistics for β1 and λ1 and a z-test of the difference between β1 and λ1 (Clogg et al., Reference Clogg, Petkova and Haritou1995).
Preliminary results
Table 6 reports the correlation matrix among the disgust sensitivity measures. For both parents and children, the pleasantness of disgusting odor is negatively correlated with the self-reported measure of disgust sensitivity. This corroborates our contention that olfaction offers an additional approach to measuring individual variance in disgust sensitivity. The behavioral avoidance measure (touch probability) also appears to offer a useful approach to measuring disgust sensitivity for both children and adults. Interestingly, the behavioral disgust measure (touch pleasantness) correlates in the expected negative direction with disgust sensitivity for children, but not for adults.
Table 6. Correlation matrix among disgust sensitivity measures.
Table 7 shows the regression results. We do not find evidence for H1, which posited that the four different measures of disgust sensitivity would correlate with the concrete measure of in-group bias in the same way. Instead, we find that a stronger negative reaction to the disgusting odors is associated with higher levels of in-group bias for parents (p < .01, see Figure 2) but lower levels of in-group bias among children (p = n.s., difference between children and parents, p = .02). Furthermore, we find that the self-reported measure of disgust sensitivity is negatively correlated with in-group bias on the concrete scenario among children (p < .05, see Figure 3) and essentially uncorrelated with the parents’ level of in-group bias (difference between children and parents, p = .02). Neither of the measures of behavioral disgust sensitivity is strongly correlated with in-group bias for parents or their children. The results reported in Table 7 also do not support H2. None of the measures of disgust sensitivity is strongly associated with in-group bias in the abstract scenario—for children or their parents.
Figure 2. Correlation between reactions to disgusting odors and in-group bias on concrete scenarios, parent sample.
Figure 3. Correlation between self-reported disgust sensitivity and in-group bias on concrete scenarios, child sample.
Table 7. Associations between measures of disgust sensitivity and in-group bias among children and their parents.
∗p < .1; ∗∗p < .05; ∗∗∗p < .01.
Discussion
Given our small sample size, we resist drawing any sweeping conclusions from these results. As nominal observations, we report that we fail to replicate the connection between disgust sensitivity and immigration attitudes that previous research has found among adults. Perhaps a larger, more representative sample would have replicated those studies. Nonetheless, Trieste is a border city with a highly educated population, and at the very least, the results suggest possible boundary conditions (e.g., geography, education) for future research to consider.
We do find promising results with respect to the concrete scenarios. Among adults, the pattern of outcomes suggests that the least intrusive measure of disgust sensitivity—disgusting odor pleasantness rating—predicts in-group bias toward a specific out-group child, while the most intrusive and common measure of disgust sensitivity (the Pathogen Disgust Scale) was uncorrelated with prejudice toward the out-group child. While we should await a larger study to replicate these findings before drawing conclusions, we are intrigued by the possibility that olfaction might be a more fruitful route to study the influence of disgust sensitivity on attitudes and behaviors that are subject to social desirability bias (e.g., discriminating against a child of a different race or ethnicity). It would also offer additional support for the theoretical contention that the behavioral immune system operates largely outside conscious awareness.
Among children, we found evidence suggesting that disgust sensitivity may be negatively associated with out-group bias. Of course, this unexpected finding may be a sampling coincidence, and we again counsel against drawing broad conclusions before replicating this study with a larger sample.
Although we were unable to complete the study as fully planned, we believe that the approach described here provides a useful blueprint for future research in this area. It shows that olfaction offers a window into disgust sensitivity above and beyond self-reported measures, at least for adults. It also underscores the need for more research on the development of political attitudes among children. A larger study using our design would be poised to shed light on how disgust sensitivity develops and relates to politically relevant attitudes before adulthood.
Concluding thoughts: Of best-laid plans and the value of registered reports
As seasoned scholars, we are accustomed to the inherent uncertainty of doing research. There are many moving parts in a research project, and each one is attached to some degree of uncertainty. Will the experimental manipulation work as planned? Will our measures pan out as we expected? Will we be able to recruit participants in the way we plan? Will the data support our hypotheses? Will our colleagues find our research question to be interesting and important? Under the traditional review process system, conducting research, especially for younger scholars, is like walking into the void. Each possible research project comes with the risk that something will go wrong and hundreds of hours of work will end up in the proverbial file drawer. As graduate students, we were coached to choose projects that minimized the risk of something going wrong. “Pick a project on a well-worn topic that you know other scholars will find interesting.” “Only use measures that others have published so you know they will likely work.” “Craft hypotheses that allow you to say something that people want to hear no matter how your findings turn out.” Not only did we hear this advice from well-meaning colleagues early in our careers, we have also been guilty of repeating it to younger scholars. From the perspective of how science should work, we can all recoil from this misguided yet well-meaning advice, but to quote the American rapper and actor Ice-T (1999), “don’t hate the playa, hate the game.”
Putting aside floods and a pandemic, our project is exactly the kind of project that a risk-averse scholar would avoid taking under the traditional review process. We tackled an understudied question: political attitude formation in children. Perhaps scholars who study public opinion are more interested in the attitudes of adults and would not find our study very interesting. After all, as one of our advisers told us when we suggested that “no one has done this” as the rationale for undertaking a soon-to-be abandoned dissertation idea, “Maybe no one has done it because it isn’t interesting.” The retort to this critique is that maybe no one has done it because it is difficult to do and someone just needs to make the case for doing it, but the adviser’s point was valid under the traditional publication process. We cannot be sure at the beginning of a project like this whether it will find an audience. If our future careers depend on publishing something, it would be best to find a different topic. In addition, we also sought to offer a different and novel way of measuring a theoretical concept (at least to scholars in political science). What if our proposed measure flopped? It would be easy to dismiss our effort as misguided from the start. And finally, what if the data did not support our hypotheses? It would be easy to offer many post hoc explanations for why our research design failed to produce the results that it should have.
Registered reports address each of these risks and make high-risk, high-reward research possible. By submitting the proposed research design before conducting the research, we were able to get an evaluation from the reviewers and the editors about the perceived importance of our topic, the soundness of our proposed measures, the logical coherence of our hypotheses, and the quality of our research design before we undertook the hard work of collecting the data and before the results could color their judgments about the value of our research. Had we learned that neither the reviewers nor the editors thought our project was interesting, important, or sound, that would have saved us the pain of learning this after we had invested resources in completing the study. Happily for us, though, after incorporating the reviewers and editors’ feedback, we were essentially told that our research question was important, our proposed measures were sound, our hypotheses were logically coherent, and our research design appropriate. Had our findings not supported the hypotheses, it could not be because we had failed to pick a good topic or execute a sound research design. It would simply be because that is the way the cookie crumbles sometimes. The world is a messy place and so good science will also produce messy results. A registered report allows researchers to tackle important questions that the traditional review process discourages.
Our experience illustrates yet another benefit of registered reports. We are used to all of the risks that we just mentioned, but we are less acquainted with the risk of a flood and a pandemic making our research impossible. Even in less extraordinary times, there is always the looming risk that a research project will not come to fruition despite the best-laid plans. Promised funding can disappear; equipment can break; data files can become corrupted; and more. In the terrible event that science is interrupted by disaster, tragedy, or just plain old bad luck, the registered report is still published, and there is a record of one’s research question and research design along with the imprimatur of the peer-review process. If it were not for the registered report process at Politics and the Life Sciences, we would have been left with an unfinished study and a sad tale to tell our colleagues and students. Yet another tale telling them to play it safe. Thankfully, we do not have a sad tale, but an optimistic one of how the review process can actually encourage scholars to take a risk for the good of science and be rewarded for it.
Open access
