The role of the NASEM and scientific consensus reports

In 1863, President Abraham Lincoln and the U.S. Congress created the National Academy of Sciences (NAS, now the NASEM) to bring scientific research to bear on issues of “national importance.” ⁸ Over time, the NAS created a system in which experts serve pro bono to create scientific reports. Some reports address specific problems, such as a recommendation that led the Union to correct a navigation problem on its ironclads just a year after the NAS was founded. ^{Reference Cochrane9} Others review the scientific consensus on matters of public concern, such as the report created for President George W. Bush on the long-term effects of traumatic brain injuries sustained by U.S. soldiers in Afghanistan and Iraq. ¹⁰

However, the extent to which the release of consensus reports influences public discourse and, ultimately policy, is unclear. Although one study using an experimental design found a significant reduction in polarization when participants were given information about scientific consensus on a particular issue, ^{Reference Bolsen and Druckman6} research has yet to study the effect of consensus reports in real-world settings, especially in the multimedia environment of traditional news and social media that characterizes the public’s information sources today.

Here we examine how the GE crops report affected media coverage and public opinion of GMOs. Acknowledging the complexities surrounding GE crops, the NASEM stated, “Claims and research that extol $\ldots$ benefits and risks of GE crops have created a confusing landscape for the public and for policymakers.” ¹ Thus, the purpose of the GE crops report was to resolve some of this confusion. The report did so in its findings by showing that some claimed benefits — such as the potential for greatly increased crop yields — had not met expectations, while some claimed risks — such as health risks from consumption of GE foods — were not supported by scientific evidence. ¹

Public opinion on GMOs

As the purpose of the GE crops report suggests, GMOs have been a source of contention and sometimes confusion for the public. ^{Reference Brossard, Popp, Jahn, Matlock and Kemper11} The use of genetic modification in food prompts a variety of concerns, including how the technology might affect the environment, its impact on human health, and concerns about corporate control and economic access for small farmers or developing countries. ^{Reference Brossard, Popp, Jahn, Matlock and Kemper11,Reference Bawa and Anilakumar12,Reference Frewer13,Reference Scott, Inbar and Rozin14} Some of the health concerns among the U.S. public contradict scientific consensus, as overall, the U.S. public believes that GMOs pose a risk to human health, ^{Reference Funk and Rainie15,Reference Runge16} which is not supported by scientific evidence. ¹

In general, although public support for GMOs in the United States has been mixed, the majority of Americans agree that GMOs in foods should be labeled. ^{Reference Runge16,Reference Hallman, Cuite and Morin17} In fact, during the weeks leading up to the release of the GE crops report, discussion of legislation requiring the labeling of foods containing GMOs dominated Twitter conversations about GMOs, as public figures, such as former Democratic presidential candidate Bernie Sanders, took a public stance in favor of labeling (Figure 1).

On a global level, a meta-analysis of studies published between 1990 and 2010 found that both public risk and benefit perceptions of GMOs have been increasing. ^{Reference Frewer13} This trend suggests that public opinions of the technology are becoming more ambivalent. It is well documented that exposure to information about GMOs through media use can impact public views of the technology and that these media effects will depend on the values and views of science that individuals hold as they process the information. ^{Reference Brossard and Shanahan18,Reference Brossard and Nisbet19} Less understood, however, is how the dynamics of traditional and social media shape public views of controversial science issues, such as GMOs, ^{Reference Brossard and Weitz20} which becomes increasingly important as more U.S. adults get their news from online and social media sources. ^{Reference Mitchell, Gottfried, Barthel and Shearer21}

Table 1. Boolean search strings used to capture all potentially relevant posts and articles about (1) the GE crops report and (2) GMOs in general.

The GE crops report’s reach and media coverage

Within a mixed-media environment populated by an increasingly ambivalent (but still largely risk-focused) public opinion context, we examine the influence of the GE crops report on media coverage and on public discourse and opinions of GMOs. After its release in May 2016, the report was downloaded more than 42,400 times from the NASEM publications website (as of March 2018), and in addition to being featured in major U.S. media outlets, the report was mentioned on Twitter more than 4,000 times during the month of its release. The Altmetric score of subsequent online coverage of the report placed it in the 99th percentile of the 5.6 million research outputs tracked by Altmetric (which calculates a score based on the online attention a publication receives to supplement more traditional citation-based measures) and indicates that the report was mentioned in 144 news outlets. ²² Although Altmetric scores can be intentionally or unintentionally misrepresentative, we use them here in combination with the download and Twitter mention numbers to provide context for some of the ways the report spread through social and traditional media before we analyze the impact of the report on discourse of GMOs.

Methods

To systematically assess the reach and impact of the GE crops report as it moved through social media, we combined human coding with an intelligent algorithm for three objectives: (1) sort data sets of hundreds of articles and millions of tweets to capture English-language news and Twitter coverage of the report, (2) situate the report’s release in broader patterns of discussion of GMOs on Twitter, and (3) assess the levels of negative or positive sentiment associated with discussion of the report in traditional and social media.

The content analysis used news and social media data collected through an automated nonparametric content analysis software from Crimson Hexagon called ForSight. The software generated content based on two different Boolean search strings. The first search string was designed to capture a census of relevant news stories and tweets explicitly referencing the GE crops report. The second search string collected a census of news stories and tweets related to GMOs in general (Table 1). These searches produced four data sets: (1) tweets related to the GE crops report, (2) tweets related to GMOs, (3) news articles related to the GE crops report, and (4) news articles related to GMOs.

We then combined those data with nationally representative public opinion data collected for the Annenberg Public Policy Center of the University of Pennsylvania by the research company SSRS, which conducted baseline, pre-report, and post-report surveys by phone. This analysis used three weekly surveys: one conducted on January 26, 2016, as the baseline; one conducted May 10–15, 2016, seven to two days before the release of the GE crops report (pre-report survey); and one conducted May 17–21, 2016, on the day of the release through the four days following (post-report survey). Response rates and numbers of respondents for the surveys are as follows: baseline, 6% and 1,034 respondents; pre-report, 6% and 1,025 respondents; post-report, 8% and 1,008 respondents. Although the nonresponse rates for the surveys are high, they are within industry standards for dual-frame sample telephone surveys. ²³ Additionally, studies indicate that higher response rates do not significantly affect estimates for demographic, social, and political items similar to those captured in these surveys. ^{23,Reference Keeter, Hatley, Kennedy and Lau24,Reference Keeter25,Reference Groves and Peytcheva26} To address any underrepresentation of certain demographic groups, the Annenberg Center introduced a weight, which was used in this analysis.

Table 2. Search strings used to identify the topics covered by articles about the GE crops report (includes the percentage of all articles about the GE crops report that included the terms).

Survey data and regression analyses

For analysis of the survey data, hierarchical ordinary least squares (OLS) regression models captured the change in risk and benefit perceptions. This model captures the relationships between independent variables and the dependent variable by analyzing “blocks” of the independent variables that are grouped by type and entered in their assumed causal order. Demographics are entered before values, for example, because gender and age are more likely to affect political ideology than vice versa. The OLS regression model adds the blocks of similar variables (demographic variables, value variables, etc.) one at a time to isolate how much of the total variance in individuals’ responses to the dependent variable each block explains.

This study used two OLS regression models. The first measured change in the dependent variable risk perception between the pre- and post-report survey (Table 3). The second used the same model but for the dependent variable benefit perception (Table 4). Both models included the following control and independent variables:

Table 3. OLS regression predicting risk perceptions of GMOs and change in perceptions pre- to post-report.

$N=1545$ . $\text{}^{\ast }p\leqslant 0.05$ ; $\text{}^{\ast \ast }p\leqslant 0.01$ ; $\text{}^{\ast \ast \ast }p\leqslant 0.001$ . Note: Reported coefficients for the interaction terms are before-entry coefficients to address multicollinearity effects between the interaction terms and their composite main effect variables.

Pre-/post-report is the independent variable and coded as a dummy variable, with pre-report survey respondents coded as 1 and post-report survey respondents coded as 2.

For demographic characteristics, the model controlled for age ( $M=48.43$ , $SD=20.07$ ), gender (female  $=$  2; 51.6%), race (nonwhite  $=$  2; 35.0%), education (highest attained; median  $=$  “some college, no degree”), and household income (median  $=$  $50,000 to less than $100,000). It also controlled for value-related items, including political ideology (“very conservative”  $=$  5; $M=3.12$ , $SD=1.21$ ), science reliable, and scientists unbiased. Science reliable is captured in a single item asking respondents how much they agree or disagree with this statement: “Scientific studies are reliable sources of information” (“strongly agree”  $=$  5; $M=3.56$ , $SD=1.05$ ). Scientists unbiased is a single item asking respondents how much they agree or disagree with this statement: “Scientists produce unbiased results and findings” ( $M=2.88$ , $SD=1.12$ ).

Media use was controlled for with four items measured on a seven-point scale ranging from “many times a day”  $=$  7 to “never”  $=$  1. Respondents were asked to indicate how often they use the following: (1) “Newspapers and news magazines online & offline” ( $M=3.95$ ; $SD=2.03$ ); (2) “Television online & offline” ( $M=5.11$ ; $SD=1.88$ ); (3) “Online-only news sites, such as Slate.com” ( $M=2.64$ ; $SD=2.00$ ); and (4) “Social media platforms such as Facebook and Twitter” ( $M=4.13$ ; $SD=2.42$ ).

Credibility in different actors was controlled for with three measures, each asking respondents, “When it comes to providing information on GMOs, how credible do you feel the following groups are?”: (1) university scientists (“extremely credible”  $=$  5; $M=3.48$ ; $SD=1.08$ ); (2) corporations ( $M=2.23$ , $SD=1.15$ ); and (3) activist groups like Greenpeace ( $M=2.87$ , $SD=1.25$ ).

Trust NAS report was controlled for with an item asking respondents, “How much trust, if any, would you have that a report from the NAS would be unbiased?” (“A great deal”  $=$  4; $M=2.49$ , $SD=0.93$ ). We use this item to capture trust in the NASEM, as well.

Figure 1. Topics of conversation about GMOs on Twitter before and after the release of the GE crops report. There are clear changes in focus from regulation and labeling to health and safety of the technology after the report was released.

The dependent variables risk/benefit perception are each a single item asking respondents how much they agree or disagree that “GMOs are risky/beneficial for society” (“Strongly agree”  $=$  5; risk: $M=3.33$ , $SD=1.16$ ; benefit: $M=2.81$ , $SD=1.15$ ).

Each of the interaction terms was created by standardizing the component variables and multiplying them to reduce multicollinearity effects: (1) scientific studies are reliable * pre-/post-report and (2) university scientists are credible * pre-/post-report. Reported coefficients for the interaction terms are before-entry coefficients to address multicollinearity effects between the interaction terms and their composite main effect variables.

Before beginning the OLS analyses, we first ran a one-way analysis of variance (ANOVA) to assess whether risk and benefit perceptions of GMOs had changed in the time from the baseline survey in January to the pre-report survey in May. This showed us whether the trends that we captured between the pre- and post-report surveys were part of a larger trend beyond the effects of the report (Table 5).

Results

Impact on media coverage and Twitter discourse

An analysis of coverage of the GE crops report in news outlets indicated that reporting primarily focused on health and safety issues. This focus of news coverage is not surprising given the salience of health and safety for public discourse of GMOs ^{Reference Runge16,Reference Ferber28} ; Twitter discussion mirrored this focus as well (Figure 1). The focus on health and safety issues marked a change in the GMO conversation, which had been dominated by coverage of regulation and labeling, following presidential candidate Bernie Sanders’s tweets urging passage of legislation requiring the labeling of foods containing GMOs in the weeks prior to the report’s release.

More surprising than the shift to a health and safety focus with media coverage of the GE crops report is that the coverage and subsequent discourse appears to have altered the sentiment of Twitter conversation about GMOs. In the week following the report’s release, tweets concerning GMOs became markedly less negative and more positive (Figure 2). A week after the report’s release, the conversation returned to pre-report levels of negativity, before changing to a more neutral discussion initiated by Sanders’s comments on labeling. Although negative sentiment rebounded to pre-report levels a week after the report’s release, the reduction in negativity immediately following the report’s release suggests that the report had a short-term effect on public discussion of GMOs.

Figure 2. Volume of tweets about GMOs and the proportion of related sentiments being expressed over time, with a sharp decrease in negative sentiment and increase in positive sentiment following the GE crops report release.

Impact on national public opinion

Importantly, the drop in negative comments on Twitter is consistent with our analysis of nationally representative public opinion data collected in the same week. Although Twitter offers a useful window into discussions within a specific subpopulation, only about 21% of the U.S. public uses Twitter, and it is not a nationally representative sample of U.S. adults. ^{Reference Greenwood, Perrin and Duggan29} To assess the larger impact of the report on public opinion of GMOs, we combined our large-scale media data with nationally representative pre- and post-report surveys: a baseline survey conducted in January 2016 (several months before the report’s release), a pre-report survey fielded the week before the report’s release, and a post-report survey run a few days after the release. Using this design, we captured public opinion coinciding with the report’s release while controlling for any natural variability in response by comparing with the baseline survey.

Figure 3. Interaction effects of (1) perceived credibility of university scientists for information on GMOs (left) and (2) perceived reliability of scientific reports (right) on individuals’ perceptions of the riskiness of GMOs to society pre- and post-report.

An OLS regression analysis indicated that there was an overall reduction in public perceptions of the societal risk of GMOs following the report’s release, after controlling for demographics, political ideology, media use, and beliefs about the reliability of scientific studies, objectivity of scientists’ results, and credibility of different actors (Table 3). However, we did not find a change in overall public perceptions of the societal benefit of GMOs (Table 4).

To assess whether this reduction in perceived risk came primarily from those members of the public most likely to be the “choir” for a report such as the NASEM GE crops report, we included two interaction terms in the model. The interactions allowed us to check for different changes in risk perceptions pre- to post-report release depending on a person’s trust in the credibility of scientists for information on GMOs and in the reliability of scientific reports, two characteristics that relate to how individuals perceive the information in a report like the NASEM GE crops report and to how they perceive the risks of GMOs. ^{Reference Savadori30,Reference Frewer, Scholderer and Bredahl31}

Controlling for other characteristics, we expected that after the report’s release, individuals with higher levels of trust and perceived credibility in scientists and in scientific reports would be more likely to change their perceptions of GMOs to be in line with the report findings (seeing reduced risk or a more mixed view of the risks and benefits) than those with low trust. Surprisingly, we found the opposite. Not only did the report coincide with an overall reduction in public perceptions of the societal risk associated with GMOs, but also the overall reduction appears to be primarily due to reduced risk perceptions among those individuals with low levels of trust in the credibility of scientists and in the reliability of scientific reports (Figure 3).

We also tested whether risk and benefit perceptions changed from the baseline survey to the pre-report survey to ensure that the decrease we saw in risk perceptions from pre- to post-report was not associated with a larger trend unrelated to the report. We did not find any significant changes in these models, which further supports the conclusion that the reduction in risk perceptions coinciding with the GE crops report’s release are in fact attributable to the report (Table 5).

Table 4. OLS regression predicting benefit perceptions of GMOs and change in perceptions pre- to post-report.

$N=\text{1,555}$ . $\text{}^{\ast }p\leqslant 0.05$ ; $\text{}^{\ast \ast }p\leqslant 0.01$ ; $\text{}^{\ast \ast \ast }p\leqslant 0.001$ . Notes: Pre-/post-report interacting with levels of belief in the credibility of university scientists was significant at the $p\leqslant 0.10$ level, but, given the large sample size of this study, it was not included as a significant result. Reported coefficients for the interaction terms are before-entry coefficients to address multicollinearity effects between the interaction terms and their composite main effect variables.

Discussion

The significant reduction in societal risk perceptions of GMOs — especially among those who would seem least likely either to trust scientific consensus reports or to have reduced risk perceptions following the report’s release — occurs despite only 18% of the post-report survey respondents indicating that they were at least somewhat aware that the NASEM had recently released the report, and less than half of the pre- and post-report respondents indicated that they had heard of the NASEM at all.

As seen in Table 3, the effect size of the pre- to post-report release for risk reduction is small. Even a small effect size is noteworthy, however, because of the complex information environment into which the report emerged and because of how few respondents were aware of the report at all. In the large amount of general news and GMO-specific information and discussion, the GE crops report was able to significantly impact public opinion in the days following the report’s release. Although the survey used in the study had a low response rate, our confidence in the conclusion that a consensus report can affect public opinion and discourse about a controversial scientifically relevant issue is bolstered by the fact that we saw similar effects across both the survey data and the sentiment of Twitter discussions.

Table 5. ANOVA testing for change in perceived benefit and risk of GMOs from the baseline national survey in January 2017 to the pre-report survey in early May 2017.

Given that negative sentiments about GMOs returned to pre-report levels in the week following the report’s release, it is also difficult to know what the reduction in public risk perceptions and the short-term reduction in negative sentiment of GMO Twitter discussion mean for the report’s long-term effects on opinion and discourse surrounding GMOs. Twitter, of course, is only one medium in which this discourse occurs, and we cannot capture here how substantial or lasting, and in what ways, the effects we find are for understanding overall public opinion of GMOs. To assess how substantial these changes are, we would also need to follow the effects of the GE crops report as it spreads through multiple pathways to interact with the many other public-opinion- and policy-shaping factors at play in the discussion about GMOs in the United States, which is beyond the scope of this analysis.

Large focusing events can change Twitter discussion of an issue over the long term, as the Fukushima Daiichi nuclear accident in 2011 did for discussion of nuclear energy. Following the accident, Twitter discourse of nuclear energy switched to a health and safety focus that lasted even after news coverage returned to the business and economic focus that had dominated nuclear energy discussion before the accident. ^{Reference Li32} As the GE crops report release is an event orders of magnitude smaller than the Fukushima accident, it is not surprising that Twitter discussion returned to a focus on legislation a week after the report release as GM labeling news recaptured news attention. Research on Twitter in particular, however, suggests that even short-term impacts can have ripple effects in other areas of discourse. One way this can occur is through the influence of “issue publics,” or Twitter users who closely follow and frequently discuss particular topics. ^{Reference Price33} These groups can disproportionately shape discussion because they are most invested in and vocal about a particular issue, which, in turn, can shape other people’s perceptions of the issue or perceptions of public opinion on the issue. ^{Reference Li32,Reference Price33,Reference Converse and Apter34}

Despite the low public awareness of both the report and NASEM, however, we do see that the GE crops report at least temporarily affected public discourse and changed media focus from legislation-dominated coverage to the health and safety and agronomic aspects of GMOs that the report covered. In the days following its release, the report reduced the level of negativity surrounding GMO discussions and increased public ambivalence — reduced risk perceptions without necessarily increasing benefit perceptions — about GE crops. The decreased negativity and perceived risk we observed in the Twitter discourse and in the survey are especially noteworthy in part because risk- and health-based concerns have been among the dominate foci of discussion and coverage of GMOs and GE crops for decades. ^{Reference Runge16,Reference Ferber28} This finding of the report reducing the perceptions of risk without increasing the perceptions of benefit could correspond to the ambivalent findings within the report itself or could reflect the framing in mainstream news of GMOs as safe but with several caveats. The increased ambivalence itself is important, however, because for a complex issue such as GMOs, not only are there mixtures of valid risks and benefits to weigh but also a large amount of misinformation, especially concerning the effects on human health. ^{Reference Lull, Scheufele, Jamieson, Kahan and Scheufele35} The GE crops report may have helped inform those considerations by providing evidence that enables a better understanding of particular risks and benefits while possibly countering considerations made on misinformation.

Of particular interest for furthering understanding of the impact of the mixed traditional and social media environment on information on GMOs is the finding that the largest reduction in perceived risk was among the portion of respondents who were least likely to trust such reports or to trust scientists for credible information about GMOs. These respondents, given their stated trust levels, would not be expected to have their attitudes move in agreement with the findings of the GE crops report if they knew it was the source of the incoming information on GMOs. If that expectation is true, that their attitudes did move in agreement with the report’s findings suggests that the report’s information could have reached these respondents through indirect pathways. By diffusing through traditional and social media news and discussion, the information of the report could have had greater opportunity to influence the perceptions of those members of the public who would otherwise be less trusting of information from a scientific report.

Those who do not believe that university scientists provide credible information on GMOs and those who do not believe that scientific studies are reliable are somewhat less likely to use social media than are other respondents in the survey data. That still leaves many who do report using social media platforms at least weekly: 40% of those who do not find university scientists credible and 57% of those who do not find scientific studies reliable. Information or affect from the GE crops report or discussion around it could have reached them through the Twitter coverage and discourse captured here or through other social media platforms, such as Facebook or reddit, not to mention through other media and interpersonal discussion. The breadth of avenues in the media environment mean that a wide-range of audiences could purposefully or incidentally receive information on the report. This breadth also means there could be opportunities for information on the report to reach people through sources they do trust, even if they do not find the original authors or the report itself credible or reliable. Research examining the interconnections of these pathways and how they reach and potentially influence different people is, of course, very difficult. Further studies mapping media diets and the interconnectedness of media platforms can expand understanding of the direct and indirect influences of information and discussion on perceptions of GMOs and other science issues.

Conclusion

Despite polarization and politics, or perhaps because of them, the NASEM appears to be able to play a role in providing information that can help shape media coverage and public discourse of science issues of national importance, at least in the short term in the case of GMOs. Simply providing people with more information alone typically does not change public opinion on nationally important and potentially controversial science issues, as the evidence synthesized in the recent NASEM report on communicating science indicates. ³⁶ This analysis suggests, however, that NASEM consensus reports can produce short-term effects on discussion and opinion in a confusing information landscape.