No CrossRef data available.
Article contents
Structural problems require structural solutions
Published online by Cambridge University Press: 30 August 2023
Abstract
Chater & Loewenstein criticize behavioral scientists' reliance on individual-level (“i-frame”) analysis, observing that this impoverishes policy interventions and stymies scientific progress. We extend their analysis to argue that structural factors bias and perpetuate behavioral science toward the i-frame. Addressing this problem fully will require structural changes to the training, peer review, and granting structures that confront research scientists.
- Type
- Open Peer Commentary
- Information
- Copyright
- Copyright © The Author(s), 2023. Published by Cambridge University Press
References
Adam, D. (2019). Science funders gamble on grant lotteries. Nature, 575(7785), 574–575.CrossRefGoogle ScholarPubMed
Ahmed, E. (2019). How to make innovative research the norm. Studies in Philosophy, Politics and Economics, 1(1), 19–23.Google Scholar
Avin, S. (2019). Mavericks and lotteries. Studies in History and Philosophy of Science Part A, 76, 13–23.CrossRefGoogle ScholarPubMed
Currie, A. (2019). Creativity, conservativeness & the social epistemology of science. Studies in History and Philosophy of Science Part A, 76, 1–4.CrossRefGoogle ScholarPubMed
Farah, M. J., & Hook, C. J. (2017). Trust and the poverty trap. Proceedings of the National Academy of Sciences of the USA, 114(21), 5327–5329.CrossRefGoogle ScholarPubMed
Feldon, D. F., Litson, K., Jeong, S., Blaney, J. M., Kang, J., Miller, C., … Roksa, J. (2019). Postdocs’ lab engagement predicts trajectories of PhD students’ skill development. Proceedings of the National Academy of Sciences of the USA, 116(42), 20910–20916.CrossRefGoogle ScholarPubMed
Heesen, R., & Bright, L. K. (2021). Is peer review a good idea? The British Journal for the Philosophy of Science, 72(3), 635–663.CrossRefGoogle Scholar
Keltner, D., Van Kleef, G. A., Chen, S., & Kraus, M. W. (2008). A reciprocal influence model of social power: Emerging principles and lines of inquiry. Advances in Experimental Social Psychology, 40, 151–192.CrossRefGoogle Scholar
Kraus, M. W., & Torrez, B. (2020). A psychology of power that is embedded in societal structures. Power, Status and Hierarchy, 33, 86–90.Google ScholarPubMed
Kuhn, T. S. (1970). The structure of scientific revolutions. University of Chicago Press.Google Scholar
Lauer, M. S. (2021). Long-term trends in the age of principal investigators supported for the first time on NIH R01-equivalent awards. NIH Extramural Nexus. https://nexus.od.nih.gov/all/2021/11/18/long-term-trends-in-the-age-of-principal-investigators-supported-for-the-first-time-on-nih-r01-awards/Google Scholar
Lauer, M. S., & Roychowdhury, D. (2021). Inequalities in the distribution of national institutes of health research project grant funding. eLife, 10, e71712.CrossRefGoogle ScholarPubMed
Luukkonen, T. (2012). Conservatism and risk-taking in peer review: Emerging ERC practices. Research Evaluation, 21(1), 48–60.CrossRefGoogle Scholar
Lykes, M. B. (2017). Community-based and participatory action research: Community psychology collaborations within and across borders. In Bond, M. A., Serrano-García, I., Keys, C. B., & Shinn, M. (Eds.), APA handbook of community psychology: Methods for community research and action for diverse groups and issues (pp. 43–58). American Psychological Association.Google Scholar
Morton, J. M. (2017). Reasoning under scarcity. Australasian Journal of Philosophy, 95(3), 543–559.CrossRefGoogle Scholar
National Research Council. (2005). Bridges to independence: Fostering the independence of new investigators in biomedical research.Google Scholar
Paluck, E. L., Porat, R., Clark, C. S., & Green, D. P. (2021). Prejudice reduction: Progress and challenges. Annual Review of Psychology, 72, 533–560.CrossRefGoogle ScholarPubMed
Pratto, F., Sidanius, J., & Levin, S. (2006). Social dominance theory and the dynamics of intergroup relations: Taking stock and looking forward. European Review of Social Psychology, 17(1), 271–320.CrossRefGoogle Scholar
Rowbottom, D. P. (2022). Peer review may not be such a bad idea: Response to Heesen and Bright. British Journal for Philosophy of Science, 73(4), 927–940.CrossRefGoogle Scholar
Sidanius, J., & Pratto, F. (2001). Social dominance: An intergroup theory of social hierarchy and oppression. Cambridge University Press.Google Scholar
Simon, A. F., & Wilder, D. (2018). Action research in social psychology. Archives of Scientific Psychology, 6(1), 169–177.CrossRefGoogle Scholar
Stanford, P. K. (2019). Unconceived alternatives and conservatism in science: The impact of professionalization, peer-review, and big science. Synthese, 196(10), 3915–3932.CrossRefGoogle Scholar
You have
Access
Chater & Loewenstein (C&L) have offered a useful criticism of behavioral scientists' reliance on individual-level (“i-frame”) analysis to investigate social problems, neglecting potential structural (“s-frame”) interventions. They focus on the example of nudges, but the problem is applicable to the behavioral sciences generally. For example, the emphasis on reducing of individual-level racial prejudice has masked the much broader potential impact of addressing racism at the systemic level (Paluck, Porat, Clark, & Green, Reference Paluck, Porat, Clark and Green2021).
C&L provide an important first step in drawing attention to this critical problem. We suggest building on their analysis by applying the content of their insight to the target of their criticism: Behavioral scientists' reliance on i-frame analysis. The priorities and activities of researchers throughout the behavioral sciences are affected by the field's structures and the social system in which those structures are embedded, not just by intellectual mistakes made by researchers about which frameworks to apply to a subject matter. We argue that structural changes are necessary for making good on the analysis they present, because the tendencies toward i-frame research and policy interventions that C&L identify are ones that are selected for and incentivized by these institutions.
Behavioral science research is structurally biased in favor of i-frame research in several respects. A hierarchically organized system of higher education trains and credentials most behavioral scientists, and a likewise organized system of research funding organizes their research activities.
The way researchers are trained ensures that i-frame research remains the predominant way of approaching psychological questions. Most published research in the behavioral sciences focuses on individuals rather than structural relationships. This includes the subdisciplines of psychology that aim to make contact with public policy interventions, like behavioral economics and social psychology. Extant research furnishes material with which new researchers are taught. Moreover, early career researchers are trained using an apprenticeship model. Senior investigators train lab members junior to them, which influences trainees' intellectual development (Feldon et al., Reference Feldon, Litson, Jeong, Blaney, Kang, Miller and Roksa2019).
Beyond the training process for new scientists, the incentive structures of peer review further entrench the i-frame. “Peer review” can be interpreted generously here: It refers not only to the process of evaluating work for publication in academic journals, but evaluating job candidates, grant proposals, tenure and promotion cases, and award-worthiness. All of these are a form of peer review, and as such subject to structural limitations.
At a minimum, peer review places considerable practical obstacles in the way of generating pathbreaking s-frame work. Reviewers will be more likely to recommend for publication papers that are on topics and hypotheses that they study and support (NCR, 2005); tenure and promotion are subject to the same logic. This lends an inherent conservatism to the peer review process (Kuhn, Reference Kuhn1970).
This problem is reinforced by the steadily rising median age of National Institutes of Health (NIH) grant recipients over the past four decades, which suggests an increase in the difficulty of accessing these grants (Lauer, Reference Lauer2021; Lauer & Roychowdhury, Reference Lauer and Roychowdhury2021; National Research Council, 2005). Researchers respond to this heightened risk by proposing more conservative projects that reflect the status quo (Luukkonen, Reference Luukkonen2012; National Research Council, 2005).
The risks of the bias toward the i-frame in training and peer review go beyond the question of which ideas get financial support – they also constrict what ideas are generated (Stanford, Reference Stanford2019). Grant-making processes are not simply making pathbreaking work harder to get funded. They are also selecting against the time, effort, and intellectual habits that might be required to come up with novel research in the first place. The greater prevalence of i-frame studies and training methods makes i-frame ways of interrogating problems more practically available than s-frame approaches. This in turn makes s-frame studies costlier to pursue for those few who still generate s-frame research projects after i-frame heavy professional training.
We should expect all of these factors to be robust against even the most cogently argued target articles. Behavioral scientists will likely continue to run the studies that earn them career advancement and research funding, despite compelling arguments against the predominant frames of their research.
There is nothing inherent to human behavior that requires the behavioral sciences to focus on the individual. There are a number examples of s-frame research that the field can look to. These include social dominance theory (Pratto, Sidanius, & Levin, Reference Pratto, Sidanius and Levin2006; Sidanius & Pratto, Reference Sidanius and Pratto2001); how class and social power affect prosocial behavior (Keltner, Van Kleef, Chen, & Kraus, Reference Keltner, Van Kleef, Chen and Kraus2008; Kraus & Torrez, Reference Kraus and Torrez2020); how basic decision-making processes are influenced by resource scarcity and social location (Farah & Hook, Reference Farah and Hook2017; Morton, Reference Morton2017; Morton & Paul, Reference Morton and Paul2019). We believe this research should complement rather than replace i-frame research.
Building a field that is hospitable to more s-frame research requires, at a minimum, reconstruction of the field's training and reward structures. Existing theoretical and empirical work suggests alternatives that could address these structural problems. There have been longstanding calls for the reintegration of approaches like action research and community psychology into academic institutions, which could help train new researchers in ways that encourage s-frame analysis (Lykes, Reference Lykes, Bond, Serrano-García, Keys and Shinn2017; Simon & Wilder, Reference Simon and Wilder2018). Possible alternatives to preproduction peer review of articles include postpublication review (Heesen & Bright, Reference Heesen and Bright2021; Rowbottom, Reference Rowbottom2022), and possible alternative research funding models include the use of grant lotteries rather than evaluated submissions (Adam, Reference Adam2019; Ahmed, Reference Ahmed2019; Avin, Reference Avin2019; Currie, Reference Currie2019). Regardless of whether we adopt these particular interventions, we will need to implement structural-level solutions if we want to make good on the promise of C&L's analysis.
Financial support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Competing interest
None.