Book contents
- Critical Thinking in Psychology
- Critical Thinking in Psychology
- Copyright page
- Contents
- Figures
- Tables
- Contributors
- Preface
- Chapter 1 An Introduction to Critical Thinking: Maybe It Will Change Your Life
- Chapter 2 Nobelists Gone Wild
- Chapter 3 Why Science Succeeds, and Sometimes Doesn’t
- Chapter 4 Critical Thinking and the Rejection of Unsubstantiated Claims
- Chapter 5 Promoting Critical Thinking by Teaching, or Taking, Psychology Courses
- Chapter 6 Avoiding and Overcoming Misinformation on the Internet
- Chapter 7 Critical Thinking Impacts Our Everyday Lives
- Chapter 8 Research Suffers When We All Agree
- Chapter 9 When All Is Just a Click Away
- Chapter 10 Critical Thinking
- Chapter 11 Evaluating Experimental Research
- Chapter 12 Critical Thinking as Scientific Reasoning
- Chapter 13 Critical Thinking in STEM Disciplines
- Chapter 14 Why Would Anyone Do or Believe Such a Thing?
- Chapter 15 Conclusion: How to Think Critically about Politics … and Anything Else!
- Index
- References
Chapter 11 - Evaluating Experimental Research
Published online by Cambridge University Press: 19 December 2019
Book contents
- Critical Thinking in Psychology
- Critical Thinking in Psychology
- Copyright page
- Contents
- Figures
- Tables
- Contributors
- Preface
- Chapter 1 An Introduction to Critical Thinking: Maybe It Will Change Your Life
- Chapter 2 Nobelists Gone Wild
- Chapter 3 Why Science Succeeds, and Sometimes Doesn’t
- Chapter 4 Critical Thinking and the Rejection of Unsubstantiated Claims
- Chapter 5 Promoting Critical Thinking by Teaching, or Taking, Psychology Courses
- Chapter 6 Avoiding and Overcoming Misinformation on the Internet
- Chapter 7 Critical Thinking Impacts Our Everyday Lives
- Chapter 8 Research Suffers When We All Agree
- Chapter 9 When All Is Just a Click Away
- Chapter 10 Critical Thinking
- Chapter 11 Evaluating Experimental Research
- Chapter 12 Critical Thinking as Scientific Reasoning
- Chapter 13 Critical Thinking in STEM Disciplines
- Chapter 14 Why Would Anyone Do or Believe Such a Thing?
- Chapter 15 Conclusion: How to Think Critically about Politics … and Anything Else!
- Index
- References
Summary
Experiments allow researchers to determine cause and effect relations between variables. As such, they are a critical component in the advance of scientific psychology. In this chapter, we discuss the theory behind the design of good experiments, and provide a sample study for evaluation. We outline three important types of replication, and give an overview of historical events that led to a renewed vigilance regarding replicability. Finally, we discuss generalizability of research in terms of four factors: Subjects (or participants), materials used in the experiment, dependent measures, and the experimental situation. Effects that generalize across these sets of factors are robust. We end the chapter with a set of 18 critical-thinking questions that should be borne in mind while reading and evaluating experimental research. Referring to these questions will help to sharpen critical thinking skills about experimental research.
- Type
- Chapter
- Information
- Critical Thinking in Psychology , pp. 249 - 279Publisher: Cambridge University PressPrint publication year: 2020
References
Aarts, A. A., et al. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. DOI:10.1126/science.aac4716Google Scholar
Bacon, F. (1620/2000)The new organon. Eds. Jardine, L. & Silverthorne, M.. Originally published as Novum Organum, sive indicia vera de Interpretationes Naturae in 1620. Cambridge: Cambridge University Press.Google Scholar
Banaji, M. R., & Crowder, R. G. (1989). The bankruptcy of everyday memory. American Psychologist, 44, 1185–1193. DOI:10.1037/0003-066X.44.9.1185CrossRefGoogle Scholar
Bem, D. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100(3), 407–425. DOI:10.1037/a0021524CrossRefGoogle ScholarPubMed
Bones, A. K. (2012). We knew the future all along: Scientific hypothesizing is much more accurate than other forms of precognition – A satire in one part. Perspectives on Psychological Science, 7(3), 307–309. DOI:10.1177/1745691612441216CrossRefGoogle ScholarPubMed
Boring, E. G. (1929). A history of experimental psychology. New York and London: Century.Google Scholar
Camerer, C. F., et al. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature: Human Behavior, 2, 637–644. DOI:10.1038/s41562-018–0399–zGoogle Scholar
Craik, F. I. M., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology: General, 104(3), 263–294. DOI:10.1037/0096–3445.104.3.268Google Scholar
Cummings, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. New York: Routledge.Google Scholar
Deese, J. (1959). On the prediction of occurrence of particular verbal instructions in immediate recall. Journal of Experimental Psychology, 58, 17–22. DOI:10.1037/h0046671Google Scholar
Doyen, S., Klein, O., Pichon, C. L., & Cleeremans, A. (2012). Behavioral priming: It’s all in the mind, but whose mind? PloS ONE, 7(1), e2908. DOI:10.1371/journal.pone.0029081Google Scholar
Freyd, J. J., & Gleaves, D. H. (1996). “Remembering” words not presented in lists: Relevance to the current recovered/false memory controversy. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 811–813. DOI:10.1037/0278–7393.22.3.811Google Scholar
Ganley, C. M., Mingle, L. A., Ryan, A. M., Ryan, K., Vasilyeva, M., & Perry, M. (2013). An examination of stereotype threat effect on girls’ mathematics performance. Developmental Psychology, 49(10), 1886–1897. DOI:10.1037/a0031412Google Scholar
Gallo, D. A. (2010). False memories and fantastic beliefs: 15 years of the DRM illusion. Memory & Cognition, 37, 833–848. DOI:10.3758/MC.38.7.833Google Scholar
Gates, A. I. (1917). Recitation as a factor in memorizing. Archives of Psychology, 26, 1–104.Google Scholar
Gelman, A., & Loken, E. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Online. http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdfGoogle Scholar
Hagger, M. S., et al. (2016). A multilab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science, 11(4), 546–573. DOI:10.1177/1745691616652873Google Scholar
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015). The extent and consequences of P-hacking in science. PloS Biology, 13(3), e1002106. DOI:10.1371/journal.pbio.1002106CrossRefGoogle ScholarPubMed
Ioannidis, J. P. A. (2005). Why most published research findings are false. PloS Medicine, 2(8), 0696–0701. DOI:10.1371/journal.pmed.0020124Google Scholar
Jenkins, J. J. (1979). Four points to remember: A tetrahedral model of memory experiments. In Cermak, L. S. & Craik, F. I. M. (Eds.), Levels of processing in human memory (pp. 429–446). Hillsdale, NJ: Erlbaum.Google Scholar
Kahneman, D. (2012). A proposal to deal with questions about priming effects. Online. https://tinyurl.com/yyj8v7nxGoogle Scholar
Kantowitz, B. H., Roediger, III, H. L., & Elmes, D. (2015). Experimental psychology (10th ed.). Belmont, CA: Wadsworth.Google Scholar
Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196–217. DOI:10.1207/s15327957pspr0203_4CrossRefGoogle ScholarPubMed
Klein, R. A., et al. (2014). Investigating variation in replicability: A “many labs” replication project. Social Psychology, 45(3), 142–152. DOI:10.1027/1864-9335/a000178Google Scholar
McDermott, K. B., & Naaz, F. (2014). Is recitation an effective tool for adult learners? Journal of Applied Research in Memory and Cognition, 3, 207–213. DOI:10.1016/j.jarmac.2014.06.006Google Scholar
Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38, 379–387. DOI:10.1037/0003-066X.38.4.379CrossRefGoogle Scholar
Pashler, H., & Wagenmakers, E. J. (2012). Editors’ introduction to the special section on replicability in psychological science: A crisis of confidence? Perspectives on Psychological Science, 7(6), 528–530. DOI:10.1177/1745691612465253Google Scholar
Ranehill, E., Dreber, A., Johannesson, M., Leiberg, S., Sul, S., & Weber, R. A. (2015). Assessing the robustness of power posing: No effect on hormones and risk tolerance in a large sample of men and women. Psychological Science, 26(5), 653–656. DOI:10.1177/0956797614553946CrossRefGoogle Scholar
Roediger, III, H. L. (2012). Psychology’s woes and a partial cure: The value of replication. The APS Observer, 25(2), 9, 27–29.Google Scholar
Roediger, III, H. L., & Gallo, D. A. (2002). Processes affecting accuracy and distortion in memory: An overview. In Eisen, M. L., Goodman, G. S., & Quas, J. A. (Eds.), Memory and suggestibility in the forensic interview (pp. 3–28). Mahwah, NJ: Erlbaum.Google Scholar
Roediger, III, H. L., & Gallo, D. A. (2016). Associative memory illusions. In Pohl, R. F. (Ed.), Cognitive illusions: Intriguing phenomena in thinking, judgment and memory (2nd ed.) (pp. 390–405). New York: Psychology Press.Google Scholar
Roediger, III, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, 249–255. DOI:10.1111/j.1467–9280.2006.01693.xGoogle Scholar
Roediger, III, H. L., & McCabe, D. P. (2007) Evaluating Experimental Research: Critical Issues. In Sternberg, R. J., Roediger, III H. L., & Halpern, D. F. (Eds.), Critical thinking in psychology (pp. 15–36). New York: Cambridge University Press.Google Scholar
Roediger, III, H. L., & McDermott, K. (1995). Creating false memories: Remembering words not presented in lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 803–814. DOI:10.1037/0278–7393.21.4.803Google Scholar
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. DOI:10.1177/0956797611417632Google Scholar
Spellman, B. A. (2015). A short (personal) future history of Revolution 2.0. Perspectives on Psychological Science, 10(6), 886–899. DOI:10.1177/1745691615609918Google Scholar
Tulving, E. (1967). The effects of presentation and recall of material in free-recall learning. Journal of Verbal Learning and Verbal Behavior, 6, 175–184. DOI:10.1016/S0022–5371(67)80092–6Google Scholar
Tulving, E. (1985). Memory and consciousness. Canadian Psychology, 26, 1–12. DOI:10.1037/h0080017CrossRefGoogle Scholar
Underwood, B. J. (1965). False recognition produced by implicit verbal responses. Journal of Experimental Psychology, 70, 122–129. DOI:10.1037/h0022014Google Scholar
- 1
- Cited by