Multimedia Learning About Physical Systems

doi:10.1017/CBO9780511816819.029

28 - Multimedia Learning About Physical Systems

Published online by Cambridge University Press: 05 June 2012

Mary Hegarty

Edited by

Richard Mayer

Show author details

Mary Hegarty: Affiliation:
University of California, Santa Barbara
Richard Mayer: Affiliation:
University of California, Santa Barbara

Book contents

Get access

Summary

Abstract

This chapter examines how people understand diagrams, animations, and multimedia presentations in order to learn about physical systems. Comprehension of these representations is analyzed within an information-processing framework that specifies the cognitive processes involved in understanding external displays, including attention, encoding, inference, and integration of different representations. The chapter first reviews the literature on comprehension of physical systems from diagrams (both static and animated) alone and then reviews how people understand physical systems from multimedia presentations in which diagrams are augmented by verbal instruction. The picture that emerges is that diagram comprehension is an active process of knowledge construction, rather than a passive process of internalizing the information presented in an external display. Because multimedia understanding depends on active information processing, it can be influenced considerably by the abilities, skills, and knowledge of the student. What is learned from a multimedia display is jointly determined by aspects of the display and aspects of the learner.

Introduction

Diagrams accompanied by text have been a common means of representing and communicating information throughout history (Ferguson, 1992). In recent years, with advances in graphic technologies, innovations such as animations and interactive visualizations have made diagrammatic representations even more prevalent in scientific and technical discourse and in everyday life. These new media, and graphical displays in general, are believed to have enormous potential for education and training. But in order to realize this potential, we need basic research on how people comprehend and make inferences from diagrams and multimedia displays.

Type: Chapter
Information: The Cambridge Handbook of Multimedia Learning , pp. 447 - 466

DOI: https://doi.org/10.1017/CBO9780511816819.029 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2005

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Ainsworth, S., & Labeke, N. (2004). Multiple forms of dynamic representation. Learning and Instruction, 14, 241–255CrossRef Google Scholar

Baggett, W. B., & Graesser, A. C. (1995). Question answering in the context of illustrated expository text. Proceedings of the 17th Annual Conference of the Cognitive Science Society (pp. 334–339). Mahwah, NJ: Lawrence Erlbaum Associates

Britton, B. K., & Graesser, A. C. (Eds.). (1996). Models of understanding text. Mahwah, NJ: Lawrence Erlbaum AssociatesGoogle Scholar

Chi, M. T. H., Leeuw, N., Chiu, M., & LaVancher, C. (1994). Eliciting self-explanations improves learning. Cognitive Science, 18, 439–478Google Scholar

Cohen, C. A., Hegarty, M., Keehner, M., & Montello, D. R. (2003). Spatial abilities in the representation of cross sections. Proceedings of the 25th Annual Meeting of the Cognitive Science Society. Mahwah, NJ: ErlbaumGoogle Scholar

Dar, T., Joskowicz, L., & Rivlin, E. (1999). Understanding mechanical motion: From images to behaviors. Artificial Intelligence, 112, 147–179CrossRef Google Scholar

Eley, M. (1983). Representing the cross-sectional shapes of contour-mapped landforms. Human Learning, 2, 279–294Google Scholar

Faraday, P., & Sutcliffe, A. (1997a). An empirical study of attending and comprehending multimedia presentations. In Proceedings of the ACM Multimedia Conference (265–275). New York: ACM PressGoogle Scholar

Faraday, P., & Sutcliffe, A. (1997b). Designing effective multimedia presentations. In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI'97 (272–278). New York: ACM PressGoogle Scholar

Ferguson, E. S. (1992). Engineering and the Mind's Eye. Cambridge, MA: MIT PressGoogle Scholar

Ferguson, E. L., & Hegarty, M. (1995). Learning with real machines or diagrams: Application of knowledge to real-world problems. Cognition and Instruction, 13, 129–160CrossRef Google Scholar

Garg, A. X., Norman, G., & Sperotable, L. (2001). How medical students learn spatial anatomy. The Lancet, 357, 363–364CrossRef Google Scholar PubMed

Garg, A. X., Norman, G. R., Spero, L., & Maheshwari, P. (1999). Do virtual computer models hinder anatomy learning. Academic Medicine, 74, 87–89CrossRef Google Scholar PubMed

Hegarty, M., (1992). Mental animation: Inferring motion from static diagrams of mechanical systems. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 1084–1102Google Scholar PubMed

Hegarty, M. (2004). Mechanical reasoning as mental simulation. TRENDS in Cognitive Sciences, 8, 280–285CrossRef Google Scholar PubMed

Hegarty, M., & Just, M. A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32, 717–742CrossRef Google Scholar

Hegarty, M., Kriz, S., & Cate, C. (2003). The roles of mental animations and external animations in understanding mechanical systems. Cognition & Instruction, 21, 325–360CrossRef Google Scholar

Hegarty, M., & Kozhevnikov, M. (1999). Spatial abilities, working memory and mechanical reasoning. In Gero, J. & Tversky, B. (Eds.), Visual and spatial reasoning in design (pp. 221–239). Sydney, Australia: Key Centre of Design and CognitionGoogle Scholar

Hegarty, M., Narayanan, N. H., & Freitas, P. (2002). Understanding machine from multimedia and hypermedia presentations. In Otero, J., Graesser, A. C., & Leon, J. (Eds.), The psychology of science text comprehension (pp. 357–384). Mahwah, NJ: Lawrence Erlbaum AssociatesGoogle Scholar

Hegarty, M., Quillici, J., Narayanan, N. H., Holmquist, S., & Moreno, R. (1999). Multimedia instruction: Lessons from evaluation of a theory-based design. Journal of Educational Multimedia and Hypermedia, 8, 119–150Google Scholar

Hegarty, M., & Sims, V. K. (1994). Individual differences in mental animation during mechanical reasoning. Memory and Cognition, 22, 411–430CrossRef Google Scholar PubMed

Hegarty, M., & Steinhoff, K. (1997). Use of diagrams as external memory in a mechanical reasoning task. Learning and Individual Differences, 9, 19–42CrossRef Google Scholar

Heiser, J., & Tversky, B. (2002). Diagrams and descriptions in acquiring complex systems. Proceedings of the 24th Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum AssociatesGoogle Scholar

Isaak, M. I., & Just, M. A. (1995). Constraints on the processing of rolling motion: The curtate cycloid illusion. Journal of Experimental Psychology: Human Perception & Performance, 21(6): 1391–1408Google Scholar PubMed

Kaiser, M. K., Proffitt, D. R., Whelan, S. M., & Hecht, H. (1992). Influence of animation on dynamical judgments. Journal of Experimental Psychology: Human Perception and Performance, 18, 669–690Google Scholar PubMed

Kali, Y., & Orion, N. (1996). Spatial abilities of high school students in the perception of geologic structures. Journal of Research in Science Teaching, 33, 369–3913.0.CO;2-Q>CrossRef Google Scholar

Kintsch, W., Britton, B. K., Fletcher, C. R., Mannes, S. M., & Nathan, M. J. (1993). A comprehension-based approach to learning and understanding. In Medin, D. L. (Ed.), The psychology of learning and motivation (Vol. 30, pp. 165–214). New York: Academic PressGoogle Scholar

Kriz, S., & Hegarty, M. (2004). Constructing and revising mental models of a mechanical system: The role of domain knowledge in understanding external visualizations. In Forbus, K., Gentner, D., & Regier, T. (Eds.), Proceedings of the 26th Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum AssociatesGoogle Scholar

Larkin, J. H., & Simon, H. A. (1987). Why a diagram is (sometimes) worth ten thousand words. Cognitive Science, 11, 65–100CrossRef Google Scholar

Lowe, R. K. (1999). Extracting information from an animation during complex visual learning. European Journal of Psychology of Education, 14, 225–244CrossRef Google Scholar

Lowe, R. (2003). Animation and learning: Selective processing of information in dynamic graphics. Learning and Instruction, 13, 157–176CrossRef Google Scholar

Markman, A. B. (1999). Knowledge representation. Mahwah, NJ: Lawrence Erlbaum AssociatesGoogle Scholar

Mayer, R. E., & Anderson, R. B. (1991). Animations need narrations: An experimental test of the dual-code hypothesis. Journal of Educational Psychology, 83, 484–490CrossRef Google Scholar

Mayer, R. E., & Gallini, J. (1990). When is an illustration worth ten thousand words?Journal of Educational Psychology, 82, 715–726CrossRef Google Scholar

Mayer, R. E., & Sims, V. K. (1994). For whom is a picture worth a thousand words? Extensions of a dual-coding theory of multimedia learning. Journal of Educational Psychology, 86, 389–401CrossRef Google Scholar

Macaulay, D. (1988). The way things work. Boston: Houghton Mifflin CompanyGoogle Scholar

Macaulay, D. (1998). The New Way Things Work [CD-ROM]. New York: DK Interactive LearningGoogle Scholar

McCloskey, M. (1983). Naïve theories of motion. In Gentner, D. & Stevens, A. (Eds.), Mental models (pp. 229–324). Mahwah, NJ: ErlbaumGoogle Scholar

McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge and levels of understanding in learning from text. Cognition and Instruction, 14, 1–43CrossRef Google Scholar

Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91, 358–368CrossRef Google Scholar

Moreno, R., & Mayer, R. E. (2000). A coherence effect in multimedia learning: The case for minimizing irrelevant sounds in the design of multimedia instructional messages. Journal of Educational Psychology, 92, 117–125CrossRef Google Scholar

Narayanan, N. H., & Hegarty, M. (1998). On designing comprehensible hypermedia manuals. International Journal of Human-Computer Studies, 48, 267–301CrossRef Google Scholar

Narayanan, N. H., & Hegarty, M. (2002). Multimedia design for communication of dynamic information. International Journal of Human-Computer Studies, 57, 279–315CrossRef Google Scholar

Narayanan, N. H., Suwa, M., & Motoda, H. (1994). A study of diagrammatic reasoning from verbal and gestural data. Proceedings of the 16th Annual Conference of the Cognitive Science Society (pp. 652–657). Mahwah, NJ: Lawrence Erlbaum AssociatesGoogle Scholar

Oestermeier, U., & Hesse, F. W. (2000). Verbal and visual causal arguments. Cognition, 75, 65–104CrossRef Google Scholar PubMed

Paivio, A. (1986). Mental representations: A dual-coding approach. New York: Oxford University PressGoogle Scholar

Palmer, S. E. (1978). Fundamental aspects of cognitive representation. In Rosch, E. E. & Lloyd, B. B. (Eds.), Cognition and Categorization (pp. 259–303) Mahwah, NJ: Lawrence Erlbaum AssociatesGoogle Scholar

Palmiter, S. L., & Elkerton, J., (1993). Animated demonstrations for learning procedural computer-based tasks. Human-Computer Interaction, 8, 193–216CrossRef Google Scholar

Palmiter, S. L., Elkerton, J., & Baggett, P. (1991). Animated demonstrations vs. written instructions for learning procedural tasks: A preliminary investigation. International Journal of Man-Machine Studies, 34, 687–701CrossRef Google Scholar

Pane, J. F., Corbett, A. T., & John, B. E. (1996). Assessing dynamics in computer-based instruction. In Tauber, M. J. (Ed.), Proceedings of the ACM Conference on Human Factors in Computing Systems (pp. 797–804). New York: ACMGoogle Scholar

Park, O. C., & Gittelman, S. S. (1992). Selective use of animation and feedback in computer-based instruction. Educational Technology, Research and Development, 40, 125–167CrossRef Google Scholar

Rensink, R. (2002). Change detection. Annual Review of Psychology, 53, 245–277CrossRef Google Scholar PubMed

Rieber, L. P. (1989). The effects of computer animated elaboration strategies and practice on factual and application learning in an elementary science lesson. Journal of Educational Computing Research, 5, 431–444CrossRef Google Scholar

Rieber, L. P. (1990). Using computer animated graphics with science instruction with children. Journal of Educational Psychology, 83, 135–140CrossRef Google Scholar

Rieber, L. P. (1991). Animation, incidental learning, and continuing education. Journal of Educational Psychology, 83, 318–328CrossRef Google Scholar

Rieber, L. P., & Hannafin, M. J. (1988). Effects of textual and animated orienting activities and practice on learning from computer-based instruction. Computers in the Schools, 5, 77–89CrossRef Google Scholar

Rozenblit, L. G., & Keil, F. C. (2002). The misunderstood limits of folk science: An illusion of explanatory depth. Cognitive Science, 26, 521–562CrossRef Google Scholar

Scaife, M., & Rogers, Y. (1996). External cognition: How do graphical representations work?International Journal of Human-Computer Studies, 45, 185–213CrossRef Google Scholar

Schwan, S., & Riempp, R. (2004). The cognitive benefits of interactive videos. Learning to tie nautical knots. Learning and Instruction, 14, 275–291CrossRef Google Scholar

Schwartz, D. L. (1995). Reasoning about the referent of a picture versus reasoning about the picture as the referent: An effect of visual realism. Memory and Cognition, 23, 709–722CrossRef Google Scholar PubMed

Schwartz, D. L., and Black, J. B. (1996a). Shuttling between depictive models and abstract rules: Induction and fall-back. Cognitive Science 20, 457–497CrossRef Google Scholar

Schwartz, D. L., and Black, J. B. (1996b). Analog imagery in mental model reasoning: Depictive models. Cognitive Psychology 30, 154–219CrossRef Google Scholar

Slamecka, N.J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4, 592–604Google Scholar

Stenning, K. (1998). Distinguishing semantic from processing explanations of usability of representations: Applying expressiveness analysis to animation. In Lee, J. (Ed.), Intelligence and multimodality in multimedia interfaces: Research and applications. Cambridge, MA: AAAI PressGoogle Scholar

Tufte, E. R. (1997). Visual explanations: Images and quantities, evidence and narrative. Cheshire, CT: Graphics PressGoogle Scholar

Tversky, B., Morrison, J. B., & Betrancourt, M. (2002). Animation: Can it facilitate?International Journal of Human-Computer Studies, 57, 247–262CrossRef Google Scholar

Zacks, J. M., & Tversky, B. (2001). Event structure in perception and conception. Psychological Bulletin, 27, 3–21CrossRef Google Scholar