Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-8ctnn Total loading time: 0 Render date: 2024-12-25T13:05:53.210Z Has data issue: false hasContentIssue false

20 - The Modality Principle in Multimedia Learning

from Part V - Principles for Managing Essential Processing in Multimedia Learning

Published online by Cambridge University Press:  19 November 2021

By
Juan Cristobal Castro-Alonso  and
John Sweller
Edited by
Richard E. Mayer  and
Logan Fiorella
Richard E. Mayer
Affiliation:
University of California, Santa Barbara
Logan Fiorella
Affiliation:
University of Georgia
Get access

Summary

When dealing with instructional information, working memory can be divided into auditory and visual processors. The capacity limits of each processor are a major impediment when students are required to learn new material. Nevertheless, there is one strategy that can effectively expand working memory capacity by using the partially independent status of the auditory and visual processors. Under specific and well-defined conditions, presenting some information in visual mode and other information in auditory mode can increase effective working memory capacity and so reduce the effects of cognitive overload. This effect is called the instructional modality effect or modality principle. It is an instructional principle that can substantially increase learning. This chapter discusses the theory and data that underpin the principle and the instructional implications that flow from the principle.

Keywords

modality principleauditiory processorvisual processorsplit-attention effectcognitive load theory
Type
Chapter
Information
The Cambridge Handbook of Multimedia Learning , pp. 261 - 267
Publisher: Cambridge University Press
Print publication year: 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Baddeley, A. (1992). Working memory. Science, 255(5044), 556–559.CrossRefGoogle ScholarPubMed
Baddeley, A. (2012). Working memory: Theories, models, and controversies. Annual Review of Psychology, 63(1), 1–29.Google Scholar
Baddeley, A., & Hitch, G. (1974). Working memory. In Bower, G. H. (ed.), Psychology of Learning and Motivation (Vol. 8, pp. 47–89). New York: Academic Press.Google Scholar
Bauhoff, V., Huff, M., & Schwan, S. (2012). Distance matters: Spatial contiguity effects as trade-off between gaze switches and memory load. Applied Cognitive Psychology, 26(6), 863–871.Google Scholar
Brünken, R., Steinbacher, S., Plass, J. L., & Leutner, D. (2002). Assessment of cognitive load in multimedia learning using dual-task methodology. Experimental Psychology, 49(2), 109–119.Google Scholar
Castro-Alonso, J. C. (ed.) (2019). Visuospatial Processing for Education in Health and Natural Sciences. Cham, Switzerland: Springer.Google Scholar
Castro-Alonso, J. C., Ayres, P., & Sweller, J. (2019). Instructional visualizations, cognitive load theory, and visuospatial processing. In Castro-Alonso, J. C. (ed.), Visuospatial Processing for Education in Health and Natural Sciences (pp. 111–143). Cham, Switzerland: Springer.Google Scholar
Castro-Alonso, J. C., Ayres, P., Wong, M., & Paas, F. (2018). Learning symbols from permanent and transient visual presentations: Don’t overplay the hand. Computers & Education, 116, 1–13.CrossRefGoogle Scholar
Castro-Alonso, J. C., & Sweller, J. (2020). The modality effect of cognitive load theory. In Karwowski, W., Ahram, T., & Nazir, S. (eds.), Advances in Human Factors in Training, Education, and Learning Sciences: Proceedings of the AHFE 2019 International Conference on Human Factors in Training, Education, and Learning Sciences (pp. 75–84). Cham, Switzerland: Springer.Google Scholar
Chandler, P., & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62(2), 233–246.Google Scholar
Clark, J., & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3(3), 149–210.Google Scholar
Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87–185.Google Scholar
Craig, S. D., Gholson, B., & Driscoll, D. M. (2002). Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features, and redundancy. Journal of Educational Psychology, 94(2), 428–434.Google Scholar
Crooks, S. M., Cheon, J., Inan, F., Ari, F., & Flores, R. (2012). Modality and cueing in multimedia learning: Examining cognitive and perceptual explanations for the modality effect. Computers in Human Behavior, 28(3), 1063–1071.Google Scholar
De Renzi, E., & Nichelli, P. (1975). Verbal and non-verbal short-term memory impairment following hemispheric damage. Cortex, 11(4), 341–354.Google Scholar
Dinçer, S., & Doğanay, A. (2017). The effects of multiple-pedagogical agents on learners’ academic success, motivation, and cognitive load. Computers & Education, 111, 74–100.Google Scholar
Geary, D. C. (2008). An evolutionarily informed education science. Educational Psychologist, 43(4), 179–195.Google Scholar
Ginns, P. (2005). Meta-analysis of the modality effect. Learning and Instruction, 15(4), 313–331.Google Scholar
Harskamp, E. G., Mayer, R. E., & Suhre, C. (2007). Does the modality principle for multimedia learning apply to science classrooms? Learning and Instruction, 17(5), 465–477.Google Scholar
Horvath, J. C. (2014). The neuroscience of PowerPointTM. Mind, Brain, and Education, 8(3), 137–143.Google Scholar
Inan, F. A., Crooks, S. M., Cheon, J., Ari, F., Flores, R., Kurucay, M., & Paniukov, D. (2015). The reverse modality effect: Examining student learning from interactive computer-based instruction. British Journal of Educational Technology, 46(1), 123–130.Google Scholar
Jeung, H. J., Chandler, P., & Sweller, J. (1997). The role of visual indicators in dual sensory mode instruction. Educational Psychology, 17(3), 329–343.Google Scholar
Kirsner, K., & Smith, M. C. (1974). Modality effects in word identification. Memory & Cognition, 2(4), 637–640.CrossRefGoogle ScholarPubMed
Kroll, N. E. A., Parks, T., Parkinson, S. R., Bieber, S. L., & Johnson, A. L. (1970). Short-term memory while shadowing: Recall of visually and of aurally presented letters. Journal of Experimental Psychology, 85(2), 220–224.CrossRefGoogle ScholarPubMed
Kühl, T., Scheiter, K., Gerjets, P., & Edelmann, J. (2011). The influence of text modality on learning with static and dynamic visualizations. Computers in Human Behavior, 27(1), 29–35.CrossRefGoogle Scholar
Leahy, W., Chandler, P., & Sweller, J. (2003). When auditory presentations should and should not be a component of multimedia instruction. Applied Cognitive Psychology, 17(4), 401–418.Google Scholar
Leahy, W., & Sweller, J. (2011). Cognitive load theory, modality of presentation and the transient information effect. Applied Cognitive Psychology, 25(6), 943–951.CrossRefGoogle Scholar
Lee, H., & Mayer, R. E. (2018). Fostering learning from instructional video in a second language. Applied Cognitive Psychology, 32(5), 648–654.Google Scholar
Margrain, S. A. (1967). Short-term memory as a function of input modality. Quarterly Journal of Experimental Psychology, 19(2), 109–114.Google Scholar
Mayer, R. E., Fiorella, L., & Stull, A. (2020). Five ways to increase the effectiveness of instructional video. Educational Technology Research and Development, 68(3), 837–852.CrossRefGoogle Scholar
Mayer, R. E., & Moreno, R. (1998). A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology, 90(2), 312–320.Google Scholar
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81–97.Google Scholar
Moreno, R. (2006). Does the modality principle hold for different media? A test of the method-affects-learning hypothesis. Journal of Computer Assisted Learning, 22(3), 149–158.Google Scholar
Moreno, R., & Mayer, R. E. (1999). Cognitive principles of multimedia learning: The role of modality and contiguity. Journal of Educational Psychology, 91(2), 358–368.Google Scholar
Moreno, R., Mayer, R. E., Spires, H. A., & Lester, J. C. (2001). The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19(2), 177–213.Google Scholar
Mousavi, S. Y., Low, R., & Sweller, J. (1995). Reducing cognitive load by mixing auditory and visual presentation modes. Journal of Educational Psychology, 87(2), 319–334.Google Scholar
Murdock, B. B. Jr. (1971). Four-channel effects in short-term memory. Psychonomic Science, 24(4), 197–198.Google Scholar
Mutlu-Bayraktar, D., Cosgun, V., & Altan, T. (2019). Cognitive load in multimedia learning environments: A systematic review. Computers & Education, 141, 103618.CrossRefGoogle Scholar
Oberauer, K., Lewandowsky, S., Awh, E., Brown, G. D. A., Conway, A., Cowan, N., … Ward, G. (2018). Benchmarks for models of short-term and working memory. Psychological Bulletin, 144(9), 885–958.Google Scholar
Ong, W. J. (1982). Orality and Literacy: The Technologizing of the Word. New York: Methuen.Google Scholar
Paivio, A. (1969). Mental imagery in associative learning and memory. Psychological Review, 76(3), 241–263.Google Scholar
Paivio, A. (2014). Intelligence, dual coding theory, and the brain. Intelligence, 47, 141–158.CrossRefGoogle Scholar
Park, S. (2015). The effects of social cue principles on cognitive load, situational interest, motivation, and achievement in pedagogical agent multimedia learning. Educational Technology & Society, 18(4), 211–229.Google Scholar
Penney, C. G. (1980). Order of report in bisensory verbal short-term memory. Canadian Journal of Psychology, 34(2), 190–195.CrossRefGoogle ScholarPubMed
Penney, C. G. (1989). Modality effects and the structure of short-term verbal memory. Memory & Cognition, 17(4), 398–422.Google Scholar
Peterson, L. R., & Peterson, M. J. (1959). Short-term retention of individual verbal items. Journal of Experimental Psychology, 58(3), 193–198.Google Scholar
Reinwein, J. (2012). Does the modality effect exist? and if so, which modality effect? Journal of Psycholinguistic Research, 41(1), 1–32.Google Scholar
Rollins, H. A., & Thibadeau, R. (1973). The effects of auditory shadowing on recognition of information received visually. Memory & Cognition, 1(2), 164–168.Google Scholar
Sanders, A. F., & Schroots, J. J. F. (1969). Cognitive categories and memory span. III. Effects of similarity on recall. Quarterly Journal of Experimental Psychology, 21(1), 21–28.Google Scholar
Scheiter, K., Schüler, A., Gerjets, P., Huk, T., & Hesse, F. W. (2014). Extending multimedia research: How do prerequisite knowledge and reading comprehension affect learning from text and pictures. Computers in Human Behavior, 31, 73–84.Google Scholar
Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and Instruction, 12(3), 185–233.Google Scholar
Tabbers, H. K., Martens, R. L., & van Merriënboer, J. J. G. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology, 74(1), 71–81.Google Scholar
Tindall-Ford, S., Chandler, P., & Sweller, J. (1997). When two sensory modes are better than one. Journal of Experimental Psychology: Applied, 3(4), 257–287.Google Scholar
Wong, A., Leahy, W., Marcus, N., & Sweller, J. (2012). Cognitive load theory, the transient information effect and e-learning. Learning and Instruction, 22(6), 449–457.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×