Multimodality, the study of the interaction of language with other semiotic resources such as images and sound resources, has significant implications for computer assisted language learning (CALL) with regards to understanding the impact of digital environments on language teaching and learning. In this paper, we explore recent manifestations of CALL in 3-D virtual worlds, illustrated by the example of Second Life. The multimodal analyses of a conventional face-to-face lesson and three language learning activities in Second Life highlight some of the affordances and challenges presented by 3-D virtual environments. The results suggest that while multimodal resources integrate naturally to facilitate language teaching and learning in an orderly, structured and goal-orientated manner in classroom lessons, the often uncoordinated use (or absence) of avatars’ gaze, facial expression, body posture, gesture, as well as the unclear proxemics and use of space pose problems for effective communication in a 3-D virtual world. In addition, a “technology-oriented” register, alongside traditional instructional and regulative genres and registers, is introduced to help students cope with the demands of learning a language in a 3-D virtual environment. The study raises the issue of the relative effectiveness of 3-D virtual worlds for language teaching and learning. In doing so, a digital approach to multimodal research is proposed in order to address the complexity of multimodal learning environments and the various challenges for CALL.

In this study of affordances for second language (L2) learning in World of Warcraft (WoW) group play, we compared three gameplay episodes spanning a semester-long course. Applying multimodal analysis framed by ecological, dialogical and distributed (EDD) views (Zheng and Newgarden, forthcoming), we explored four English as a second language learners’ verbalizations and avatar actions. Players learned to take skilled linguistic action as they coordinated recurrent WoW gameplay activities (questing, planning next moves, traveling, learning a skill, etc.). Frequent activities matched Common European Framework of Reference (CEFR) speaking proficiency descriptors, used widely in L2 teaching and learning (L2TL), providing evidence that players engaged in the types of communicative activities interaction-oriented classroom approaches develop. However, in the WoW context, interactions were not planned, but emerged as players dynamically directed the course of play. Furthermore, modalities of avatar-embodiment and conversing over Skype allowed players to flexibly integrate language and actions to co-act toward game goals, discuss non-game topics during play, or demonstrate comprehension with avatar actions alone, an affordance for less verbal players. This research builds on previous work (Zheng, Newgarden & Young, 2012) relating WoW’s multiplayer activities and L2 learners’ skilled linguistic actions. We refer to Chemero’s (2009) model of the animal-environment system to explain how L2 learners develop abilities to take skilled linguistic action by acting on affordances in WoW. The EDD framework presented may enable other researchers to account for more of the complexities involved in L2 learning in multimodal, multiplayer virtual environments.

Online language learning and teaching in multimodal contexts has been identified as one of the key research areas in computer-aided learning (CALL) (Lamy, 2013; White, 2014).1 This paper aims to explore meaning-making in online language learner interactions via desktop videoconferencing (DVC) and in doing so illustrate multimodal transcription and analysis as well as the application of theoretical frameworks from other fields. Recordings of learner DVC interactions and interviews are qualitatively analysed within a case study methodology. The analysis focuses on how semiotic resources available in DVC are used for meaning-making, drawing on semiotics, interactional sociolinguistics, nonverbal communication, multimodal interaction analysis and conversation analysis. The findings demonstrate the use of contextualization cues, five codes of the body, paralinguistic elements for emotional expression, gestures and overlapping speech in meaning-making. The paper concludes with recommendations for teachers and researchers using and investigating language learning and teaching in multimodal contexts.

Despite the availability and growing use of digital story software for authoring and instructional purposes, little is known about learners’ perceptions on its integration in the foreign language writing class. Following both a social semiotics approach and activity theory, this study focuses on six advanced Spanish learners’ perceptions about the production of a digital story in which they integrated a variety of modes (written, oral, images, sounds) and manipulated the semiotic resources within each mode (size, color, lines in the image mode), to convey meaning. Analyzing participants’ reflections, questionnaires, and online journals, results highlight learners’ (a) interpretation of the tools and artifacts and their effect on their understanding of a final product, (b) connections between short-term goal-oriented actions and the longer-term object-oriented activity of developing a multimodal text, and (c) linguistic reorientations when creating a digital story.

Despite a growing body of research on task-based language learning (TBLT) (Samuda & Bygate, 2008; Ellis, 2003), there is still little information available regarding the pedagogical design behind tasks and how they are implemented (Samuda & Bygate, 2008). Scholars in computer-mediated second language (L2) learning have called for research to fill in this gap by reflecting critically on task design and the subsequent implementation process (Fuchs, Hauck & Müller-Hartmann, 2012; Hampel, 2010; Hampel & Hauck, 2006; Hampel & Plaines, 2013; Hauck, 2010), instead of considering a task an “unproblematic fait accompli” (O’Dowd & Ware, 2009: 174). In response to this charge, the present case study provides a critical analysis of tasks carried out in a first-year German language course built around the weekly TV-series, Rosenheim-Cops (ZDF). These tasks drew from research on multimodality (Kress & van Leeuwen, 2001; Norris & Maier, 2014) providing a framework for understanding how multiple semiotic systems work together to create meaning. The insights provided by this study have relevance for research on multimodal task design (Hampel & Hauck, 2006) through examining the possibilities and limitations of beginner second language (L2) learners’ effective use of authentic resources. The results suggest that the tasks encouraged semiotic awareness, helped activate referential knowledge useful for accessing multimodal resources, and elicited a positive response to authentic L2 use in context. The challenges of implementing these tasks included variation in learners’ engagement with authentic multimedia resources, either due to L2 skills levels or the level of interest in the particular resource used.