Hostname: page-component-78c5997874-94fs2 Total loading time: 0 Render date: 2024-11-13T00:53:31.422Z Has data issue: false hasContentIssue false
  • English
  • Français

Multimodality in webconference-based language tutoring: An ecological approach integrating eye tracking

Published online by Cambridge University Press:  23 May 2022

Marco Cappellini Open the ORCID record for Marco Cappellini [Opens in a new window]  and
Yu-Yin Hsu Open the ORCID record for Yu-Yin Hsu [Opens in a new window]
Marco Cappellini
Affiliation:
Aix-Marseille University, Laboratoire Parole & Langage UMR 7309 CNRS, France ([email protected])
Yu-Yin Hsu
Affiliation:
The Hong Kong Polytechnic University, Hong Kong, China ([email protected])
Get access Rights & Permissions [Opens in a new window]

Abstract

Drawing on existing research with a holistic stance toward multimodal meaning-making, this paper takes an analytic approach to integrating eye-tracking data to study the perception and use of multimodality by teachers and learners. To illustrate this approach, we analyse two webconference tutoring sessions from a telecollaborative project involving pre-service teachers and learners of Mandarin Chinese. The tutoring sessions were recorded and transcribed multimodally, and our analysis of two types of conversational side sequences shows that the integration of eye-tracking data into an ecological approach provides richer results. Specifically, our proposed approach provided a window on the participants’ cognitive management of graphic and visual affordances during interaction and uncovered episodes of joint attention.

Keywords

webconferencingonline tutoringtelecollaborationeye trackingaffordancesmultimodality
Type
Research Article
Information
ReCALL , Volume 34 , Issue 3 , September 2022 , pp. 255 - 273
Check for updates
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of European Association for Computer Assisted Language Learning

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bavelas, J. B., Chovil, N., Coates, L. & Roe, L. (1995) Gestures specialized for dialogue. Personality and Social Psychology Bulletin, 21(4): 394–405. https://doi.org/10.1177/0146167295214010 CrossRefGoogle Scholar
Bezemer, J. & Kress, G. (2016) Multimodality, learning and communication: A social semiotic frame. London: Routledge. https://doi.org/10.4324/9781315687537 Google Scholar
Blin, F. (2016) The theory of affordances. In Caws, C. & Hamel, M.-J. (eds.), Language-learner computer interactions. Theory, methodology and CALL applications. Amsterdam: John Benjamins, 41–64. https://doi.org/10.1075/lsse.2.03bli Google Scholar
Bronfenbrenner, U. (1979) The ecology of human development: Experiments by nature and design. Cambridge MA: Harvard University Press.Google Scholar
Cappellini, M. (2016) Roles and scaffolding in teletandem interactions: A study of the relations between the sociocultural and the language learning dimensions in a French–Chinese teletandem. Innovation in Language Learning and Teaching, 10(1): 6–20. https://doi.org/10.1080/17501229.2016.1134859 CrossRefGoogle Scholar
Cappellini, M. (2021) Une approche multimodale intégrant l’oculométrie pour l’étude des interactions télécollaboratives par visioconférence. Les Cahiers de l’ASDIFLE, 31: 99–120.Google Scholar
Cappellini, M. & Azaoui, B. (2017) Sequences of normative evaluation in two telecollaboration projects: A comparative study of multimodal feedback through desktop videoconference. Language Learning in Higher Education, 7(1): 55–80. https://doi.org/10.1515/cercles-2017-0002 CrossRefGoogle Scholar
Cappellini, M. & Hsu, Y.-Y. (2018) Ce que l’oculométrie peut apporter dans une approche écologique des échanges en ligne. Une discussion épistémologique et une étude de cas. In Dejean-Thircuir, C., Mangenot, F., Nissen, E. & Soubrié, T. (eds.), Actes du colloque Échanger Pour Apprendre en Ligne 2018. http://hal.univ-grenoble-alpes.fr/EPAL/hal-02023002 Google Scholar
Cappellini, M. & Hsu, Y.-Y. (2020) When future teachers meet real learners through telecollaboration: An experiential approach to learn how to teach languages online. Journal of Virtual Exchange, 3: 1–11. https://doi.org/10.21827/jve.3.35751 Google Scholar
Cohen, C. (2017) Former à l’enseignement en ligne. In Guichon, N. & Tellier, M. (eds.), Enseigner l’oral en ligne: Une approche multimodale. Paris: Didier, 215–242.Google Scholar
Conklin, K., Pellicer-Sánchez, A. & Carrol, G. (2018) Eye-tracking: A guide for applied linguistics research. Cambridge: Cambridge University Press. https://doi.org/10.1017/9781108233279 CrossRefGoogle Scholar
Council of Europe (2001) Common European framework of reference for languages: Learning, teaching, assessment. Strasbourg: Language Policy Unit. http://www.coe.int/t/dg4/linguistic/source/framework_en.pdf Google Scholar
de Pietro, J.-F., Matthey, M. & Py, B. (1989) Acquisition et contrat didactique: les séquences potentiellement acquisitionnelles dans la conversation exolingue. In Weil, D. & Fugier, H. (eds.), Actes du troisième Colloque Régional de Linguistique. Strasbourg: Université des Sciences Humaines et Université Louis Pasteur, 99–124.Google Scholar
Develotte, C., Guichon, N. & Vincent, C. (2010) The use of the webcam for teaching a foreign language in a desktop videoconferencing environment. ReCALL, 22(3): 293–312. https://doi.org/10.1017/S0958344010000170 CrossRefGoogle Scholar
Develotte, C., Kern, R. & Lamy, M.-N. (eds.) (2011) Décrire la conversation en ligne: Le face à face distanciel. Lyon: ENS Editions.CrossRefGoogle Scholar
Dooly, M. (2010) The teacher 2.0. In Guth, S. & Helm, F. (eds.), Telecollaboration 2.0: Language, literacies and intercultural learning in the 21st century. Bern: Peter Lang, 277–303.Google Scholar
Dooly, M. (2016) ‘Please remove your avatar from my personal space’: Competences of the telecollaboratively efficient person. In O’Dowd, R. & Lewis, T. (eds.), Online intercultural exchange: Policy, pedagogy, practice. Abingdon: Routledge, 192–208.Google Scholar
Fuchs, C., Hauck, M. & Müller-Hartmann, A. (2012) Promoting learner autonomy through multiliteracy skills development in cross-institutional exchanges. Language Learning & Technology, 16(3): 82–102.Google Scholar
Gibson, J. J. (1979) The ecological approach to visual perception. London: Lawrence Erlbaum Associates.Google Scholar
Gillespie, J. (2020) CALL research: Where are we now? ReCALL, 32(2): 127–144. https://doi.org/10.1017/S0958344020000051 CrossRefGoogle Scholar
Goodwin, C. (2000) Action and embodiment within situated human interaction. Journal of Pragmatics, 32(10): 1489–1522. https://doi.org/10.1016/S0378-2166(99)00096-X CrossRefGoogle Scholar
Guichon, N. (2012) Vers l’intégration des TIC dans l’enseignement des langues. Paris: Didier.Google Scholar
Guichon, N. & Cohen, C. (2016) Multimodality and CALL. In Farr, F. & Murray, L. (eds.), The Routledge handbook of language learning and technology. London: Routledge, 509–521.Google Scholar
Guichon, N. & Tellier, M. (eds.) (2017) Enseigner l’oral en ligne: Une approche multimodale. Paris: Didier.Google Scholar
Guichon, N. & Wigham, C. R. (2016) A semiotic perspective on webconferencing-supported language teaching. ReCALL, 28(1): 62–82. https://doi.org/10.1017/S0958344015000178 CrossRefGoogle Scholar
Gullberg, M. & Holmqvist, K. (2006) What speakers do and what addressees look at: Visual attention to gestures in human interaction live and on video. Pragmatics & Cognition, 14(1): 53–82. https://doi.org/10.1075/pc.14.1.05gul CrossRefGoogle Scholar
Hampel, R. & Stickler, U. (2005) New skills for new classrooms: Training tutors to teach languages online. Computer Assisted Language Learning, 18(4): 311–326. https://doi.org/10.1080/09588220500335455 CrossRefGoogle Scholar
Hampel, R. & Stickler, U. (2012) The use of videoconference to support multimodal interaction in an online language classroom. ReCALL, 24(2): 116–137. https://doi.org/10.1017/S095834401200002X CrossRefGoogle Scholar
Hampel, R. & Stickler, U. (eds.) (2015) Developing online language teaching: Research-based pedagogies and reflective practices. New York: Palgrave Macmillan.CrossRefGoogle Scholar
Hauck, M. (2010) Telecollaboration: At the interface between multimodal and intercultural communicative competence. In Guth, S. & Helm, F. (eds.), Telecollaboration 2.0: Language, literacies and intercultural learning in the 21st century. Bern: Peter Lang, 219–244.Google Scholar
Helm, F. & Dooly, M. (2017) Challenges in transcribing multimodal data: A case study. Language Learning & Technology, 21(1): 166–185.Google Scholar
Holt, B. & Tellier, M. (2017) Conduire des explications lexicales. In Guichon, N. & Tellier, M. (eds.), Enseigner l'oral en ligne. Paris: Didier, 59–90.Google Scholar
Hubbard, P. (2017) An invitation to CALL: Foundations of computer-assisted language learning. In Son, J.-B. & Windeatt, S. (eds.), Language teacher education and technology: Approaches and practices. London: Bloomsbury, 153–168 Google Scholar
Jefferson, G. (1972) Side sequences. In Sudnow, D. (ed.), Studies in social interaction. New York: The Free Press, 294–338.Google Scholar
Kendon, A. (1982) The study of gesture: Some observations on its history. Recherches sémiotiques/Semiotic Inquiry, 2(1): 25–62.Google Scholar
Kessler, G. (2016) Technology standards for language teacher preparation. In Farr, F. & Murray, L. (eds.), The Routledge handbook of language learning and technology. Abingdon: Routledge, 57–70.Google Scholar
Kozar, O. (2016) Perceptions of webcam use by experienced online teachers and learners: A seeming disconnect between research and practice. Computer Assisted Language Learning, 29(4): 779–789. https://doi.org/10.1080/09588221.2015.1061021 CrossRefGoogle Scholar
Lamy, M.-N. & Hampel, R. (2007) Online communication in language learning and teaching. New York: Palgrave Macmillan. https://doi.org/10.1057/9780230592681 CrossRefGoogle Scholar
Mercier, H. & Sperber, D. (2017) The enigma of reason. London: Penguin.Google Scholar
Mondada, L. (2016) Challenges of multimodality: Language and the body in social interaction. Journal of Sociolinguistics, 20(3): 336–366. https://doi.org/10.1111/josl.1_12177 CrossRefGoogle Scholar
Mondada, L. & Pekarek Doehler, S. (2004) Second language acquisition as situated practice: Task accomplishment in the French second language classroom. The Modern Language Journal, 88(4): 501–518. https://doi.org/10.1111/j.0026-7902.2004.t01-15-.x CrossRefGoogle Scholar
O’Dowd, R. (2018) From telecollaboration to virtual exchange: State-of-the-art and the role of UNICollaboration in moving forward. Journal of Virtual Exchange, 1: 1–23. https://doi.org/10.14705/rpnet.2018.jve.1 CrossRefGoogle Scholar
O’Rourke, B. (2012) Using eye-tracking to investigate gaze behavior in synchronous computer-mediated communication for language learning. In Dooly, M. & O’Dowd, R. (eds.), Researching online foreign language interaction and exchange: Theories, methods and challenges. Bern: Peter Lang, 305–342.Google Scholar
Pekarek Doehler, S. (ed.) (2000) Approches interactionnistes de l’acquisition des langues étrangères: concepts, recherches, perspectives. Acquisition et Interaction en Langue Étrangère, 12: 1–19. https://doi.org/10.4000/aile.934 Google Scholar
Py, B. (2000) La construction interactive de la norme comme pratique et comme représentation. Acquisition et Interaction en Langue Étrangère, 12: 77–97. https://doi.org/10.4000/aile.1464 CrossRefGoogle Scholar
Rivens Mompean, A. & Cappellini, M. (2015) Teletandem as a complex learning environment: Looking for a model. DELTA, 31(3): 633–663. https://doi.org/10.1590/0102-4450430446379623426 CrossRefGoogle Scholar
Satar, H. M. (2016) Meaning-making in online language learner interactions via desktop videoconferencing. ReCALL, 28(3): 305–325. https://doi.org/10.1017/S0958344016000100 CrossRefGoogle Scholar
Satar, H. M. & Wigham, C. R. (2017) Multimodal instruction-giving practices in webconferencing-supported language teaching. System, 70: 63–80. https://doi.org/10.1016/j.system.2017.09.002 CrossRefGoogle Scholar
Satar, H. M. & Wigham, C. R. (2020) Delivering task instructions in multimodal synchronous online language teaching. Alsic, 23(1). https://doi.org/10.4000/alsic.4571 Google Scholar
Sert, O. & Balaman, U. (2018) Orientations to negotiated language and task rules in online L2 interaction. ReCALL, 30(3): 355–374. https://doi.org/10.1017/S0958344017000325 CrossRefGoogle Scholar
Sinclair, J. M. & Coulthard, R. M. (1975) Towards an analysis of discourse: The English used by teachers and pupils. Oxford: Oxford University Press.Google Scholar
Sloetjes, H. & Wittenburg, P. (2008) Annotation by category - ELAN and ISO DCR. In Calzolari, N., Choukri, K., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S. & Tapias, D. (eds.), Proceedings of the Sixth International Conference on Language Resources and Evaluation. Marrakech: European Language Resources Association, 816–820.Google Scholar
Sperber, D. & Wilson, D. (1986) Relevance: Communication and cognition. Oxford: Blackwell Publishing.Google Scholar
Stickler, U., Smith, B. & Shi, L. (2016) Using eye-tracking technology to explore online learner interactions. In Caws, C. & Hamel, M.-J. (eds.), Language-learner computer interactions: Theory, methodology and CALL applications. Amsterdam: John Benjamins, 163–186. https://doi.org/10.1075/lsse.2.08sti Google Scholar
van Lier, L. (2004) The ecology and semiotics of language learning: A sociocultural perspective. Dordrecht: Kluwer Academic. https://doi.org/10.1007/1-4020-7912-5 CrossRefGoogle Scholar
Wigham, C. R. (2017) A multimodal analysis of lexical explanation sequences in webconferencing-supported language teaching. Language Learning in Higher Education, 7(1): 81–108. https://doi.org/10.1515/cercles-2017-0001 CrossRefGoogle Scholar
Yamada, M. & Akahori, K. (2010) Awareness and performance through self- and partner’s image in videoconferencing. CALICO Journal, 27(1): 1–25. https://doi.org/10.11139/cj.27.1.1-25 CrossRefGoogle Scholar
Supplementary material: File

Cappellini and Hsu supplementary material

Cappellini and Hsu supplementary material

Download Cappellini and Hsu supplementary material(File)
File 26.6 KB