Article published in:
Language-Learner Computer Interactions: Theory, methodology and CALL applications
Edited by Catherine Caws and Marie-Josée Hamel
[Language Studies, Science and Engineering 2] 2016
► pp. 187–213

Analysing multimodal resources in pedagogical online exchanges

Methodological issues and challenges

| Université Lumière Lyon 2, France
This chapter focuses on the contribution to web conferencing-based pedagogical synchronous interactions of meaning-making multimodal resources (spoken language as well as gesture, gaze, body posture and movement). The first part of the chapter explores different methodological approaches to the analysis of multimodal semiotic resources in online pedagogical interactions. Having presented an overview of what research into synchronous web-mediated online interaction can bring to the field of CALL, we discuss the importance of determining the relevant units of analysis which will impact the granularity of transcription and orient the ensuing analyses. With reference to three of our own studies, we then explore different methods for studying multimodal online exchanges depending on the research questions and units of analysis under investigation. To illustrate the various ethical, epistemological and methodological issues at play in the qualitative examination of multimodal corpora, the second part of the chapter presents a case study that identifies the different steps involved when studying online pedagogical exchanges, from the initial data-­collection phase to the transcription of extracts of the corpus for publication.
Keywords: multimodal corpora, multimodal resources, transcription, units of analysis, webcam, web-mediated pedagogical interaction
Available under the Creative Commons Attribution-NonCommercial-NoDerivatives (CC BY-NC-ND) 4.0 license.

For any use beyond this license, please contact the publisher at rights@benjamins.nl.

Published online: 14 June 2016
https://doi.org/10.1075/lsse.2.09coh
References

References

Adolphs, S., & Varter, R.
(2013) Spoken corpus linguistics. From monomodal to multimodal. New York, NY: Routledge.Google Scholar
Allwood, J., Ahlsén, E., Lund, J., & Sundqvist, J.
(2007) Multimodality in own communication management. InJ. Toivanen & P. Juel Henrichsen(Eds.), Current trends in research on spoken language in the Nordic countries (Vol. 2, pp. 10–19). Oulu, Finland: Oulu University Press.Google Scholar
Baldry, A., & Thibault, P.J.
(2006) Multimodal transcription and text analysis. London, United Kingdom: Equinox.Google Scholar
Bétrancourt, M., Guichon, N., & Prié, Y.
(2011) Assessing the use of a trace-based synchronous tool for distant language tutoring. InH. Spada, G. Stahl, N. Miyake & N. Law(Eds.), Connecting computer-supported collaborative learning to policy and practice: CSCL2011 conference proceedings. Volume 1 – Long papers (pp.478–485). International Society of the Learning Sciences.Google Scholar
Bezemer, J.
(2014) How to transcribe multimodal interaction?InC.D. Maier & S. Norris(Eds.), Texts, images and interaction: A reader in multimodality (pp.155–169). Berlin, Germany: Mouton de Gruyter. Google Scholar
Blake, R., & Zysik, E.
(2003) Who’s helping whom?: Learner/heritage-speakers’ networked discussions in Spanish. Applied Linguistics, 24(4), 519–544. CrossrefGoogle Scholar
Brouwer, C.E.
(2003) Word searches in NNS–NS interaction: Opportunities for language learning? The Modern Language Journal, 87, 534–545. CrossrefGoogle Scholar
Calbris, G.
(2011) Elements of meaning in gesture (Mary M. Copple, trans.). Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar
Cohen, C., & Guichon, N.
(2014)  Researching nonverbal dimensions in synchronous videoconferenced-based interactions . Paper presented at CALICO Conference , University of Athens, OH, USA.
Cosnier, J., & Develotte, C.
(2011) Le face à face en ligne, approche éthologique. InC. Develotte, R. Kern & M.-N. Lamy(Eds.), Décrire la conversation en ligne: Le face à face distanciel (pp. 27–50). Lyon, France: ENS Éditions.Google Scholar
De Chanay, H.
(2011) La construction de l’éthos dans les conversations en ligne. InC. Develotte, R. Kern & M.-N. Lamy(Eds.), Décrire la conversation en ligne: Le face à face distanciel (pp. 145–172). Lyon, France: ENS Éditions.Google Scholar
Détienne, F., & Traverso, V.
(Eds.) (2009) Méthodologies d’analyse de situations coopératives de conception. Nancy, France: Presses Universitaires de Nancy.Google Scholar
Doughty, C.J., & Long, M.H.
(2003) Optimal psycholinguistic environments for distance foreign language learning. Language Learning and Technology, 7(3), 50–80. Retrieved from http://​llt​.msu​.edu​/vol7num3​/pdf​/doughty​.pdfGoogle Scholar
Ellis, R., & Barkhuizen, G.P.
(2005) Analysing learner language. Oxford, United Kingdom: Oxford University Press.Google Scholar
Erickson, T.
(1999) Persistent conversation: An introduction. Journal of Computer-Mediated Communication, 4(4), article 1. CrossrefGoogle Scholar
Flewitt, R., Hampel, R., Hauck, M., & LancasterL.
(2009) What are multimodal transcription and data?InC. Jewitt(Ed.), The Routledge handbook of multimodal analysis (pp. 40–53). London, United Kingdom: Routledge.Google Scholar
Goodwin, M.H., & Goodwin, C.
(1986) Gesture and coparticipation in theactivity of searching for a word. Semiotica, 62(1-2), 51-75. CrossrefGoogle Scholar
Guichon, N., & Cohen, C.
(2014) The impact of the webcam on an online L2 interaction. Canadian Modern Language Journal, 70(3), 331–354. CrossrefGoogle Scholar
2016). Multimodality and CALL. InL. Murray & F. Farr(Eds.) Routledge handbook of language learning and technology pp. 509 521 London, United Kingdom Routledge
Guichon, N., & Tellier, M.
(Eds.) (forthcoming). Enseigner l’oral en ligne. Paris: Didier.
Guichon, N., & Wigham, C.R.
2016). A semiotic perspective on webconferencing-supported language teaching. ReCALL, 28(1), 62–82. Crossref
Ivarsson, J., Linderoth, J., & Saljö, R.
(2009) Representation in practices. A socio-cultural approach to multimodality in reasoning. InC. Jewitt(Ed.), The Routledge handbook of multimodal analysis (pp. 201–212). London, United Kingdom: Routledge.Google Scholar
Jewitt, C.
(Ed.) (2009) The Routledge handbook of multimodal analysis. London, United Kingdom: Routledge.Google Scholar
Kern, R.G.
(1995) Restructuring classroom interaction with networked computers: Effects on quantity and quality of language production. The Modern Language Journal, 79(4), 457–476. CrossrefGoogle Scholar
(2014) Technology as pharmakon: The promise and perils of the Internet for foreign language education. The Modern Language Journal, 98(1), 340–358. CrossrefGoogle Scholar
Kost, C.R.
(2008) Use of communication strategies in a synchronous CMC environment. InS. Sieloff Magnan(Ed.), Mediating discourse online (pp. 153–189). Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar
Lamy, M.-N., & Hampel, R.
(2007) Online communication in language learning and teaching. Houndmills, United Kingdom: Palgrave Macmillan. CrossrefGoogle Scholar
McCafferty, S.G.
(2008) Material foundations for second language acquisition: Gesture, metaphor and internalization. InS.G. McCafferty & G. Stam(Eds.), Gesture: Second language acquisition and classroom research (pp. 47–65). New York, NY: Routledge.Google Scholar
McCafferty, S.G., & Stam, G.
(Eds.) (2008) Gesture: Second language acquisition and classroom research. New York, NY: Routledge.Google Scholar
McNeill, D.
(1992) Hand and mind: What gestures reveal about thought. Chicago, IL: The University of Chicago Press.Google Scholar
Norris, S.
(2004) Analyzing multimodal interaction – A methodological framework. New York, NY: Routledge.Google Scholar
Ochs, E.
(1979) Transcription as Theory. In E. Ochs & B. Schieffelin (Eds.), Developmental Pragmatics (pp. 43-72). New York: Academic Press.Google Scholar
Pellettieri, J.
(2000) Negotiation in cyberspace: The role of chatting in the development of grammatical competence. InM. Warschauer & R. Kern(Eds.), Network-based language teaching: Concepts and practices (pp. 59–86). New York, NY: Cambridge University Press. CrossrefGoogle Scholar
Satar, H.M.
(2013) Multimodal language learner interactions via desktop videoconferencing within a framework of social presence: Gaze. ReCall, 25(1), 122–142. CrossrefGoogle Scholar
Siemens, G., & Baker, R.S.J.
(2012) Learning analytics and educational data mining: Towards communication and collaboration. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge , New York, NY, 252–254. Crossref
Sindoni, M.G.
(2013) Spoken and written discourse in online interactions: A multimodal approach. New York, NY: Routledge.Google Scholar
Sloetjes, H., & Wittenburg, P.
(2008) Annotation by category – ELAN and ISO DCR. Proceedings of the 6th International Conference on Language Resources and Evaluation , 816–820.
Stockwell, G.
(2010) Effects of multimodality in computer-mediated communication tasks. InM. Thomas & H. Reinders(Eds.), Task-based language learning and teaching with technology (pp. 83–104). London, United Kingdom: Continuum.Google Scholar
Tellier, M., & Stam, G.
(2010) Stratégies verbales et gestuelles dans l’explication lexicale d’un verbe d’action. InV. Rivière(Ed.), Spécificités et diversité des interactions didactiques (pp. 357–374). Paris, France: Rivenue Éditions.Google Scholar
ten Have, P.
(1999) Doing conversation analysis: A practical guide. London, United Kingdom: Sage.Google Scholar
Wang, Y.
(2007) Task design in videoconferencing-supported distance language learning. CALICO Journal, 24(3), 591–630. CrossrefGoogle Scholar
White, J., & Ranta, L.
(2002) Examining the interface between metalinguistic task performance and oral production in a second language. Language Awareness, 11(4), 259–290. CrossrefGoogle Scholar
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., & Sloetjes, H.
(2006) ELAN: A professional framework for multimodality research. Proceedings of Language Resources and Evaluation Conference , 1556–1559.
Yakura, E.K.
(2004.) “Informed consent” and other ethical conundrums in videotaping interactions. In P. Levine & R. Scollon(Eds.), Discourse and technology – Multimodal discourse analysis (pp. 184–195). Washington, DC: Georgetown University Press.Google Scholar