Computer-assisted visual articulation feedback in L2 pronunciation instruction
Language learning is a multimodal endeavor; to improve their pronunciation in a new language, learners access not only auditory information about speech sounds and patterns, but also visual information about articulatory movements and processes. With the development of new technologies in computer-assisted pronunciation training (CAPT) comes new possibilities for delivering feedback in both auditory and visual modalities. The present paper surveys the literature on computer-assisted visual articulation feedback, including direct feedback that provides visual models of articulation and indirect feedback that uses visualized acoustic information as a means to inform articulation instruction. Our focus is explicitly on segmental features rather than suprasegmental ones, with visual feedback conceived of as providing visualizations of articulatory configurations, movements, and processes. In addition to discussing types of visual articulation feedback, we also consider the criteria for effective delivery of feedback, and methods of evaluation.
Keywords: multimodality, visual feedback, articulation, CAPT, segmental features
Published online: 31 May 2018
Abberton, E., & Fourcin, A. J.
Akahane-Yamada, R., McDermott, E., Adaichi, T., Kawahara, H., & Pruitt, J. S.
(1998) Computer-based second language production training by using spectrographic representation and HMM-based speech recognition scores. Paper presented at the 1998 International Conference on Spoken Language Processing, Sydney, Australia. Retrieved from http://www.mirlab.org/conference_papers/International_Conference/ICSLP%201998/PDF/AUTHOR/SL980429.PDF (16 December, 2015).
Aliaga-García, C., & Mora, J. C.
Badin, P., Ben Youssef, A., Bailly, G., Elisei, F., & Hueber, T.
Ballard, K. J., Smith, H. D., Paramatmuni, D., McCabe, P., Theodoros, D. G., & Murdoch, B. E.
Bernhardt, B., Gick, B., Bacsfalvi, P., & Ashdown, J.
Bernhardt, B., Gick, B., Bacsfalvi, P., & Adler-Bock, M.
Bruce, C. J., Spittell, P. C., Montgomery, S. C., Bailey, K. R., Tajik, A. J., & Seward, J. B.
Catford, J. C. & Pisoni, D. B.
(2016) Wireless, handheld ultrasound for iOS and Android debuts. [Press release]. Retrieved from https://www.clarius.me/aium-debut-pr/.
Cleland, J., Scobbie, J. M., Nakai, S., & Wrench, A.
(2015) Helping children learn non-native articulations: the implications for ultrasound-based clinical intervention. Paper presented at the 2015 International Conference of Phonetic Sciences, Glasgow, Scotland. Retrieved from http://www.icphs2015.info/pdfs/Papers/ICPHS0698.pdf (12 August, 2015).
de Bot, C. L. J.
Demenko, G., Wagner, A., & Cylwik, N.
Dowd, A., Smith, J., & Wolfe, J.
Gick, B., Bernhardt, B., Bacsfalvi, P., & Wilson, I.
Hardison, D. M.
Jenson, P. G., & Westermeier, F. X.
(1968) The effect of visual feedback on pronunciation in foreign language learning. Retrieved from http://files.eric.ed.gov/fulltext/ED015689.pdf (29 August, 2015).
Kalikow, D. N., & Swets, J. A.
Kartushina, N., Hervais-Adelman, A., Frauenfelder, U. H., & Golestani, N.
Katz, W., Campbell, T., Wang, J., Farrar, E., Eubanks, J. C., Balasubramanian, A., Prabhakaran, B., & Rennaker, R.
(2014) Opti-Speech: A real-time, 3D visual feedback system for speech training. In Proceedings of Interspeech 2014, Singapore (pp.1174–1178). Retrieved from https://www.utdallas.edu/~wangjun/paper/Interspeech14_opti-speech.pdf (22 January, 2016).
Katz, W. F., & Mehta, S.
Kelsey, C. A., Minifie, F. D., & Hixon, T. J.
Lee, J., Jang, J., & Plonksy, L.
Léon, P. R., & Martin, P.
Levis, J. M. & Pickering, L.
Levitt, J. S., & Katz, W. F.
Massaro, D. W., & Light, J.
Mattheyses, W. & Verhelst, W.
Moisik, S. R., Esling, J. H., Bird, S., & Lin, H.
Navarra, J., & Soto-Faraco, S.
Neri, A., Cucchiarini, C., Strik, H., & Boves, L.
Noguchi, M., Yamane, N., Tsuda, A., Kazama, M., Kim, B., & Gick, B.
Olson, D. J.
Öster, A. -M.
Patten, I., & Edmonds, L. A.
Pillot-Loiseau, C., Kamiyama, T., & Kocjančič Antolík, T.
(2015) French /y/-/u/ contrast in Japanese learners with/without ultrasound feedback: vowels, non-words and words. Paper presented at the 2015 International Conference of Phonetic Sciences, Glasgow, Scotland. Retrieved on from http://www.icphs2015.info/pdfs/Papers/ICPHS0485.pdf (12 August, 2015).
R Core Team
Suemitsu, A., Dang, J., Ito, T., & Tiede, M.
Tateishi, M., & Winters, S.
(2013) Does ultrasound training lead to improved perception of a non-native sound contrast? Evidence from Japanese learners of English. Paper presented at the 2013 meeting of the Canadian Linguistic Association, Victoria, BC, Canada. Retrieved from http://homes.chass.utoronto.ca/~cla-acl/actes2013/Tateishi_and_Winters-2013.pdf (12 August, 2015).
Thomson, R., & Derwing, T.
Tilsen, S., Das, D., & McKee, B.
Tsui, H. M.
Vardanian, R. M.
Wilson, I., & Gick, B.
Wojtczak, J., & Bonadonna, P.
Wu, Y., Gendrot, C., Hallé, P., & Adda-Decker, M.
(2015) On improving the pronunciation of French /r/ in Chinese learners by using real-time ultrasound visualization. Paper presented at the 2015 International Conference of Phonetic Sciences, Glasgow, Scotland. Retrieved from http://www.icphs2015.info/pdfs/Papers/ICPHS0786.pdf (12 August, 2015).[ p. 152 ]
Cited by other publications
Ciriza, María del Puy & Ahmed Rivera-Campos
Johnson, Khia Anne, Gloria Madeleine Mellesmoen, Roger Yu-Hsiang Lo & Bryan Gick
Klaus, Adam, Daniel R. Lametti, Douglas M. Shiller & Tara McAllister
Levis, John M.
Li, Joanne Jingwen, Samantha Ayala, Daphna Harel, Douglas M. Shiller & Tara McAllister
Terband, Hayo, Aravind Namasivayam, Edwin Maas, Frits van Brenk, Marja-Liisa Mailend, Sanne Diepeveen, Pascal van Lieshout & Ben Maassen
This list is based on CrossRef data as of 31 october 2020. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.