Computer-assisted visual articulation feedback in L2 pronunciation instruction
A review
Jennifer Abel | University of British Columbia
Language learning is a multimodal endeavor; to improve their pronunciation in a new language, learners access not only auditory information about speech sounds and patterns, but also visual information about articulatory movements and processes. With the development of new technologies in computer-assisted pronunciation training (CAPT) comes new possibilities for delivering feedback in both auditory and visual modalities. The present paper surveys the literature on computer-assisted visual articulation feedback, including direct feedback that provides visual models of articulation and indirect feedback that uses visualized acoustic information as a means to inform articulation instruction. Our focus is explicitly on segmental features rather than suprasegmental ones, with visual feedback conceived of as providing visualizations of articulatory configurations, movements, and processes. In addition to discussing types of visual articulation feedback, we also consider the criteria for effective delivery of feedback, and methods of evaluation.
Keywords: multimodality, visual feedback, articulation, CAPT, segmental features
Article outline
- 1.Introduction
- 2.Situating visual articulation feedback in the context of L2 learning
- 3.Types of computer-assisted visual articulation feedback
- 3.1Indirect feedback using visualized acoustic information
- 3.2Direct feedback using visualized articulatory information
- 3.2.1Ultrasound-based feedback
- 3.2.2Feedback using intra-oral techniques
- 3.3Simulation approaches
- 4.Criteria for the effective use of visual articulation feedback
- 5.Methods of evaluation and reporting
- 6.Summary and conclusions
- Acknowledgements
-
References
Published online: 31 May 2018
https://doi.org/10.1075/jslp.00006.bli
https://doi.org/10.1075/jslp.00006.bli
References
Abberton, E., & Fourcin, A. J.
Akahane-Yamada, R., McDermott, E., Adaichi, T., Kawahara, H., & Pruitt, J. S.
(1998) Computer-based second language production training by using spectrographic representation and HMM-based speech recognition scores. Paper presented at the 1998 International Conference on Spoken Language Processing, Sydney, Australia. Retrieved from http://www.mirlab.org/conference_papers/International_Conference/ICSLP%201998/PDF/AUTHOR/SL980429.PDF (16 December, 2015).
Aliaga-García, C., & Mora, J. C.
Anderson, F.
Badin, P., Ben Youssef, A., Bailly, G., Elisei, F., & Hueber, T.
Ballard, K. J., Smith, H. D., Paramatmuni, D., McCabe, P., Theodoros, D. G., & Murdoch, B. E.
Bernhardt, B., Gick, B., Bacsfalvi, P., & Ashdown, J.
Bernhardt, B., Gick, B., Bacsfalvi, P., & Adler-Bock, M.
Boersma, P.
Bruce, C. J., Spittell, P. C., Montgomery, S. C., Bailey, K. R., Tajik, A. J., & Seward, J. B.
Carey, M.
Catford, J. C. & Pisoni, D. B.
Clarius
(2016) Wireless, handheld ultrasound for iOS and Android debuts. [Press release]. Retrieved from https://www.clarius.me/aium-debut-pr/.
Cleland, J., Scobbie, J. M., Nakai, S., & Wrench, A.
(2015) Helping children learn non-native articulations: the implications for ultrasound-based clinical intervention. Paper presented at the 2015 International Conference of Phonetic Sciences, Glasgow, Scotland. Retrieved from http://www.icphs2015.info/pdfs/Papers/ICPHS0698.pdf (12 August, 2015).
de Bot, C. L. J.
Demenko, G., Wagner, A., & Cylwik, N.
Dowd, A., Smith, J., & Wolfe, J.
Engwall, O.
Gick, B., Bernhardt, B., Bacsfalvi, P., & Wilson, I.
Hardison, D. M.
Hincks, R.
Jenson, P. G., & Westermeier, F. X.
(1968) The effect of visual feedback on pronunciation in foreign language learning. Retrieved from http://files.eric.ed.gov/fulltext/ED015689.pdf (29 August, 2015).
Kalikow, D. N., & Swets, J. A.
Kartushina, N., Hervais-Adelman, A., Frauenfelder, U. H., & Golestani, N.
Katz, W., Campbell, T., Wang, J., Farrar, E., Eubanks, J. C., Balasubramanian, A., Prabhakaran, B., & Rennaker, R.
(2014) Opti-Speech: A real-time, 3D visual feedback system for speech training. In Proceedings of Interspeech 2014, Singapore (pp.1174–1178). Retrieved from https://www.utdallas.edu/~wangjun/paper/Interspeech14_opti-speech.pdf (22 January, 2016).
Katz, W. F., & Mehta, S.
Kelsey, C. A., Minifie, F. D., & Hixon, T. J.
Lambacher, S.
Lee, J., Jang, J., & Plonksy, L.
Léon, P. R., & Martin, P.
Levis, J. M. & Pickering, L.
Levitt, J. S., & Katz, W. F.
Massaro, D. W., & Light, J.
Mattheyses, W. & Verhelst, W.
Moisik, S. R., Esling, J. H., Bird, S., & Lin, H.
Molholt, G.
Navarra, J., & Soto-Faraco, S.
Neri, A., Cucchiarini, C., Strik, H., & Boves, L.
Noguchi, M., Yamane, N., Tsuda, A., Kazama, M., Kim, B., & Gick, B.
Olson, D. J.
Öster, A. -M.
Ouni, S.
Patten, I., & Edmonds, L. A.
Pillot-Loiseau, C., Kamiyama, T., & Kocjančič Antolík, T.
(2015) French /y/-/u/ contrast in Japanese learners with/without ultrasound feedback: vowels, non-words and words. Paper presented at the 2015 International Conference of Phonetic Sciences, Glasgow, Scotland. Retrieved on from http://www.icphs2015.info/pdfs/Papers/ICPHS0485.pdf (12 August, 2015).
Quintana-Lara, M.
R Core Team
Saito, K.
Schwartz, B.
Stone, M.
Suemitsu, A., Dang, J., Ito, T., & Tiede, M.
Tateishi, M., & Winters, S.
(2013) Does ultrasound training lead to improved perception of a non-native sound contrast? Evidence from Japanese learners of English. Paper presented at the 2013 meeting of the Canadian Linguistic Association, Victoria, BC, Canada. Retrieved from http://homes.chass.utoronto.ca/~cla-acl/actes2013/Tateishi_and_Winters-2013.pdf (12 August, 2015).
Thomson, R., & Derwing, T.
Tilsen, S., Das, D., & McKee, B.
Truscott, J.
Tsui, H. M.
Vardanian, R. M.
Wilson, I.
Wilson, I., & Gick, B.
Wojtczak, J., & Bonadonna, P.
Wu, Y., Gendrot, C., Hallé, P., & Adda-Decker, M.
(2015) On improving the pronunciation of French /r/ in Chinese learners by using real-time ultrasound visualization. Paper presented at the 2015 International Conference of Phonetic Sciences, Glasgow, Scotland. Retrieved from http://www.icphs2015.info/pdfs/Papers/ICPHS0786.pdf (12 August, 2015).
Cited by
Cited by 18 other publications
Angelovska, Tanja
Barbier, Guillaume, Ryme Merzouki, Mathilde Bal, Shari R. Baum & Douglas M. Shiller
Bu, Yaohua, Tianyi Ma, Weijun Li, Hang Zhou, Jia Jia, Shengqi Chen, Kaiyuan Xu, Dachuan Shi, Haozhe Wu, Zhihan Yang, Kun Li, Zhiyong Wu, Yuanchun Shi, Xiaobo Lu & Ziwei Liu
Cibelli, Emily
Ciriza, María del Puy & Ahmed Rivera-Campos
Demir, Yusuf & Galip Kartal
Johnson, Khia Anne, Gloria Madeleine Mellesmoen, Roger Yu-Hsiang Lo & Bryan Gick
Klaus, Adam, Daniel R. Lametti, Douglas M. Shiller & Tara McAllister
Levis, John M.
Li, Joanne Jingwen, Samantha Ayala, Daphna Harel, Douglas M. Shiller & Tara McAllister
Mozaffari, M. Hamed & Won-Sook Lee
Roon, Kevin D., Jaekoo Kang & D.H. Whalen
Terband, Hayo, Aravind Namasivayam, Edwin Maas, Frits van Brenk, Marja-Liisa Mailend, Sanne Diepeveen, Pascal van Lieshout & Ben Maassen
This list is based on CrossRef data as of 23 april 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.