Article In: AILA Review: Online-First Articles
Opportunities associated with emerging technologies and oral language assessment
This content is being prepared for publication; it may be subject to changes.
Abstract
In recent years, there has been a rapid increase in the use of technology to assist in the delivery of second language oral communication (OC) assessments. In particular, video mediation, virtual reality, and spoken dialogue systems (SDSs) have been employed for this purpose. These emerging technologies have helped in assessing aspects of the OC construct that have been previously neglected, such as interactional competence. Thus, the purpose of this paper is to showcase how these technologies have been incorporated into OC assessments, what aspects of the construct they can help to assess, areas where they are inadequate, and what new challenges they introduce. The paper concludes with directions for future research.
Article outline
- Opportunities associated with emerging technologies and oral language assessment
- Video-mediated OC assessments
- Virtual environment OC assessments
- Rating in Video-mediated and virtual environments
- Spoken dialogue systems for assessing oral communication
- Rule-based SDSs
- Large language model-driven SDSs
- Conclusions and future research
- Author queries
References
References (43)
Aseeri, F. M. (2025). Judging a book by its cover: Investigating the influence of test takers’ physical attractiveness in synchronous oral communication assessment. Unpublished doctoral dissertation, Iowa State University.
Brooks, L. (2009). Interacting in pairs in a test of oral proficiency: Co-constructing a better performance. Language Testing, 26, 342–366.
Buunk, A. P., & Dijkstra, P. (2011). Does attractiveness sell? Women’s attitude toward a product as a function of model attractiveness, gender priming, and social comparison orientation. Psychology & Marketing, 28(9), 958–973.
Chukharev-Hudilainen, E., & Ockey, G. J. (2021). The development and evaluation of interactional competence elicitor for oral language assessments. ETS Research Report Series, 2021(1), 1–20.
Dai, D. W. (2024). Assessing interactional competence: Principles, test development and validation through an L2 Chinese IC test (p. 446). Peter Lang International Academic Publishers.
Dombi, J., Sydorenko, T., & Timpe-Laughlin, V. (2024). Openings and closings in human-human versus human-spoken dialogue system conversations. Language Learning & Technology, 28(2), 32–61. [URL].
Ducasse, A. M., & Brown, A. (2009). Assessing paired orals: Raters’ orientation to interaction. Language Testing, 26, 423–443.
Galaczi, E., & Taylor, L. (2018). Interactional competence: Conceptualisations, operationalisations, and outstanding questions. Language Assessment Quarterly, 15(3), 219–236.
Gukturk, N., & Chukharev, E. (2024). Exploring the potential of a spoken dialog system-delivered paired discussion task for assessing interactional competence. Language Assessment Quarterly, 21(1), 60–99.
Hosoda, M., Stone-Romero, E. F., & Coats, G. (2003). The effects of physical attractiveness on job-related outcomes: A meta-analysis of experimental studies. Personnel Psychology, 56(2), 431–462.
Jamieson, J., Eignor, D. R., Grabe, W. and Kunnan, A. J. (2008). “Frameworks for a new TOEFL”. In C. A. Chapelle, J. Jamieson, and M. K. Enright (Eds.), Building a validity argument for the test of English as a foreign language (pp. 55–95). Routledge.
Jenkins, J. (2016). International tests of English: Are they fit for purpose? In H. Liao (Ed.), Critical reflections on foreign language education: Globalization and local interventions. The Language Training and Testing Center.
Karatay, Y. (2022). Development and validation of spoken dialog system-based oral communication tasks in an ESP context (publication No. 29165842). Doctoral dissertation, Iowa State University ProQuest Dissertations Publishing.
Karatay, Y., & Xu, J. (2025). Exploring the potential of conversational AI for assessing second language oral proficiency. TESOL Quarterly (first view).
Kim, G. (2025). Developing a GPT-powered dynamic pragmatics assessment using the expanded evidence centered design. Language Assessment Research Conference. Fullerton, California, USA.
Kim, J., & Craig, D. A. (2012). Validation of a videoconferenced speaking test. Computer Assisted Language Learning, 25(3), 257–275.
Kurt, S. (2025). Development and evaluation of a virtual reality environment for the Oral English Certification Test. Unpublished doctoral dissertation, Iowa State University.
Lee, H. W., Mullooly, A., Devine, A., & Galaczi, E. (2024). Exploring interaction in video-call paired speaking tests: A look at scores, language, and perceptions. Applied Linguistics, 45(3), 533–552.
Levis, J. (2020). Revisiting the intelligibility and nativeness principles. Journal of Second Language Pronunciation, 6(3), 310–328.
McNamara, T. (2018). A challenge for language testing: The assessment of English as a Lingua Franca. Language Assessment Research Conference, Ames, Iowa, March 21–23.
Muhanna, M. A. (2015). Virtual reality and the CAVE: Taxonomy, interaction challenges and research directions. Journal of King Saud University-Computer and Information Sciences, 27(3), 344–361.
Na, I. (2025, April). GenAI in assessing interactional competence: Examining differences between human-human and human-ai interactions. Language Assessment Research Conference. Fullerton, California, USA.
Nakatsuhara, F., Inoue, C., Berry, V., & Galaczi, E. (2017). Exploring the use of video-conferencing technology in the assessment of spoken language: A mixed-methods study. Language Assessment Quarterly, 14(1), 1–18.
Neiriz, R. (2023). Eliciting interactive oral communication samples through a spoken dialogue system to measure interactional competence. Unpublished doctoral dissertation, Iowa State University.
Ockey, G. J. (2009). The effects of a test taker’s group members’ personalities on the test taker’s second language group oral discussion test scores. Language Testing, 26 (2), 161–186.
(2014). The potential of the L2 group oral to elicit discourse with a mutual contingency pattern and afford equal speaking rights in an ESP context. English for Specific Purposes, 35, 17–29.
Ockey, G. J., & Chukharev-Hudilainen, E. (2021). Human versus computer partner in the paired oral discussion test. Applied Linguistics, 42(5), 924–944.
Ockey, G. J., Chukharev-Hudilainen, E., & Hirch, R. (2023). Assessing interactional competence: Interactional Competence Elicitor (ICE) vs a human partner. Language Assessment Quarterly, 20(4–5), 377–398.
Ockey, G. J., Gu, L., & Keehner, M. (2017). Web-based virtual environments for facilitating assessment of L2 Oral communication ability. Language Assessment Quarterly, 14(4), 346–359.
Ockey, G. J., & Hirch, R. R. (2020). A step toward the assessment of English as a lingua franca. In G. J. Ockey & B. A. Green (Eds.), Another generation of fundamental considerations in language testing (pp. 9–28). Springer.
Ockey, G. J., Koyama, D., Setoguchi, E., & Sun, A. (2015). The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students. Language Testing, 32(1), 39–62.
Ockey, G. J., & Li, Z. (2015). New and not so new methods for assessing oral communication. Language Value, 7(1), 1–21.
Ockey, G. J., & Neiriz, R. (2021). Evaluating technology-mediated second language oral communication assessment delivery models. Assessment in Education: Principles, Policy & Practice, 28(4), 350–368.
Ockey, G. J., Timpe-Laughlin, V., Davis, L., & Gu, L. (2019). Exploring the potential of a video-mediated interactive speaking assessment (Report No. ETS RR-19-05). Princeton, NJ: Educational Testing Service.
Ockey, G. J. & Qi, Q. (in press). Why multiple speech varieties when assessing L2 Chinese and English listening. In Wagner, E., & In’nami, Y. (Eds.), The Theory and Practice of Second Language Listening Assessment.
Poehner, M., & Lantolf, J. (2023). Advancing L2 Dynamic Assessment: Innovations in Chinese contexts. Language Assessment Quarterly, 20(1), 1–19.
Ramanarayanan, V., Evanini, K., & Tsuprun, E. (2019). Beyond monologues. In Zechner, K., & Evanini, K. (Eds.). Automated speaking assessment: Using language technologies to score spontaneous speech (pp. 176–191). Routledge.
Ramanarayanan, V., Suendermann-Oeft, D., Lange, P., Mundkowsky, R., Ivanov, A. V., Yu, Z., Qian, Y., & Evanini, K. (2017). Assembling the jigsaw: How multiple open standards are synergistically combined in the HALEF multimodal dialog system. In Dahl, D. A. Multimodal interaction with W3C standards: Toward natural user interfaces to everything (pp. 295–310). Springer International Publishing.
Song, J., & Hsu, W. L. (2021). Design and implementation of a classroom-based virtual reality assessment. In Salaberry, M. & Burch, A. (Eds.), Assessing speaking in context: Expanding the construct and its applications (pp. 265–282). Channel View Publications.
Vo, S. (2019). Effects of task types on interactional competence in oral communication assessment (Publication No. 17345). Doctoral dissertation, Iowa State University. ProQuest Dissertation and Theses Global.