This paper discusses the development of an assessment to satisfy the International Civil Aviation Organization (ICAO) Language Proficiency Requirements. The Versant Aviation English Test utilizes speech recognition technology and a computerized testing platform, such that test administration and scoring are fully automated. Developed in collaboration with the U.S. Federal Aviation Administration, this 25-minute test is delivered via a telephone or computer. Two issues of interest are discussed. The first concerns the practicalities of assessing candidates in each of six separate dimensions of spoken proficiency: Pronunciation, Structure, Vocabulary, Fluency, Comprehension, and Interactions. Although an automated scoring system can objectively segregate these skills, we question whether human raters have the capacity to do this in oral interviews. The second issue discussed is how an automated test can provide a valid assessment of spoken interactions. Tasks were designed to simulate the information exchange between pilots and controllers on which candidates’ proficiency in ‘Interactions’ could be measured, for example, by eliciting functions such as correcting miscommunications and providing clarification. It is argued that candidate ability can be probed and estimated in a fair and standardized way by presenting a series of independent items which are targeted in difficulty at the various ICAO levels.
(2008) Final report on a survey of aviation English tests. Unpublished report.
Bachman, Lyle F; Palmer, Adrian S.
(1983) The construct validity of the FSI oral interview. Language Learning, 31(1), 67–86.
Bachman, Lyle F.; Palmer, Adrian S.
(1996) Language Testing in Practice. Oxford. Oxford University Press.
(2003) Interviewer variation and the co–construction of speaking proficiency. Language Testing, 20(1), 1–25.
Brown, Gillian; Yule, George
(1983) Discourse Analysis. Cambridge: Cambridge University Press.
Brown, Gillian; Anderson, Anne; Shillcock, Richard; Yule, George
(1984) Teaching Talk: Strategies for Production and Assessment.,Cambridge: Cambridge University Press.
(1996) Testing tasks: Issues in task design and the group oral. Language Testing, 13(1), 23–51.
(1987) A Guide to Language Testing. Boston, MA: Heinle and Heinle.
Hinofotis, Frances B.
(1983) The structure of oral communication is an educational environment: a comparison of factor–analytic rotational procedures. In Oller, J.W., Jr (Ed.), Issues in Language Testing Research (pp. 170–187). Rowley, MA: Newbury House.
International Civil Aviation Organization (ICAO)
(2004) Manual on the Implementing of ICAO Language Proficiency Requirements. Montreal: ICAO.
(2001) The Art of Non–conversation: A Reexamination of the Validity of the Oral Proficiency Interview. New Haven, CT: Yale University Press.
Lantolf, James; Frawley, William
(1985) Oral proficiency testing: A critical analysis. Modern Language Journal, 69(4), 337–345.
(1988) The quantitative study of communicative success: Politeness and accidents in aviation discourse. Language in Society, 17(3), 375–399
(1996) Measuring Second Language Performance. London: Longman.
Morrow, Daniel G; Rodvold, Michelle; Lee, Alfred
(1994) Nonroutine transactions in controller–pilot communication. Discourse Processes, 17(2), 235–258.
(2002) The FCE speaking test: Using rater reports to help interpret test scores. System, 30(2),143–54.
2008Versant Aviation English Test: Test description and validation summary. Palo Alto, CA: Author.
(2009) Comparing direct and semi-direct modes for speaking assessment: Affective effects on test-takers. Language Assessment Quarterly, 6(2), 113–125.
(1992) Accommodative questions in oral proficiency interviews. Language Testing, 9(2), 173–186.
(1977) The preference for self-correction in the organization of repair in conversation. Language, 53(2), 361–382.
(1976) Interview talk: Bringing off a research instrument. In Silverman, D. and Jones, J. (Eds.), Organisational Work: The language of grading, the grading of language (pp. 133–150). London: Collier Macmillan.
(2001) Examining dialogue: Another approach to content specification and to validating inferences drawn from test scores. Language Testing, 18(3), 275–302.
(1984) Conversational Style: Analyzing Talk among Friends. Norwood, NJ: Ablex.
van Lier, L.
(1989) Reeling, writhing, drawling, stretching, and fainting in coils: Oral proficiency interviews as conversation. TESOL Quarterly, 23(3), 489–508.
Weir, Cyril J.
(2005) Language Testing and Validation: An Evidence-based Approach. New York: Palgrave Macmillan.
Cited by 6 other publications
Alderson, J. Charles
2011. The Politics of Aviation English Testing. Language Assessment Quarterly 8:4 ► pp. 386 ff.
2013. Effects of Targeted Test Preparation on Scores of Two Tests of Oral English as a Second Language. TESOL Quarterly 47:1 ► pp. 148 ff.
Moder, Carol Lynn
2012. Aviation English. In The Handbook of English for Specific Purposes, ► pp. 227 ff.
Oliver, Rhonda, Honglin Chen & Stephen Moore
2016. Review of selected research in applied linguistics published in Australia (2008–2014). Language Teaching 49:4 ► pp. 513 ff.
2018. Innovative assessment of aviation English in a virtual world: Windows into cognitive and metacognitive strategies. ReCALL 30:2 ► pp. 196 ff.
2020. Investigating Target Tasks, Task Phases, and Indigenous Criteria for Military Aviation English Assessment. Language Assessment Quarterly 17:4 ► pp. 337 ff.
This list is based on CrossRef data as of 27 september 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.