Oral performance tests designed to assess candidates’ ability to function in a particular professional role typically involve role plays which simulate aspects of professional interaction between a candidate, in his/her professional role, and an interlocutor, in the role of client or consumer of the professional services. The validity of such tests is dependent both on the incorporation of appropriate content and on the extent to which the interaction between candidate and interlocutor simulates authentic professional interaction. Materials for such role plays are commonly produced according to specifications developed by language testers following consultation with professional informants, but there has been little investigation of how well tasks reflect their purpose (i.e. the aims/intentions of the professional experts). Test interlocutors, generally language teachers, normally undergo some form of training in the test administration procedure, but there has been little analysis of how well their behaviour reflects client behaviour.
This paper reports on an investigation into the role which expert professional informants can play in providing post hoc content validation of tasks used in this kind of occupation-specific test, focusing on both the content of the tasks and the behaviour of the interlocutors. The study draws on data from the oral interaction component of an occupation-specific test, the Occupational English Test (McNamara 1996), a 4-skills test for overseas-trained health professionals (with attention in this study to nurses). The study involved a two-day workshop where representatives of the nursing profession, in conjunction with the researchers, analysed both written test materials and audiotaped test data, contributing to an analysis of the realism of the tasks and the features of interlocutor behaviour.
(1990) Fundamental Considerations in Language Testing. Oxford, Oxford University Press.
Brown, A.
(1994) LSP testing: the role of linguistic and real-world criteria. In Khoo, R. (ed.) LSP: Problems and Prospects. Singapore, SEAMEO Regional Language Centre.
Brown, A.
(1995) The effect of rater variables in the development of an occupation-specific language performance test. Language Testing, 12, 1: 1–15.
Cafarella, C.
(1994) Assessor accommodation in the 1992 Victorian Certificate of Education Italian oral test. Unpublished MA thesis, University of Melbourne.
Cumming, A. and R. Berwick
(1996) Validation in language testing. Clevedon, Avon, Multilingual Matters
Davies, A.D.
(1986) Indirect ESP testing: old innovations. In Portal, M. (ed.) Innovations in Language Testing. Windsor, Berks., NFER-Nelson.
Davies, A.
(1995) Testing communicative language or testing language communicatively: what? how?Melbourne Papers In Language Testing 4,1:1–20.
Douglas, D.
forthcoming) Field-specific language testing. Unpublished manuscript.
Douglas, D. and L. Selinker
(1993) Performance on a general versus a field-specific test of speaking proficiency by international teaching assistants. In Chapelle, C. and D. Douglas (eds) A new decade of language testing research. Alexandria, VA, TESOL Publications.
Elder, C.
(1993) How do subject specialist construe classroom language proficiency?Language Testing 10, 3: 235–254.
Elder, C.
(1994) Are raters’ judgements of teacher proficiency wholly language based?Melbourne Papers In Language Testing 2,2:41–59.
Filipi, A.
(1994) Interaction or interrogation? A study of talk occurring in a sample of the 1992 Victorian Certificate of Education Italian oral common assessment task. Unpublished MA thesis, University of Melbourne.
Lazaraton, A. and N. Saville
(1994) Processes and outcomes in oral assessment. Paper presented at the Language Testing Research Colloquium, Washington, April.
Levinson, S.
(1983) Pragmatics. Cambridge, Cambridge University Press.
McNamara, T.F.
(1990) Assessing the Second Language Proficiency of Health Professionals. Unpublished PhD Thesis, University of Melbourne.
McNamara, T.F.
(1996) Measuring Second Language Performance. London, Longman.
McNamara, T.F. and T. Lumley
(1993) The effect of interlocutor and assessment mode variables in offshore assessments of speaking skills in occupational settings. Paper presented at the Fifteenth Language Testing Research Colloquium, University of Cambridge, August.
Morton, J., G. Wigglesworth and D. Williams
forthcoming) Approaches to the evaluation of interviewer behaviour in oral tests. In Wigglesworth, G. and G. Brindley eds access:: issues in English language test design and delivery Sydney NCELTR
O’Loughlin, K.
(1995) Test taker performance in direct and semi-direct versions of an oral proficiency test. Unpublished manuscript.
Ross, S.
(1992) Accommodative questions in oral proficiency interviews. Language Testing 9,2: 173–186.
Ross and Berwick
(1992) The discourse of accommodation in oral proficiency interviews. Studies in Second Language Acquisition 141:159–176.
Wesche, M.B.
(1987) Second language performance testing: The Ontario Test of ESL as an example. Language Testing 4,1: 28–47.
Wesche, M.B.
(1992) Performance testing for work-related second language assessment. In Shohamy, E. and R. Walton (eds) Language assessment for feedback: testing and other strategies. Washington, D.C., National Foreign Language Center Publications.
Wolf, L.F. and J.K. Smith
(1993) The consequence of consequence: Motivation, anxiety and test performance. Applied Measurement in Education 8,3:227–242.
Young, R. and M. Milanovic
(1992) Discourse variation in oral proficiency interviews. Studies in Second Language Acquisition 141: 403–424.
Cited by
Cited by 9 other publications
Assassi, Tarek & Tarek Ghodbane
2023. The Recurring Issue of Aviation English Test Validity: Echoes from Test-takers and Assessors of the English for Aviation Language Testing System in Algeria. English Studies at NBU 9:2 ► pp. 239 ff.
Brooks, Lindsay
2009. Interacting in pairs in a test of oral proficiency: Co-constructing a better performance. Language Testing 26:3 ► pp. 341 ff.
Elder, Catherine & Alan Davies
2006. ASSESSING ENGLISH AS A LINGUA FRANCA. Annual Review of Applied Linguistics 26
Hamp-Lyons, Liz & Jane Lockwood
2009. THE WORKPLACE, THE SOCIETY AND THE WIDER WORLD: THE OFFSHORING AND OUTSOURCING INDUSTRY. Annual Review of Applied Linguistics 29 ► pp. 145 ff.
Harding, Luke, John Pill & Kerry Ryan
2011. Assessor Decision Making While Marking a Note-Taking Listening Test: The Case of the OET. Language Assessment Quarterly 8:2 ► pp. 108 ff.
Kim, Hyejeong & Catherine Elder
2015. Interrogating the construct of aviation English: Feedback from test takers in Korea. Language Testing 32:2 ► pp. 129 ff.
Macqueen, Susy, John Pill & Ute Knoch
2016. Language test as boundary object: Perspectives from test users in the healthcare domain. Language Testing 33:2 ► pp. 271 ff.
McNamara, T.F. & Tom Lumley
1997. The effect of interlocutor and assessment mode variables in overseas assessments of speaking skills in occupational I settings. Language Testing 14:2 ► pp. 140 ff.
Woodward-Kron, Robyn & Catherine Elder
2016. A comparative discourse study of simulated clinical roleplays in two assessment contexts: Validating a specific-purpose language test. Language Testing 33:2 ► pp. 251 ff.
This list is based on CrossRef data as of 25 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.