Article published in:Developing, Modelling and Assessing Second Languages
Edited by Jörg-U. Keßler, Anke Lenzing and Mathias Liebner
[Processability Approaches to Language Acquisition Research & Teaching 5] 2016
► pp. 207–238
The cognitive processes elicited by L2 listening test tasks – A validation study
This paper is concerned with an investigation into the validity of a listening comprehension test that was developed for a large-scale assessment project. The study draws on qualitative data, employing a think-aloud technique and stimulated recall interviews. The informants (n=18) were purposefully and randomly sampled from a group (n=121) of year 9 learners (ages 14–16) of English as a foreign language (EFL) in German schools. Subjects were asked to think aloud while they were solving the multiple choice-items of the listening test. Construct-relevant and -irrelevant processes were identified and analysed with regard to their distribution across the two subsamples and their relative contribution to correct item responses. The results provide validity evidence for the listening tests in general. A few few test items, however, were shown to elicit test-taking processes and strategies that compromise the measurement outcomes.1. Introduction
Available under the Creative Commons Attribution-NonCommercial-NoDerivatives (CC BY-NC-ND) 4.0 license.
For any use beyond this license, please contact the publisher at firstname.lastname@example.org.
Published online: 29 June 2016
Borsboom, D., Cramer, A.O.J., Kievit, R.A., Zand Scholten, A., & Franic, S.
Borsboom, D., VanHeerden, J., & Mellenbergh, G.
Campbell, D.T., & Fiske, D.W.
Creswell, J.W., Plano Clark, V.L., Gutmann, M.L., & Hanson, W.E.
Cronbach, L.J., & Meehl, P.E.
Denzin, N.K., & Lincoln, Y.S.
Di Pardo, A.
Ericsson, K.A., & Simon, H.A.
Friese, M., & Fiedler, K.
Gass, S.M., & Mackey, A.
Gernsbacher, M.A., & Foertsch, J.A.
Graesser, A.C., Gernsbacher, M.A., & Goldman, S.R.
Graesser, A.C., Singer, M., & Trabasso, T.
Graesser, A.C., Wiemer-Hastings, P., & Wiemer-Hastings, K.
Grotjahn, R., & Eckes, T.
Haladyna, T.M., & Downing, S.M.
Kintsch, W., Patel, V.L., & Ericsson, K.A.
Kintsch, W., & van Dijk, T.A.
Lewins, A., & Silver, C.
McKoon, G., & Ratcliff, R.
Miles, M.B., & Huberman, A.M.
Nold, G., & Rosssa, H.
Nold, G., Rossa, H., & Hartig, J.
(2008) Proficiency scaling in DESI listening and reading EFL tests: Task characteristics, item difficulty and cut-off points. In L. Taylor & C.J. Weir (Eds.), Multilingualism and assessment. Achieving transparency, assuring quality, sustaining diversity. Proceedings of the ALTE Berlin conference, May 2005 (pp. 94–116). Cambridge: Cambridge University Press.
O'Malley, M., & Chamot, A.U.
Pienemann, M., & Keßler, J.-U.
Rogers, T.T., & McClelland, J.L.
Roser, M., & Gazzaniga, M.S.
Rupp, A., Ferne, T., & Choi, H.
(2011) Processing the L2 comprehension process: Testing Processability Theory’s predictions in an ERP study of adult learners of L2 Swedish. Master’s thesis, Lund University. Accessed from: http://lup.lub.lu.se/luur/download?func=downloadFile&recordOId=2439371&fileOId=2439378
Selting, M., Auer, P., Barden, B., Bergmann, J.R., Couper-Kuhlen, E., & Günthner, S.. et al.
Van der Veen, A., Huff, K., Gierl, M., McNamara, D.D., Louwerse, M., & Graesser, A.
(2007) Developing and validating instructionally relevant reading competency profiles measured by the critical reading section of the SAT reasoning test. In D.S. McNamara (Ed.), Reading comprehension strategies. Theories, interventions, and technologies (pp. 137-172J). New York, NY: Lawrence Erlbaum Associates.
Van Someren, M.W., Barnard, Y.F., & Sandberg, J.A.
Webb, E., Campbell, D.T., Schwartz, R.D., & Sechrest, L.