Cover not available
Part of
Developing, Modelling and Assessing Second Languages
Edited by Jörg-U. Keßler, Anke Lenzing and Mathias Liebner
[Processability Approaches to Language Acquisition Research & Teaching 5] 2016
► pp. 135–162

Psychometric approaches to language testing and linguistic profiling – A complementary relationship?

| University of Paderborn

The present study investigates as to whether and to what extent Linguistic Profiling can complement shortcomings of proficiency rating scales that are based on the Common European Framework of Reference (CEFR) (CoE 2001). In order to shed light on possible interfaces between the second language acquisition theory Processability Theory (PT) (Pienemann 1998, 2005) and the CEFR, learners were rated according to the CEFR and diagnosed with two linguistic profiling tools: Rapid Profile (Mackey, Pienemann, & Thornton 1991; Pienemann & Mackey 1993; Keßler 2006, 2008) and Autoprofile (Lin 2012). The emergence criterion (Pienemann 1998; Pallotti 2007) as used in PT as the starting point to determine acquisition is highly predictive in nature and thus taken as the point of departure of an integration of PT into the CEFR. The results show correspondences between CEFR levels and PT stages and suggest a reexamination of early CEFR levels in terms of the complexity of operations beginning learners are assumed to manage.

Available under the Creative Commons Attribution-NonCommercial-NoDerivatives (CC BY-NC-ND) 4.0 license.

For any use beyond this license, please contact the publisher at [email protected].

Published online: 29 June 2016
DOI logo
https://doi.org/10.1075/palart.5.07hag
References
Bachman, L
(1990) Fundamental considerations in language testing. Oxford: Oxford University Press.Google Scholar
(2004) Statistical analyses for language assessment (pp. 153-205, 255-279). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Bachman L., & Cohen, A.D
(1998) Interfaces between second language acquisition and language testing research. Cambridge: Cambridge University Press.Google Scholar
Bresnan, J
(2001) Lexical functional syntax. Malden, MA: Blackwell.Google Scholar
Brindley, L.F
(1998) Describing language development. In L.F. Bachman & A.D. Cohen (Eds.), Interfaces between second language acquisition and language testing research (pp. 112-140). Cambridge: Cambridge University Press.Google Scholar
Clahsen, H
(1985) Profiling second language acquisition. In K. Hyltenstam & M. Pienemann (Eds.), Modelling and assessing second language development. Clevedon: Multilingual Matters.Google Scholar
Council of Europe
(1992) Transparency and coherence in language learning in Europe: Objectives, assessment and certification. Symposium held in Rüschlikon, 10–16 November 1991. Strasbourg: Council for Cultural Cooperation.Google Scholar
(2001) Common European Framework of Reference for languages: Learning, teaching, assessment. Cambridge: Cambridge University Press.Google Scholar
(2006) The Common European Framework of Reference in its political and educational dimensions. Cambridge: Cambridge University Press.Google Scholar
(2012) Illustrations of the European levels of language proficiency. Accessed on 15 December 2012 from: [URL]Google Scholar
Crystal, D., Fletcher, P., & Garman, M
(1976) The grammatical analysis of language disability. London: Arnold.Google Scholar
Crystal, D
(1982) Profiling linguistic disability. San Diego, CA: Singular.Google Scholar
DiBiase, B., & Kawaguchi, S
(2002)  Exploring the typological plausibility of Processability Theory: language development in Italian second language and Japanese second language . Second Language Research, 18(3), 274 - 302. DOI logoGoogle Scholar
DiBiase, B
(2008) Focus-on-form and development in L2 learning. In J.-U. Keßler, (Ed.), Processability approaches to second language acquisition (pp. 197-220). Newcastle upon Tyne: Cambridge Scholars.Google Scholar
Ellis, N
(2005) At the interface. Dynamic interactions of explicit and implicit language knowledge. Studies in Second Language Acquisition, 27(2), 305-352. DOI logoGoogle Scholar
(2007) The weak interface, consciousness, and form-focused instruction: Mind the doors. In S. Fotos & H. Nassaji (Eds.), Form-focused instruction and teacher education. Studies in honor of Rod Ellis (pp. 197-215). New York, NY: Routledge.Google Scholar
Håkansson, G., Salameh, E.K., & Nettelbladt, U
(2003) Measuring language development in bilingual children: Swedish- Arabic children with and without language impairment. Linguistics, 41, 255-288. DOI logoGoogle Scholar
Håkansson, G., & Norrby, C
(2006) Processability Theory applied to spoken and written L2 Swedish. In F. Mansouri (Ed.). Second language acquisition research: Theory construction and testing (pp. 81-94). Newcastle upon Tyne: Cambridge Scholars.Google Scholar
Harsch, C
(2005) Der gemeinsame Europäische Referenzrahmen für Sprachen: Leistungen und Grenzen. Bedeutung des Referenzrahmen im Kontext der Beurteilung von Sprachvermögen am Beispiel des semikreativen Schreibens im DESI Projekt. Unpublished PhD dissertation. University of Augsburg.Google Scholar
Hymes, D
(1974) Foundations in sociolinguistics. London: Tavistock.Google Scholar
Kaplan, R.M., & Saccuzzo, D.P
(2010) Psychological testing: Principles, applications, and issues (8th ed.). Belmont, CA: Wadsworth, Cengage Learning.Google Scholar
Kawaguchi, S
(2005) Argument structure and syntactic development in Japanese as a second language. In M. Pienemann, Cross-linguistic aspects of Processability Theory. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Kawaguchi, S., DiBiase, B., & Pienemann, M
(2005) Extending Processability theory. In M. Pienemann (Ed.), Cross-linguistic aspects of Processability Theory. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Keßler, J.-U
(2006) Englischerwerb im Anfangsunterricht diagnostizieren: Linguistische Profilanalysen am Übergang von der Primarstufe in die Sekundarstufe I. Tübingen: Narr.Google Scholar
(2008) Communicative tasks and second language profiling: Linguistic and pedagogical implications. In J. Eckerth & S. Siekmann (Eds.), Task-based language learning and teaching. Theoretical, methodological, and pedagogical perspectives (pp. 291-310). Frankfurt: Peter Lang.Google Scholar
Keßler, J.-U., & Plesser, A
(2011) Teaching English grammar (Standard Wissen Lehramt Englisch). Paderborn: Schöningh/UTB.Google Scholar
Lado, R
(1961) Language testing: The construction and use of foreign language tests. London: Longman.Google Scholar
Lenzing A., & Plesser, A
(2010) Challenging the scope-precision dilemma in language testing: The common European framework and linguistic profiling. Paper presented at the 10th International Symposium of Processability Approaches to Language Acquisition (PALA). University of Western Sydney, Australia, 19-21 September.
Lenzing, A
(2010) Rapid profile. A screening procedure for second language acquisition assessment. Unpublished User Manual.Google Scholar
Levelt, W.J.M
(1989) Speaking: From intention to articulation. Cambridge, MA: The MIT Press.Google Scholar
Lin, B.J
(2012) Is automatic linguistic profiling feasible in an ESL context? Unpublished PhD dissertation. University of Newcastle.Google Scholar
Little, D
(2008) The Common European Framework of Reference for languages and the development of policies for the integration of adult migrants. Council of Europe: Strasbourg. Accessed on 20 February 2013 from: [URL]Google Scholar
Mackey, A., Pienemann, M., & Thornton, I
(1991) Rapid profile: A second language screening procedure. Language and Language Education, 1(1), 61-82.Google Scholar
Meisel, J.M., Clahsen, H., & Pienemann, M
(1981) On determining developmental stages in second language acquisition. Studies in Second Language Acquisition, 3, 109-135. DOI logoGoogle Scholar
Michell, J
(1999) Measurement in psychology. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
North, B
(2007)  The Common European Framework of Reference: Development, theoretical and practical issues . Accessed on 14 August 2014 from: [URL]
Neuendorf, K.A
(2002) The content analysis guidebook. Thousand Oaks, CA: Sage.Google Scholar
Pallotti, G
(2007) An operational definition of the emergence criterion. Applied Linguistics, 28(3), 361-382. DOI logoGoogle Scholar
Pienemann, M., Johnston, M., & Brindley, G
(1988)  Constructing an acquisition-based procedure for second language assessment . Studies in second language acquisition, 10, 217-243. DOI logoGoogle Scholar
Pienemann, M., & Mackey, A
(1993) An empirical study of children’s ESL development and Rapid Profile. In P. McKay (Ed.), ESL development: Language and literacy in schools, Vol. 2: Documents on bandscale development and language acquisition (pp. 115–259). Canberra: National Languages & Literacy Institute of Australia and Commonwealth of Australia.Google Scholar
Pienemann, M
(1998) Language processing and second language development. Processability Theory. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Pienemann, M., DiBiase, B., & Kawaguchi, S
(2005) Extending Processability Theory. In M. Pienemann (Ed.), Cross-linguistic aspects of Processability Theory (pp. 199-251). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Pienemann, M
(Ed.) (2005) Cross-linguistic aspects of Processability Theory. Amsterdam: John Benjamins. DOI logoGoogle Scholar
(2007) Processability Theory. In B. VanPatten & J. Williams (Eds.), Theories in second language acquisition: An introduction. Malden, MA: Blackwell.Google Scholar
Pienemann, M., & Keßler, J.-U
(2007) Measuring bilingualism. In P. Auer & L. Wei (Eds.), Handbook of applied linguistics. Vol. 5: Multilingualism (pp. 247-275). Berlin: Mouton de Gruyter,Google Scholar
Pienemann, M., Lin, B-J., & Chung, L.-Y
(2009) The feasibility of auto-profiling online. In E. Damiani, et al. (Eds.), New dimensions in intelligent interactive multimedia systems (pp. 189-198). Berlin: Springer.Google Scholar
Pienemann, M., & Keßler, J.-U
(Eds.) (2011) Studying Processabilty Theory: Introductory textbook. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Plesser, A
(2008) A study on writing and speaking. Unpublished dissertation. University of Paderborn.Google Scholar
(2011) The interlanguage approach: Example tasks. Unpublished User Manual.Google Scholar
Rasinger, S.M
(2008) Quantitative research in linguistics. An introduction. London: Continuum.Google Scholar
Rohrmann, B
(2007) Verbal qualifiers for rating scales: Sociolinguistic considerations and psychometric data. University of Melbourne, Australia. Accessed on 14 August 2014 from: [URL]Google Scholar
Schaefer, E
(2008) Rater bias patterns in an EFL writing assessment. Language Testing, 25, 465-493. DOI logoGoogle Scholar
Shohamy, E
(2000) The relationship between language testing and second language acquisition, revisited. System, 28, 541-553. DOI logoGoogle Scholar
Stevens, S.S
(1946) On the theory of scales of measurement. Science, 103, 677–680. DOI logoGoogle Scholar
Wigglesworth, G
(1993) Exploring bias analysis as a tool for improving rater consistency in assessing oral interaction. Language Testing 10, 305-319. DOI logoGoogle Scholar
Wilkins, D.A
(1978) Proposal for levels definition. In J.L.M. Trim (Ed.), Some possible lines of development of an overall structure for a European unit/credit scheme for foreign language learning by adults (pp. 71-78). Strasbourg: Council of Europe.Google Scholar
Winke, P., Gass, S., & Myford, C
(2013) Raters' L2 background as a potential source of bias in rating oral performance. Language Testing, 30, 231-252. DOI logoGoogle Scholar