Cover not available
Part of
Instructed Second Language Acquisition Research Methods
Edited by Laura Gurzynski-Weiss and YouJin Kim
[Research Methods in Applied Linguistics 3] 2022
► pp. 329–352

Chapter 14
Speaking

Complexity, accuracy, fluency, and functional adequacy (CAFFA)

| University of Amsterdam
| University of Amsterdam

This chapter focuses on the assessment of oral performance in a second language (L2), viewed from the perspective of task-based language assessment and instructed second language acquisition. The notion of L2 proficiency, as presented in the Common European Framework of References (Council of Europe, 2001), rests on two pillars that have to be taken into account when measuring L2 speaking: (1) the linguistic dimension, referring to the complexity, accuracy, and fluency (CAF) of the speaker’s utterances; and (2) the communicative dimension, concerning the appropriateness and effectiveness of the message to be conveyed, labeled functional adequacy (FA). Whereas the majority of studies on the assessment of learner performance have focused on CAF, little attention has been devoted to the functional aspects of L2 performance. The rationale underlying the present chapter is that assessing L2 speaking is impossible without considering both CAF and FA (henceforth, CAFFA) and the mutual relationship between the two constructs. It is argued that L2 speaking should not only be assessed by measures along the CAF-triad but also in terms of FA. The chapter discusses the theoretical underpinnings and applicability of a rating scale for FA (Kuiken & Vedder, 2017, 2018).

Keywords: L2 speaking, CAF (complexity, accuracy, fluency), FA (functional adequacy), CAFFA, rating scale
Article outline
  • 1.What is CAFFA and why is it important?
    • 1.1Levelt’s model of speech production
  • 2.What we know and what we need to know about CAFFA in ISLA
    • 2.1CAFFA: Definition of constructs and operationalizations
    • 2.2CAFFA: Studies on the assessment of oral performance
  • 3.Data elicitation and interpretation
    • 3.1Guidelines for conducting CAFFA research
    • 3.2The WISP project
    • 3.3Development of a rating scale for FA
  • 4.Advice to future CAFFA researchers
    • 4.1Type of research
    • 4.2Target and source languages
    • 4.3Measures
    • 4.4Task type
    • 4.5Rater training
  • 5.Troubleshooting CAFFA research in ISLA
    • 5.1L1 and L2 speakers in one and the same study
    • 5.2Rater variables
    • 5.3Scale adaptations
  • 6.Conclusions
  • 7.Further reading and additional resources
    • 7.1Books and journal articles
    • 7.2Journals, professional organizations, and websites
  • References
Published online: 8 December 2022
DOI logo
https://doi.org/10.1075/rmal.3.14kui
References
Brezina, V., & Pallotti, G.
(2019) Morphological complexity in written L2 texts. Second Language Research, 35(1), 99–120. DOI logoGoogle Scholar
Bulté, B., & Housen, A.
(2012) Defining and operationalising L2 complexity. In A. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp. 21–46). John Benjamins. DOI logoGoogle Scholar
Council of Europe
(2001) Common European framework of reference for languages: Learning, teaching, assessment. Cambridge University Press.Google Scholar
De Bot, K.
(1992) A bilingual production model: Levelt’s ‘speaking’ model adapted. Applied Linguistics 13 (1), 1–24. DOI logoGoogle Scholar
De Clercq, B., & Housen, A.
(2019) The development of morphological complexity: A cross-linguistic study of L2 French and English. Second Language Research, 35 (1), 71–98. DOI logoGoogle Scholar
De Jong, N. H., Steinel, M. P., Florijn, A. F., Schoonen, R., & Hulstijn, J. H.
(2012) The effect of task complexity on functional adequacy, fluency and lexical diversity in speaking performances of native and non-native speakers. In A. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency. Complexity, accuracy and fluency in SLA (pp. 121–142). John Benjamins. DOI logoGoogle Scholar
Duijm, K., Schoonen, R., & Hulstijn, J. H.
(2018) Professional and non-professional raters’ responsiveness to fluency and accuracy in L2 speech: An experimental approach. Language Testing, 35 (4), 501–527. DOI logoGoogle Scholar
Ekiert, M., Lampropoulou, S., Révész, A., & Torgersen, E.
(2018) The effects of task type and L2 proficiency on discourse appropriacy in oral task performance. In N. Taguchi & Y-J. Kim (Eds.), Task-based approaches to assessing pragmatics (pp. 247–264). John Benjamins. DOI logoGoogle Scholar
Ekiert, M., Révész, A., Torgersen, E., & Moss, E.
(2022) The role of pausing in L2 oral task performance: Toward a complete construct of functional adequacy. TASK 2 (1), 33–59. DOI logoGoogle Scholar
Foster, P., & Skehan, P.
(1996) The influence of planning and task type on second language performance. Studies in Second Language Acquisition, 18 (3), 299–323. DOI logoGoogle Scholar
Foster, P., & Wiggleworth, G.
(2016) Capturing accuracy in second language performance: The case for a weighted clause ratio. Annual Review of Applied Linguistics, 36 , 98–116. DOI logoGoogle Scholar
Gilabert, R., Barón Pares, J., & Levkina, M.
(2011) Manipulating task complexity across task types and modes. In P. Robinson (Ed.), Second language task complexity: Researching the Cognition Hypotheses of language learning and performance (pp. 105–138). John Benjamins. DOI logoGoogle Scholar
González-Lloret, M.
(2022) The present and future of functional adequacy. TASK 2 (1), 146–157. DOI logoGoogle Scholar
Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z.
(2004) Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers, 36 (2), 193–202. DOI logoGoogle Scholar
Grice, H. P.
(1975) Logic and conversation. In P. Cole, & J. L. Morgan (Eds.), Speech acts (pp. 41–58). Academic Press.Google Scholar
Herraiz Martínez, A.
(2018) Functional adequacy: The influence of English-medium instruction, English proficiency, and previous language learning experiences [Unpublished doctoral dissertation]. Universitat Jaume I, Castellón de la Plana.
Herraiz Martínez, A., & Alcón Soler, E.
(2019) Pragmatic outcomes in the English-medium instruction context. Applied Pragmatics, 1 (1), 68–91, DOI logoGoogle Scholar
Housen, A., Kuiken, F., & Vedder, I.
(Eds.) (2012) Dimensions of L2 performance and proficiency. Complexity, accuracy and fluency in SLA. John Benjamins. DOI logoGoogle Scholar
Hulstijn, J. H., Schoonen, R., De Jong, N. H., Steinel, M. P., & Florijn, A. F.
(2012) Linguistic competence of learners of Dutch as a second language at the B1 and B2 levels of speaking proficiency of the Common European Framework of Reference for Languages (CEFR). Language Testing, 29 (2), 203–221. DOI logoGoogle Scholar
Izumi, S.
(2003) Comprehension and production processes in second language learning: In search of the psycholinguistic rationale of the Output Hypothesis. Applied Linguistics 24 (2), 168–197. DOI logoGoogle Scholar
Knoch, U.
(2009) Diagnostic assessment of writing: A comparison of two rating scales. Language Testing, 26 (2), 275–304. DOI logoGoogle Scholar
Kormos, J.
(2006) Speech production and second language acquisition. Psychology Press.Google Scholar
Kuiken, F., & Vedder, I.
(2014) Rating written performance: What do raters do and why? Language Testing, 31 (3), 329–348. DOI logoGoogle Scholar
(2017) Functional adequacy in L2 writing: Towards a new rating scale. Language Testing, 34 (3), 321–336. DOI logoGoogle Scholar
(2018) Assessing functional adequacy of L2 performance in a task-based approach. In N. Taguchi, & Y.-J. Kim (Eds.), Task-based approaches to teaching and assessing pragmatics (pp. 265–285). John Benjamins. DOI logoGoogle Scholar
Kuiken, F., Vedder, I., & Gilabert, R.
(2010) Communicative adequacy and linguistic complexity in L2 writing. In I. Bartning, M. Martin, & I. Vedder (Eds.), Communicative proficiency and linguistic development: Intersections between SLA and language testing research (pp. 81–100). European Second Language Association.Google Scholar
Kyle, K.
(2016) Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices of syntactic sophistication [Doctoral dissertation, Georgia State University]. ScholarWorks. [URL]
Lahmann, C., Steinkrauss, R., & Schmid, M. S.
(2019) Measuring linguistic complexity in long-term L2 speakers of English and L1 attriters of German. International Journal of Applied Linguistics, 29 (2), 173–191. DOI logoGoogle Scholar
Lambert, C., & Nakamura, S.
(2019) Proficiency-related variation in syntactic complexity: A study of English L1 and L2 oral descriptive discourse. International Journal of Applied Linguistics, 29(2), 248–264. DOI logoGoogle Scholar
Larsen-Freeman, D.
(2009) Adjusting expectations: The study of complexity, accuracy, and fluency in second language acquisition. Applied Linguistics, 30 (4), 579–589. DOI logoGoogle Scholar
Levelt, W. J. M.
(1989) Speaking. From intention to articulation. The MIT Press.Google Scholar
Long, M.
(2015) Second language acquisition and task-based language teaching. Wiley Blackwell.Google Scholar
Lu, X.
(2010) Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15 (4), 474–496. DOI logoGoogle Scholar
McNamara, T., & Roever, C.
(2007) Testing: The social dimension. Blackwell. DOI logoGoogle Scholar
Michel, M.
(2017) Complexity, accuracy, and fluency in L2 production. In S. Loewen & M. Sato (Eds.), The Routledge handbook of instructed second language acquisition (pp. 50–68). Routledge. DOI logoGoogle Scholar
Norris, J. M., & Ortega, L.
(2009) Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30 (4), 555–578. DOI logoGoogle Scholar
Nuzzo, E., & Bove, G.
(2020) Assessing functional adequacy across tasks: A comparison of learners’ and speakers’ written texts. E-JournALL, 7 (2), 9–27. DOI logoGoogle Scholar
(2022) Exploring the pedagogical use of the rating scale for functional adequacy in L1. TASK 2 (1), 115–136. DOI logoGoogle Scholar
Ortega, L.
(2003) Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24 (4), 492–518. DOI logoGoogle Scholar
Pallotti, G.
(2009) CAF: Defining, refining and differentiating constructs. Applied Linguistics, 30 (4), 590–601. DOI logoGoogle Scholar
(2015) A simple view of linguistic complexity. Second Language Research, 31 (1), 117–134. DOI logoGoogle Scholar
Paquot, M.
(2019) The phraseological dimension in interlanguage complexity research. Second Language Research, 35 (1), 121–145. DOI logoGoogle Scholar
Pill, J., & Smart, C.
(2021) Rating: Behavior and training. In P. Winke & T. Brunfaut (Eds.), The Routledge handbook of second language acquisition and language testing (pp. 135–144). Routledge. DOI logoGoogle Scholar
Révész, A., Ekiert, M., & Torgersen, E.
(2016) The effects of complexity, accuracy and fluency on communicative adequacy in oral task performance. Applied Linguistics, 37 (6), 828–848. DOI logoGoogle Scholar
Rezaei, A. R., & Lovorn, M.
(2010) Reliability and validity of rubrics for assessment through writing. Assessing Writing, 15 ( 1 ), 18–39. DOI logoGoogle Scholar
Robinson, P.
(2001) Task complexity, cognitive resources and syllabus design: A triadic framework for examining task influences on SLA. In P. Robinson (Ed.), Cognition and second language instruction (pp. 185–316). Cambridge University Press. DOI logoGoogle Scholar
Skehan, P.
(1996) A framework for the implementation of task-based instruction. Applied Linguistics 17 (1), 38–62. DOI logoGoogle Scholar
Timpe, V.
(2013) Assessing intercultural communicative competence: The dependence of receptive sociopragmatic competence and discourse competence on learning opportunities and input. Peter Lang.Google Scholar
Timpe-Laughlin, V.
(2018) Pragmatics in task-based language assessment. Opportunities and challenges. In N. Taguchi, & Y.-J. Kim (Eds.), Task-based approaches to assessing pragmatics (pp. 288–304). John Benjamins. DOI logoGoogle Scholar
Upshur, J. A., & Turner, C. E.
(1995) Constructing rating scales for second language tests. ELT Journal, 49 (1), 3–12. DOI logoGoogle Scholar
Vasylets, O., Gilabert, R., & Manchón, R. M.
(2019) Differential contribution of oral and written modes to lexical, syntactic and propositional complexity in L2 performance in instructed contexts. Instructed Second Language Acquisition, 3 (2), 206–227. DOI logoGoogle Scholar
Vercellotti, M. A.
(2019) Finding variation: Assessing the development of syntactic complexity in ESL speech. International Journal of Applied Linguistics, 29 (2), 233–247. DOI logoGoogle Scholar
Wolfe-Quintero, K., Inagaki, S., & Kim, H.-Y.
(1998) Second language development in writing: Measures of fluency, accuracy, and complexity. University of Hawai’i Press.Google Scholar
Yuan, F., & Ellis, R.
(2003) The effects of pre-task planning and on-line planning on fluency, complexity and accuracy in L2 monologic production. Applied Linguistics 24 (1), 1–27. DOI logoGoogle Scholar
Zhang, Y., & Elder, C.
(2014) Investigating native and non-native English-speaking teacher raters’ judgements of oral proficiency in the College English Test-Spoken English Test (CET-SET). Assessment in Education: Principles, Policy & Practice, 21 (3), 306–325. DOI logoGoogle Scholar