Chapter published in:
Task-Based Approaches to Teaching and Assessing Pragmatics
Edited by Naoko Taguchi and YouJin Kim
[Task-Based Language Teaching 10] 2018
► pp. 266–285

Chapter 11
Assessing functional adequacy of L2 performance in a task-based approach

| University of Amsterdam
| University of Amsterdam
When assessing L2 performance in task-based research, dimensions of complexity, accuracy, and fluency (CAF) have typically been evaluated. Less attention has, however, been devoted to the functional dimension. This paper argues that it is critical to consider the functional dimension of L2 performance in addition to complexity, accuracy, and fluency. Functional adequacy as a task-related construct is viewed in the present study as a component of L2 pragmatics, referring to the appropriateness and felicity of the utterances of the speaker/writer within a particular context, and evaluated by the listener/reader. The study investigates the applicability of a rating scale developed for the assessment of functional adequacy in the L2, considered from the perspective of task-based language teaching (TBLT) and task-based language assessment (TBLA), as successful task completion. In the rating scale, four components of functional adequacy are distinguished: content, task requirements, comprehensibility, and coherence and cohesion. A group of non-expert raters judged the oral and written samples of two groups of university students of Dutch L2 and Italian L2. The results show that the scale appears to be a reliable and efficient tool for assessing the functional adequacy of written and spoken L2 production.
Article outline
  • Introduction
  • Assessment of L2 pragmatics
  • The construction of a rating scale for functional adequacy
  • Methodology
    • Participants
    • L2 learners
    • Tasks
    • Rating procedure
    • Data analysis
  • Results
    • Interrater reliability
    • Correlations between dimensions of functional adequacy
    • Correlations between task 1 and task 2
    • Raters’ perceptions of functional adequacy
  • Discussion
  • Conclusion
  • Acknowledgements
  • Notes
  • References
Published online: 15 August 2018
https://doi.org/10.1075/tblt.10.11kui
References
Austin, J. L.
(1962) How to do things with words. Oxford: Oxford University Press.Google Scholar
Bachman, L. F.
(2002) Some reflections on task-based language performance assessment. Language Testing, 19, 453–476. CrossrefGoogle Scholar
Bardovi-Harlig, K.
(2009) Conventional expressions as a pragmalinguistic resource: Recognition and production of conventional expressions in L2 pragmatics. Language Learning, 59, 755–795. CrossrefGoogle Scholar
Barron, A.
(2003) Acquisition in interlanguage pragmatics. Learning how to do things with words in a study abroad context. Amsterdam: John Benjamins. CrossrefGoogle Scholar
Bouton, L. F.
(1994) Conversational implicature in the second language. Learned slowly when not deliberately taught. Journal of Pragmatics, 22, 157–167. CrossrefGoogle Scholar
(1999) Developing non-native speaker skills in interpreting conversational implicature in English: Explicit teaching can ease the process. In E. Hinkel (Ed.), Culture in second language teaching and learning (pp. 47–70). Cambridge: Cambridge University Press.Google Scholar
Bridgeman, B., Powers, D., Stone, E., & Mollaun, P.
(2012) TOEFL iBT speaking test scores as indicators of oral communicative language proficiency. Language Testing, 29(1), 91–108. CrossrefGoogle Scholar
Brown, P., & Levinson, S. D.
(1987) Politeness: Some universals in language usage. Cambridge: Cambridge University Press.CrossrefGoogle Scholar
Cortés Velásquez, D., & Nuzzo, E.
2017). Assessing L1 functional adequacy: Can we use the same scale as for L2. Paper presented at TBLT 2017, Barcelona.
Council of Europe
(2001) Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge: Cambridge University Press.Google Scholar
Cumming, A., Kantor, R., & Powers, D. E.
(2002) Decision making while rating ESL/EFL writing tasks: A descriptive framework. Modern Language Journal, 86, 67–96. CrossrefGoogle Scholar
De Jong, N. H., Steinel, M. P., Florijn, A. F., Schoonen, R., & Hulstijn, J. H.
(2012a) The effect of task complexity on functional adequacy, fluency and lexical diversity in speaking performances of native and non-native speakers. In A. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency. Complexity, accuracy and fluency in SLA (pp. 121–142). Amsterdam: John Benjamins. CrossrefGoogle Scholar
(2012b) Facets of speaking proficiency. Studies in Second Language Acquisition, 34(1), 5–34. CrossrefGoogle Scholar
Faone, S., Pagliara, F., & Vitale, G.
(2017) How to access L2 information-gap tasks through functional adequacy rating scales. Paper presented at TBLT 2017, Barcelona.
González-Lloret, M.
(2016) A practical guide to integrating technology into task-based language teaching. Washington, DC: Georgetown University Press.Google Scholar
Grice, H. P.
(1975) Logic and conversation. In P. Cole & J. L. Morgan (Eds.), Speech acts (pp. 41–58). New York, NY: Academic Press.Google Scholar
Housen, A., Kuiken, F., & Vedder, I.
(Eds.) (2012) Dimensions of L2 performance and proficiency. Complexity, accuracy and fluency in SLA. Amsterdam: John Benjamins. CrossrefGoogle Scholar
Hulstijn, J. H., Alderson, J. C., & Schoonen, R.
(2010) Developmental stages in second-language acquisition and levels of second-language proficiency: Are there links between them? In I. Bartning, M. Martin, & I. Vedder (Eds.), Communicative proficiency and linguistic development: Intersections between SLA and language testing research (pp. 11–20). (Eurosla Monographs Series 1).Google Scholar
Iwashita, N., Brown, A., McNamara, T., & O’Hagan, S.
(2008) Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics, 29(1), 24–49. CrossrefGoogle Scholar
Knoch, U.
(2007) ‘Little coherence, considerable strain for reader’: A comparison between two rating scales for the assessment of coherence. Assessing Writing, 12(2), 108–128. CrossrefGoogle Scholar
(2009) Diagnostic assessment of writing: A comparison of two rating scales. Language Testing, 26(2), 275. CrossrefGoogle Scholar
(2011) Rating scales for diagnostic assessment of writing: What should they look like and where should the criteria come from? Assessing Writing, 16(2), 81–96. CrossrefGoogle Scholar
Kuiken, F., & Vedder, I.
(2014) Rating written performance: What do raters do and why? Language Testing, 31(3), 329–348. CrossrefGoogle Scholar
(2017) Functional adequacy in L2 writing. Towards a new rating scale. Language Testing. Language Testing, 34(3), 321–336.CrossrefGoogle Scholar
Kuiken, F., Vedder, I., & Gilabert, R.
(2010) Communicative adequacy and linguistic complexity in L2 writing. In I. Bartning, M. Martin, & I. Vedder (Eds.), Communicative proficiency and linguistic development: Intersections between SLA and language testing research (pp. 81–100). (Eurosla Monographs Series 1).Google Scholar
Long, M. H.
(2015), Second language acquisition and task-based language teaching. Malden, MA: Wiley Blackwell.Google Scholar
Martinez-Flor, A., & Usó-Juan, E.
(Eds.) (2010) Speech act performance: Theoretical, empirical, and methodological issues. Amsterdam: John Benjamins. CrossrefGoogle Scholar
McNamara, T.
(1996) Measuring second language performance. New York, NY: Addison Wesley, Longman.Google Scholar
(1997) ‘Instruction’ in second language performance assessment: Whose performance? Applied Linguistics, 18(4), 446–466. CrossrefGoogle Scholar
McNamara, T., & Roever, C.
(2007) Testing: The social dimension. Malden, MA: Blackwell.Google Scholar
Norris, J. M.
(2001) Identifying rating criteria for task-based EAP assessment. In T. Hudson & J. D. Brown (Eds.), A focus on language test development: Expanding the language proficiency construct across a variety of tests (Technical Report #21, pp. 163–204). Honolulu, HI: University of Hawai’i, Second Language Teaching and Curriculum Center.Google Scholar
(Ed.) (2002) Special issue: Task-based language assessment. Language Testing, 19(4). CrossrefGoogle Scholar
(2009) Task-based teaching and testing. In M. Long & C. Doughty (Eds.), Handbook of second and foreign language teaching (pp. 578–594). Cambridge: Blackwell. CrossrefGoogle Scholar
(2016) Current uses for task-based language assessment. Annual Review of Applied Linguistics, 36, 230–244.CrossrefGoogle Scholar
Norris, J. M., Brown, J. D., Hudson, T., & Yoshioka, J.
(1998) Designing second language performance assessments (Technical Report #18). Honolulu, HI: University of Hawai’i, Second Language Teaching and Curriculum Center.Google Scholar
Pallotti, G.
2017). Percorsi di educazione linguistica efficace per ridurre le diseguaglianze. In M. Vedovelli Ed. L’italiano dei nuovi italiani Atti del XIX Convegno Nazionale GISCEL pp. 505 520 Roma Aracne
Roever, C.
(2005) Testing ESL Pragmatics: Development and validation of a web-based assessment battery. Berlin: Peter Lang.CrossrefGoogle Scholar
(2011) Testing of second language pragmatics: Past and future. Language Testing, 28(4), 463–481. CrossrefGoogle Scholar
(2012) What learners get for free: Learning of pragmatic formulae in ESL and EFL environments. The ELT Journal, 66(1), 10–21. CrossrefGoogle Scholar
Ross, S. J., & Kasper, G.
(Eds.) (2013) Assessing second language pragmatics. Houndmills: Palgrave, Macmillan. CrossrefGoogle Scholar
Schoonen, R., Vergeer, M., & Eiting, M.
(1997) The assessment of writing ability: Expert readers versus lay readers. Language Testing, 14(2), 157–184. CrossrefGoogle Scholar
Searle, J. R.
(1969) Speech acts: An essay in the philosophy of language. Cambridge: Cambridge University Press, CrossrefGoogle Scholar
(1975) Indirect speech acts. In P. Cole & J. L. Morgan (Eds.). Syntax and semantics 3: Speech acts (pp. 59–82). New York, NY: Academic Press.Google Scholar
Shehadeh, A.
(2012) Task-based language assessment: Components, developments, and implementation. In C. Coombe, P. Davidson, B. O’Sullivan, & S. Stoynoff (Eds.), The Cambridge guide to second language assessment (pp. 156–163). Cambridge: Cambridge University Press.Google Scholar
Taguchi, N.
(2009) Corpus-informed assessment of L2 comprehension of conversational implicatures. TESOL Quarterly, 43, 738–749. CrossrefGoogle Scholar
(2011) The effect of L2 proficiency and study-abroad experience in pragmatic comprehension. Language Learning, 61, 904–939. CrossrefGoogle Scholar
(2012) Context, individual differences and pragmatic competence. New York, NY: Multilingual Matters.Google Scholar
(2013) Production of routines in L2 English: Effect of proficiency and study-abroad experience. System, 41, 109–121. CrossrefGoogle Scholar
Thompson, I.
(1991) Foreign accents revisited: Factors relating to transfer of accent from the first language to a second language. Language and Speech, 24(3), 265–272.Google Scholar
Upshur, J. A., & Turner, C. E.
(1995) Constructing rating scales for second language tests. ELT Journal, 49(1), 3–12. CrossrefGoogle Scholar
Vitale, M., De Meo, A., & Pettorino, M.
(2012) Foreign accent and persuasiveness. Native and non native voices in a radio spot. In A. De Meo & M. Pettorino (Eds.), Prosodic and rhythmic aspects of L2 acquisition. The case of Italian (pp. 213–222). Newcastle upon Tyne: Cambridge Scholars.Google Scholar
Walters, F.
(2013) Interfaces between a discourse completion test and a conversation analysis informed test of L2 pragmatic competence. In S. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 172–195). Houndmills: Palgrave Macmillan. CrossrefGoogle Scholar
Cited by

Cited by 6 other publications

Del Bono, Federica & Elisabetta Bonvino
2021. Dalla valutazione in DaD alle scale di valutazione dell’adeguatezza. EuroAmerican Journal of Applied Linguistics and Languages 8:2 ► pp. 105 ff. Crossref logo
Galbraith, David & Ineke Vedder
2019. METHODOLOGICAL ADVANCES IN INVESTIGATING L2 WRITING PROCESSES. Studies in Second Language Acquisition 41:3 ► pp. 633 ff. Crossref logo
Kuiken, Folkert, Ineke Vedder, Alex Housen & Bastien De Clercq
2019. Variation in syntactic complexity: Introduction. International Journal of Applied Linguistics Crossref logo
Narke, Pankaj
2021.  In Task-Based Language Teaching and Assessment, ► pp. 249 ff. Crossref logo
Nuzzo, Elena & Giuseppe Bove
2020. Assessing functional adequacy across tasks: A comparison of learners’ and native speakers’ written texts. EuroAmerican Journal of Applied Linguistics and Languages 7:2 ► pp. 9 ff. Crossref logo
Schmidgall, Jonathan & Donald E. Powers
2021. Predicting communicative effectiveness in the international workplace: Support for TOEIC® Speaking test scores from linguistic laypersons. Language Testing 38:2 ► pp. 302 ff. Crossref logo

This list is based on CrossRef data as of 31 march 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.