Cover not available
Article published In:
The Assessment of Functional Adequacy in Language Performance: Special issue of the Journal on Task-Based Language Teaching and Learning 2:1 (2022)
Edited by Folkert Kuiken and Ineke Vedder
[TASK 2:1] 2022
► pp. 85–114

Holistic and analytic assessment of functional adequacy

ORCID logo | University of Modena and Reggio Emilia

This study looks at the correlation between functional adequacy (FA), holistically assessed, and analytic linguistic measures, in a corpus of texts written by Italian monolingual and multilingual primary school pupils. Texts were first evaluated using the FA rating scales by Kuiken and Vedder (2017, 2018), plus one for Coherence & Cohesion from the CEFR (Council of Europe, 2001′). They were then coded for a number of features directly bearing on FA and its subdimensions. Results show correlations between holistic scores and analytic measures, such as those between Content and the number of words or secondary idea units (r = .59 / .65). Others were less strong, yet going in the expected direction, e.g. more ambiguous referential expressions were negatively correlated to Comprehensibility. Correlations were generally stronger for monolingual than for multilingual children.

Keywords: functional adequacy, writing, text quality measures, primary school pupils, assessment
Article outline
  • Previous research
  • Methodology
  • Data analysis
    • Content
    • Coherence & Cohesion
    • Comprehensibility
  • Results: Correlation analysis
  • Results: Regression models
  • Functional adequacy
    • Content
    • Comprehensibility
    • Coherence & Cohesion – Kuiken and Vedder
    • Coherence & Cohesion – CEFR
  • Summary of findings
  • Implications for TBLT and directions for future research
  • Notes
  • References
Published online: 20 June 2022
DOI logo
https://doi.org/10.1075/task.21014.pal
References
Bartning, I., Martin, M., & Vedder, I.
(Eds.) (2010) Communicative proficiency and linguistic development: Intersections between SLA and language testing research. European Second Language Association.Google Scholar
Biber, D., Gray, B., & Staples, S.
(2016) Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37 (5), 639–668. DOI logoGoogle Scholar
Bransford, J. D., & Johnson, M. K.
(1972) Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior, 11 (6), 717–726. DOI logoGoogle Scholar
Council of Europe
(2001) Common European Framework of Reference for Languages: Learning, teaching, assessment. Council of Europe.Google Scholar
Covington, M., & McFall, J.
(2010) Cutting the Gordian knot: The Moving-Average Type–Token Ratio (MATTR). Journal of Quantitative Linguistics, 17 (2), 94–100. DOI logoGoogle Scholar
Crossley, S.
(2020) Linguistic features in writing quality and development: An overview. Journal of Writing Research, 11 (3), 415–443. DOI logoGoogle Scholar
De Jong, N., Steinel, M. P., Florijn, A. F., Schoonen, R., & Hulstijn, J. H.
(2012) The effect of task complexity on functional adequacy, fluency and lexical diversity in speaking performances of native and non-native speakers. In A. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency – Investigating complexity, accuracy and fluency in SLA (pp. 121–142). John Benjamins. DOI logoGoogle Scholar
Douglas, D.
(1994) Quantity and quality in speaking test performance. Language Testing, 11 (2), 125–144. DOI logoGoogle Scholar
Durrant, P., & Durrant, A.
(2022) Appropriateness as an aspect of lexical richness: What do quantitative measures tell us about children’s writing? Assessing Writing, 51 1, 100596. DOI logoGoogle Scholar
Fox, J., & Bouchet-Valat, M.
(2020) Rcmdr: R Commander. [URL]
Fulcher, G.
(1996) Does thick description lead to smart tests? A data-based approach to rating scale construction. Language Testing, 13 (2), 208–238. DOI logoGoogle Scholar
(2015) Re-examining language testing: A philosophical and social enquiry. Routledge. DOI logoGoogle Scholar
Grömping, U.
(2006) Relative importance for linear regression in R: The Package relaimpo. Journal of Statistical Software, 17 (1), 1–27. DOI logoGoogle Scholar
(2015) Variable importance in regression models. WIREs Computational Statistics, 7 (2), 137–152. DOI logoGoogle Scholar
Gu, L., & Hsieh, C.-N.
(2019) Distinguishing features of young English language learners’ oral performance. Language Assessment Quarterly, 16 (2), 180–195. DOI logoGoogle Scholar
Guiraud, P.
(1954) Les caractères statistiques du vocabulaire. Essai de méthodologie. Presses Universitaires de France.Google Scholar
Hulstijn, J. H., Schoonen, R., De Jong, N. H., Steinel, M. P., & Florijn, A.
(2012) Linguistic competences of learners of Dutch as a second language at the B1 and B2 levels of speaking proficiency of the Common European Framework of Reference for Languages (CEFR). Language Testing, 29 (2), 203–221. DOI logoGoogle Scholar
Iwashita, N., Brown, A., McNamara, T., & O’Hagan, S.
(2008) Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics, 29 (1), 24–49. DOI logoGoogle Scholar
Kersten, K.
(2009) Verbal inflections in L2 child narratives. Wissenschaftlicher Verlag Trier.Google Scholar
Khushik, G. A., & Huhta, A.
(2020) Investigating syntactic complexity in EFL learners’ writing across Common European Framework of Reference Levels A1, A2, and B1. Applied Linguistics, 41 (4), 506–532. DOI logoGoogle Scholar
Kuiken, F., & Vedder, I.
(2017) Functional adequacy in L2 writing: Towards a new rating scale. Language Testing, 34 (3), 321–336. DOI logoGoogle Scholar
(2018) Assessing functional adequacy of L2 performance in a task-based approach. In N. Taguchi & Y. Kim (Eds.), Task-Based Language Teaching (pp. 266–285). John Benjamins. DOI logoGoogle Scholar
Kuiken, F., Vedder, I., & Gilabert, R.
(2010) Communicative adequacy and linguistic complexity in L2 writing. In I. Bartning, M. Martin, & I. Vedder (Eds.), Communicative proficiency and linguistic development: Intersections between SLA and language testing research (pp. 81–100). European Second Language Association.Google Scholar
Lahuerta Martínez, A. C.
(2018) Analysis of syntactic complexity in secondary education EFL writers at different proficiency levels. Assessing Writing, 35 1, 1–11. DOI logoGoogle Scholar
Larson-Hall, J.
(2016) A guide to doing statistics in second language research using SPSS and R, 2nd edition. Routledge.Google Scholar
Lee, Y. Y., & Ventura, S.
(2017) Lindia. Automated Linear Regression Diagnostic. [URL]
Lindeman, R. H., Merenda, P. F., & Gold, R. Z.
(1980) Introduction to bivariate and multivariate analysis. Scott, Foresman and Company.Google Scholar
Norman, G.
(2010) Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education, 15 (5), 625–632. DOI logoGoogle Scholar
Olinghouse, N. G., & Wilson, J.
(2013) The relationship between vocabulary and writing quality in three genres. Reading and Writing, 26 (1), 45–65. DOI logoGoogle Scholar
Pallotti, G.
(2009) CAF: Defining, refining and differentiating constructs. Applied Linguistics, 30 (4), 590–601. DOI logoGoogle Scholar
Pallotti, G., Borghetti, C., & Rosi, F.
(2021) Insegnare a scrivere nella scuola primaria. Caissa.Google Scholar
R Core Team
(2020) R: A language and environment for statistical computing. R Foundation for Statistical Computing. [URL]
Révész, A., Ekiert, M., & Torgersen, E. N.
(2016) The effects of complexity, accuracy, and fluency on communicative adequacy in oral task performance. Applied Linguistics, 37 (6), 828–848. DOI logoGoogle Scholar
Roessingh, H., Nordstokke, D., & Colp, M.
(2019) Unlocking academic literacy in Grade 4: The role of handwriting. Reading & Writing Quarterly, 35 (2), 65–83. DOI logoGoogle Scholar
Savignon, S. J.
(2018) Communicative competence. In J. I. Liontas (Ed.), The TESOL Encyclopedia of English Language Teaching (pp. 1–7). John Wiley & Sons. DOI logoGoogle Scholar
Zenker, F., & Kyle, K.
(2021) Investigating minimum text lengths for lexical diversity indices. Assessing Writing, 47 1, 100505. DOI logoGoogle Scholar
Skar, G. B., Lei, P.-W., Graham, S., Aasen, A. J., Johansen, M. B., & Kvistad, A. H.
(2021) Handwriting fluency and the quality of primary grade students’ writing. Reading and Writing. DOI logoGoogle Scholar