Cover not available
Article published In:
Chinese as a Second Language (漢語教學研究—美國中文教師學會學報)
Vol. 56:2 (2021) ► pp.118–145

The effects of linguistic measures in the analysis of L2 Chinese descriptive writing

| Arizona State University

The current study investigates whether the CAF (complexity, accuracy, and fluency) analysis framework, frequently used in L2 studies of English and other languages, may effectively analyze L2 essays in Chinese, a language typologically different from English. CAF measures were examined for both their discriminative and predictive effects for L2 Chinese writing quality. The results revealed that most of the measures captured linguistic differences between lower-score and higher-score essays. Similar to findings in other L2s, more advanced L2 Chinese learners achieved syntactic complexification increasingly through phrasal features in their essays, in contrast to lower-level learners. The predictive effects of the measures for writing quality differed between lower-score and higher-score essays. T-unit length, a syntactic measure successfully analyzing Indo-European languages, did not serve as an effective predictor for L2 Chinese textual quality. The current study provides implications for both L2 writing research and L2 writing curriculum.

Keywords: lexical complexity, syntactic complexity, accuracy, fluency, discriminative effect, predictive effect, L2 Chinese
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1L2 writing constructs and measures
    • 2.2L2 writing performance in CAF measures
      • 2.2.1L2 written fluency, accuracy, and lexical complexity
      • 2.2.2L2 written syntactic complexity
      • 2.2.3Relationships among CAF sub-dimensions
    • 2.3Predictive effects of CAF measures for human judgments of L2 writing quality
  • 3.The present study
    • 3.1Participants and dataset
    • 3.2Measures in the current study
    • 3.3Analysis
  • 4.Results
    • 4.1Research question 1: CAF analysis of lower-score and higher-score essays
    • 4.2Research question 2: Predictive power of the CAF measures for human ratings of writing quality
  • 5.Discussion
    • 5.1Lower-score and higher-score writing performance according to CAF measures
    • 5.2Predictive effects of the CAF measures for human ratings of writing quality
    • 5.3Discrepancy between discriminative indicators and predictors of ratings
  • 6.Conclusion
  • Note
  • References
Published online: 4 February 2022
DOI logo
https://doi.org/10.1075/csl.21012.lia
References
ACTFL
(2012) ACTFL proficiency guidelines. Retrieved from [URL]
An, F.
(2015) Butong shuiping CSL xuexizhe zuowen liuchangxing, jufa fuzadu he zhunquexing fenxi – Yixiang jiyu T-danwei celiangfa de yanjiu 不同水平CSL学习者作文流畅性、句法复杂度和准确性分析—一项基于T单位测量法的研究 [Analysis of fluency, grammatical complexity, and accuracy of CSL writing: A study based on T-unit analysis]. Language Teaching and Linguistics Studies 语言教学与研究, 3 1, 11–20.Google Scholar
Biber, D., Gray, B., & Poonpon, K.
(2011) Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? TESOL Quarterly, 45 1, 5–35. DOI logoGoogle Scholar
Biber, D., Gray, B., & Staples, S.
(2016) Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37 1, 639–668. DOI logoGoogle Scholar
Bulté, B., & Housen, A.
(2012) Defining and operationalising L2 complexity. In A. Housen, F. Kuiken & I. Vedder (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp. 21–46). Amsterdam, The Netherlands: John Benjamins. DOI logoGoogle Scholar
(2014) Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26 1, 42–65. DOI logoGoogle Scholar
Casal, J. E., & Lee, J.
(2019) Syntactic complexity and writing quality in assessed first-year L2 writing. Journal of Second Language Writing, 44 1, 51–62. DOI logoGoogle Scholar
Chu, C.
(1998) A discourse grammar of Mandarin Chinese. New York, NY: P. Lang.Google Scholar
Crossley, S., & McNamara, D.
(2012) Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication. Journal of Research in Reading, 35 1, 115–135. DOI logoGoogle Scholar
(2014) Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners. Journal of Second Language Writing, 26 1, 66–79. DOI logoGoogle Scholar
Engber, C.
(1995) The relationship of lexical proficiency to the quality of ESL compositions. Journal of Second Language Writing, 4 1, 139–155. DOI logoGoogle Scholar
Ferris, D.
(1994) Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly, 28 1, 414–420. DOI logoGoogle Scholar
Foster, P., Tonkyn, A., & Wigglesworth, G.
(2000) Measuring spoken language: A unit for all reasons. Applied Linguistics, 21 1, 354–375. DOI logoGoogle Scholar
Garner, J., Crossley, S., & Kyle, K.
(2019) N-gram measures and L2 writing proficiency. System, 80 1, 176–187. DOI logoGoogle Scholar
Grant, L., & Ginther, A.
(2000) Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9 1, 123–145. DOI logoGoogle Scholar
Guiraud, P.
(1960) Problémes et méthodes de la statistique linguistique [Problems and methods of statistical linguistics]. Dordrecht, The Netherlands: Reidel.Google Scholar
Halliday, M., & Matthiessen, C.
(1999) Construing experience through meaning: A language-based approach to cognition. London, UK: Cassell.Google Scholar
Hartshorn, K., Evans, N., Merrill, P., Sudweeks, R., Strong-Krause, D., & Anderson, N.
(2010) Effects of dynamic corrective feedback on ESL writing accuracy. TESOL Quarterly, 44 1, 84–109. DOI logoGoogle Scholar
Hu, R.
(2021) Jiyu dapei de jufa fuzadu zhibiao jiqi yu hanyu eryu xiezuo zhiliang guanxi yanjiu 基于搭配的句法复杂度指标及其与汉语二语写作质量关系研究 [On the relationship between collocation-based syntactic complexity and Chinese second language writing]. Applied Linguistics 语言文字应用, 1 1, 132–144. DOI logoGoogle Scholar
Huang, B., & Liao, X.
(2007) Xiandai Hanyu 现代汉语 [Modern Chinese Language]. Beijing, China: Higher Education Press.Google Scholar
Hunt, K.
(1966) Recent measures in syntactic development. Elementary English, 43 1, 732–739.Google Scholar
Ishikawa, S.
(1995) Objective measurement of low-proficiency EFL narrative writing. Journal of Second Language Writing, 4 1, 51–69. DOI logoGoogle Scholar
Jarvis, S.
(2002) Short texts, best-fitting curves and new measures of lexical diversity. Language Testing, 19 1, 57–84. DOI logoGoogle Scholar
Jiang, W.
(2013) Measurements of development in L2 written production: The case of L2 Chinese. Applied Linguistics, 34 1, 1–24. DOI logoGoogle Scholar
Johnson, M.
(2017) Cognitive task complexity and L2 written syntactic complexity, accuracy, lexical complexity, and fluency: A research synthesis and meta-analysis. Journal of Second Language Writing, 37 1, 13–38. DOI logoGoogle Scholar
Johnson, M., Mercado, L., Acevedo, A.
(2012) The effect of planning sub-processes on L2 writing fluency, grammatical complexity, and lexical complexity. Journal of Second Language Writing, 21 1, 264–282. DOI logoGoogle Scholar
Kormos, J.
(2011) Task complexity and linguistic and discourse features of narrative writing performance. Journal of Second Language Writing, 20 1, 148–161. DOI logoGoogle Scholar
Kuiken, F., & Vedder, I.
(2019) Syntactic complexity across proficiency and languages: L2 and L1 writing in Dutch, Italian and Spanish. International Journal of Applied Linguistics, 29 1, 192–210. DOI logoGoogle Scholar
Kyle, K., & Crossley, S.
(2018) Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices. Modern Language Journal, 102 1, 333–349. DOI logoGoogle Scholar
Larsen-Freeman, D.
(1978) An ESL index of development. TESOL Quarterly, 12 1, 439–448. DOI logoGoogle Scholar
(1983) Assessing global second language proficiency. In H. W. Seliger & M. H. Long (Eds.), Classroom oriented research in second language acquisition (pp. 287–304). Rowley, MA: Newbury House.Google Scholar
(2006) The emergence of complexity, fluency, and accuracy in the oral and written production of five Chinese learners of English. Applied Linguistics, 27 1, 590–619. DOI logoGoogle Scholar
(2015) Saying what we mean: Making a case for ‘language acquisition’ to become ‘language development’. Language Teaching, 48 1, 491–505. DOI logoGoogle Scholar
Lesonen, S., Steinkrauss, R., Suni, M., & Verspoor, M.
(2021) Dynamic usage-based principles in the development of L2 Finnish evaluative constructions. Applied Linguistics, 42 1, 442–472. DOI logoGoogle Scholar
Lowie, W., & Verspoor, M.
(2015) Variability and variation in second language acquisition orders: A dynamic reevaluation. Language Learning, 65 1, 63–88. DOI logoGoogle Scholar
Lu, X.
(2011) A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 45 1, 36–62. DOI logoGoogle Scholar
Mazgutova, D., & Kormos, J.
(2015) Syntactic and lexical development in an intensive English for academic purposes programme. Journal of Second Language Writing, 29 1, 3–15. DOI logoGoogle Scholar
Miller, K.
(2000) Academic writers on-line: Investigating pausing in the production of text. Language Teaching Research, 4 1, 123–148. DOI logoGoogle Scholar
Mostafa, T., & Crossley, S.
(2020) Verb argument construction complexity indices and L2 writing quality: Effects of writing tasks and prompts. Journal of Second Language Writing, 49 1, 1–13. DOI logoGoogle Scholar
Norris, J., & Ortega, L.
(2009) Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30 1, 555–578. DOI logoGoogle Scholar
Parkinson, J., & Musgrave, J.
(2014) Development of noun phrase complexity in the writing of English for academic purposes students. Journal of English for Academic Purposes, 14 1, 48–59. DOI logoGoogle Scholar
Polio, C.
(1997) Measures of linguistic accuracy in second language writing research. Language Learning, 47 1, 101–143. DOI logoGoogle Scholar
Polio, C., Fleck, C., & Leder, N.
(1998) “If I only had more time:” ESL learners’ changes in linguistic accuracy on essay revisions. Journal of Second Language Writing, 7 1, 43–68. DOI logoGoogle Scholar
Polio, C., & Shea, M.
(2014) An investigation into current measures of linguistic accuracy in second language writing research. Journal of Second Language Writing, 26 1, 10–27. DOI logoGoogle Scholar
Read, J.
(2000) Assessing vocabulary. Oxford, UK: Oxford University Press. DOI logoGoogle Scholar
Reid, J.
(1990) Responding to different topic types: A quantitative analysis from a contrastive rhetoric perspective. In B. Kroll (Ed.), Second language writing: Research insights for the classroom (pp. 191–210). Cambridge, UK: Cambridge University Press. DOI logoGoogle Scholar
Révész, A., Kourtali, N., & Mazgutova, D.
(2017) Effects of task complexity on L2 writing behaviors and linguistic complexity. Language Learning, 67 1, 208–241. DOI logoGoogle Scholar
Skehan, P.
(2009) Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 30 1, 510–532. DOI logoGoogle Scholar
Spoelman, M., & Verspoor, M.
(2010) Dynamic patterns in development of accuracy and complexity: A longitudinal case study in the acquisition of Finnish. Applied Linguistics, 31 1, 532–553. DOI logoGoogle Scholar
Ure, J.
(1971) Lexical density: A computational technique and some findings. In M. Coultard (Ed.), Talking about text (pp. 27–48). Birmingham, UK: English Language Research, University of Birmingham.Google Scholar
van Hout, R., & Vermeer, A.
(2007) Comparing measures of lexical richness. In H. Daller, J. Milton & J. Treffers-Daller (Eds.), Modelling and assessing vocabulary knowledge (pp. 93–115). Cambridge, UK: Cambridge University Press. DOI logoGoogle Scholar
Verspoor, M., Lowie, W., & van Dijk, M.
(2008) Variability in second language development from a dynamic systems perspective. Modern Language Journal, 92 1, 214–231. DOI logoGoogle Scholar
Verspoor, M., Schmid, M., & Xu, X.
(2012) A dynamic usage based perspective on L2 writing. Journal of Second Language Writing, 21 1, 239–263. DOI logoGoogle Scholar
Vyatkina, N.
(2012) The development of second language writing complexity in groups and individuals: A longitudinal learner corpus study. Modern Language Journal, 96 1, 576–598. DOI logoGoogle Scholar
Wang, Y.
(2017) Hanyu eryuzhe cihui fengfuxing yu xiezuo chengji de xianguanxing 汉语二语者词汇丰富性与写作成绩的相关性 [The correlation between lexical richness and writing score of CSL learner]. Applied Linguistics 语言文字应用, 2 1, 93–101. Cite to nonCR doi: Google Scholar
Wolfe-Quintero, K., Inagaki, S., & Kim, H.
(1998) Second language development in writing: Measures of fluency, accuracy & complexity. Honolulu, HI: Second Language Teaching & Curriculum Center, University of Hawaii at Manoa.Google Scholar
Wu, J.
(2016) Yingyu muyuzhe hanyu xiezuo zhong de cihui fengfuxing fazhan yanjiu 英语母语者汉语写作中的词汇丰富性发展研究 [Research on lexical richness development in CSL writing by English native speakers]. Chinese Teaching in the World 世界汉语教学, 30 1, 129–142. Cite to nonCR doi: Google Scholar
Yang, W., Lu, X., & Weigle, S.
(2015) Different topics, different discourse: Relationships among writing topic, measures of syntactic complexity, and judgments of writing quality. Journal of Second Language Writing, 28 1, 53–67. DOI logoGoogle Scholar
Yoon, H., & Polio, C.
(2017) The linguistic development of students of English as a second language in two written genres. TESOL Quarterly, 51 1, 275–301. DOI logoGoogle Scholar
Yu, Q.
(2021) An organic syntactic complexity measure for the Chinese language: The TC-unit. Applied Linguistics, 42 1, 60–92. DOI logoGoogle Scholar
Zenker, F., & Kyle, K.
(2021) Investigating minimum text lengths for lexical diversity indices. Assessing Writing, 47 1, 1–15. DOI logoGoogle Scholar