Article In:
Chinese as a Second Language (漢語教學研究—美國中文教師學會學報): Online-First Articles

A machine learning-based investigation of syntactic complexity measures in second language acquisition of Chinese

ORCID logo | Columbia University,
| Harvard University,
L2 syntactic complexity research has shown that employing multidimensional measures to assess writing proficiency levels may yield more accurate results compared to one-dimensional measures. However, the optimal number and combination of measures have not been thoroughly explored. This study addresses these gaps by employing a novel machine learning-based approach to quantitatively investigate the effectiveness of multidimensional measures and determine the optimal combination of measures. Through the analysis of a dataset comprising 36 L2 Chinese learners, we found that multidimensional syntactic complexity measures outperformed one-dimensional measures in accurately differentiating learners. Specifically, a combination of three measures (Mean Length of Sentence, Mean Length of T-unit, and Mean Length of Clause) achieved the highest accuracy in classifying the learners. The implications of this research include the development of computer-assisted placement tests and proficiency evaluations, as well as providing practical guidance for language teachers in designing complexity-based instruction.
Keywords: syntactic complexity, machine learning, multidimensionality
Article outline
  • Introduction
  • Literature review
    • Syntactic complexity measures in L2 research
      • Overall complexity index — Mean Length of T-unit (MLT)
      • Clausal complexity via subordination — Mean number of clauses per T-unit (C/T)
      • Clausal complexity via coordination — Mean number of T-unit per sentence (T/S)
      • Phrasal complexity index — Mean Length of Clause (MLC)
    • Syntactic complexity measures in L2 Chinese research
    • Examining syntactic complexity multidimensionally
  • Research questions
  • Methods
    • Participants and setting
    • Data collection and coding
    • Complexity measures used in this study
    • Machine learning models
  • Results
    • Descriptive statistics
    • One- and two-measure binary models
    • Multinomial models using two or more measures
  • Discussion
  • Implications and pedagogical significance
  • Limitations
  • Author queries
  • References
This content is being prepared for publication; it may be subject to changes.
References (27)
References
ACTFL. (2012). ACTFL Proficiency Guidelines 2012. ACTFL. [URL]
Aydin, S. (2009). Test anxiety among foreign language learners: a review of literature. Journal of Language and Linguistic Studies, 5 (1), 127–137. DOI logoGoogle Scholar
Bardovi-Harlig, K. (1992). A second look at t-unit analysis: Reconsidering the sentence. TESOL Quarterly, 26 (2), 390. DOI logoGoogle Scholar
Byrnes, H., Maxim, H. S., & Norris, J. E. (2010). Realizing advanced foreign language writing development in collegiate education: Curricular design, pedagogy, assessment. The Modern Language Journal, 94 (s), 1–235. DOI logoGoogle Scholar
Chen, M. (2015). Hanyu zuowei di’er yuyan ziran kouyu chanchu de fuzadu, zhunquedu he liulidu yanjiu 汉语作为第二语言自然口语产出的复杂度, 准确度和流利度研究[A complexity, accuracy and fluency study on natural oral production of Chinese as a second language]. Yuyan jiaoxue yu yanjiu 语言教学与研究 [Language Teaching and Linguistic Studies], 3 1, 1–10.Google Scholar
Chen, M., & Li, Y. (2016). Hanyu muyuzhe Hanyu kouyu fuzadu yanjiu 汉语母语者汉语口语复杂度研究 [The complexity of Chinese oral speeches by Korean native speakers]. Yuyan Wenzi Yingyong 语言文字应用 [Applied Linguistics], 4 1, 61–70.Google Scholar
Flahive, D. E., & Snow, B. G. (1980). Measures of syntactic complexity in evaluating ESL compositions. Research in Language Testing, 171–176.Google Scholar
Halliday, M. A. K., & Matthiessen, C. (2006). Construing experience through meaning : A language-based approach to cognition. Continuum.Google Scholar
Han, X., & Feng, L. (2017). Hanyu kouyu jufa fuzadu fazhan zeping zhong jizhunxing zhibiao de yingyong fangfa yanjiu 汉语口语句法复杂度发展测评中基准型指标的应用方法研究 [The application method research of benchmark measures in Chinese oral syntactic complexity development assessment]. Shijie Hanyu jiaoxue 世界汉语教学 [Chinese Teaching in the World], 31 (4), 542–559.Google Scholar
Hu, R., Wu, J., & Lu, X. (2022). Word-combination-based measures of phraseological diversity, sophistication, and complexity and their relationship to second language Chinese proficiency and writing quality. Language Learning, 72 (4). DOI logoGoogle Scholar
Hunt, K. W. (1965). Grammatical structures written at three grade levels. National Council of Teachers of English. [URL]
(1977). Early blooming and late blooming syntactic structures. In C. R. Coper & L. Odell (Eds.), Evaluating Writing: Describing, Measuring, Judging. National Council of Teachers of English.Google Scholar
Jagaiah, T., Olinghouse, N. G., & Kearns, D. M. (2020). Syntactic complexity measures: variation by genre, grade-level, students’ writing abilities, and writing quality. Reading and Writing, 33(10), 2577–2638. DOI logoGoogle Scholar
Jiang, W. (2012). Measurements of development in L2 written production: the case of L2 Chinese. Applied Linguistics, 34(1), 1–24. DOI logoGoogle Scholar
Jin, H. (2007). Syntactic maturity in second language writings: a case of Chinese as a foreign language (CFL). Journal of Chinese Language Teachers Association, 42 (1).Google Scholar
Larsen-Freeman, D., & Strom, V. (1977). The construction of a second language acquisition index of development. Language Learning, 27(1), 123–134. DOI logoGoogle Scholar
Lu, X., & Wu, J. (2022). Noun-phrase complexity measures in Chinese and their relationship to L2 Chinese writing quality: a comparison with topic–comment-unit-based measures. The Modern Language Journal, 106(1), 267–283. DOI logoGoogle Scholar
Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2018). Foundations of machine learning. The MIT Press.Google Scholar
Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555–578. DOI logoGoogle Scholar
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24(4), 492–518. DOI logoGoogle Scholar
Perkins, K. (1980). Using objective methods of attained writing proficiency to discriminate among holistic evaluations. TESOL Quarterly, 14(1), 61. DOI logoGoogle Scholar
Richards, J. C. (2008). Moving beyond the plateau. Cambridge University Press.Google Scholar
Scott, C. M. (1988). Spoken and written syntax. In M. A. Nippold (Ed.), Later Language Development: Ages Nine through Nineteen, 49–95. College-Hill.Google Scholar
Varoquaux, G., Buitinck, L., Louppe, G., Grisel, O., Pedregosa, F., & Mueller, A. (2015). Scikit-learn. GetMobile: Mobile Computing and Communications, 19(1), 29–33. DOI logoGoogle Scholar
Witte, S. P., & Davis, A. J. (1980). The stability of t-unit length: a preliminary investigation. Research in the Teaching of English, 14(1), 5–17. DOI logoGoogle Scholar
Yu, Q. (2021). An organic syntactic complexity measure for the Chinese language: the tc-unit. Applied Linguistics, 42 (1), 60–92. DOI logoGoogle Scholar
Yuan, F. (2009). Measuring learner language in L2 Chinese in fluency, accuracy and complexity. Journal of Chinese Language Teachers Association, 44(3), 109–130.Google Scholar