Article In:
Chinese as a Second Language (漢語教學研究—美國中文教師學會學報): Online-First ArticlesA machine learning-based investigation of syntactic complexity measures in second language acquisition of Chinese
L2 syntactic complexity research has shown that employing multidimensional measures to assess writing proficiency levels may yield more accurate results compared to one-dimensional measures. However, the optimal number and combination of measures have not been thoroughly explored. This study addresses these gaps by employing a novel machine learning-based approach to quantitatively investigate the effectiveness of multidimensional measures and determine the optimal combination of measures. Through the analysis of a dataset comprising 36 L2 Chinese learners, we found that multidimensional syntactic complexity measures outperformed one-dimensional measures in accurately differentiating learners. Specifically, a combination of three measures (Mean Length of Sentence, Mean Length of T-unit, and Mean Length of Clause) achieved the highest accuracy in classifying the learners. The implications of this research include the development of computer-assisted placement tests and proficiency evaluations, as well as providing practical guidance for language teachers in designing complexity-based instruction.
Keywords: syntactic complexity, machine learning, multidimensionality
Article outline
- Introduction
- Literature review
- Syntactic complexity measures in L2 research
- Overall complexity index — Mean Length of T-unit (MLT)
- Clausal complexity via subordination — Mean number of clauses per T-unit (C/T)
- Clausal complexity via coordination — Mean number of T-unit per sentence (T/S)
- Phrasal complexity index — Mean Length of Clause (MLC)
- Syntactic complexity measures in L2 Chinese research
- Examining syntactic complexity multidimensionally
- Syntactic complexity measures in L2 research
- Research questions
- Methods
- Participants and setting
- Data collection and coding
- Complexity measures used in this study
- Machine learning models
- Results
- Descriptive statistics
- One- and two-measure binary models
- Multinomial models using two or more measures
- Discussion
- Implications and pedagogical significance
- Limitations
- Author queries
-
References
This content is being prepared for publication; it may be subject to changes.
References (27)
ACTFL. (2012). ACTFL Proficiency Guidelines 2012. ACTFL. [URL]
Aydin, S. (2009). Test anxiety among foreign language learners: a review of literature. Journal of Language and Linguistic Studies,
5
(1), 127–137. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bardovi-Harlig, K. (1992). A second look at t-unit analysis: Reconsidering the sentence. TESOL Quarterly,
26
(2), 390. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Byrnes, H., Maxim, H. S., & Norris, J. E. (2010). Realizing advanced foreign language writing development in collegiate education: Curricular design, pedagogy, assessment. The Modern Language Journal,
94
(s), 1–235. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Chen, M. (2015).
Hanyu zuowei di’er yuyan ziran kouyu chanchu de fuzadu, zhunquedu he liulidu yanjiu
汉语作为第二语言自然口语产出的复杂度, 准确度和流利度研究[A complexity, accuracy and fluency study on natural oral production of Chinese as a second language]. Yuyan jiaoxue yu yanjiu 语言教学与研究 [Language Teaching and Linguistic Studies],
3
1, 1–10.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Chen, M., & Li, Y. (2016).
Hanyu muyuzhe Hanyu kouyu fuzadu yanjiu
汉语母语者汉语口语复杂度研究 [The complexity of Chinese oral speeches by Korean native speakers]. Yuyan Wenzi Yingyong 语言文字应用 [Applied Linguistics],
4
1, 61–70.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Flahive, D. E., & Snow, B. G. (1980). Measures of syntactic complexity in evaluating ESL compositions. Research in Language Testing, 171–176.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Halliday, M. A. K., & Matthiessen, C. (2006). Construing experience through meaning : A language-based approach to cognition. Continuum.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Han, X., & Feng, L. (2017).
Hanyu kouyu jufa fuzadu fazhan zeping zhong jizhunxing zhibiao de yingyong fangfa yanjiu
汉语口语句法复杂度发展测评中基准型指标的应用方法研究 [The application method research of benchmark measures in Chinese oral syntactic complexity development assessment]. Shijie Hanyu jiaoxue 世界汉语教学 [Chinese Teaching in the World],
31
(4), 542–559.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hu, R., Wu, J., & Lu, X. (2022). Word-combination-based measures of phraseological diversity, sophistication, and complexity and their relationship to second language Chinese proficiency and writing quality. Language Learning,
72
(4). ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hunt, K. W. (1965). Grammatical structures written at three grade levels. National Council of Teachers of English. [URL]
(1977). Early blooming and late blooming syntactic structures. In C. R. Coper & L. Odell (Eds.), Evaluating Writing: Describing, Measuring, Judging. National Council of Teachers of English.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Jagaiah, T., Olinghouse, N. G., & Kearns, D. M. (2020). Syntactic complexity measures: variation by genre, grade-level, students’ writing abilities, and writing quality. Reading and Writing, 33(10), 2577–2638. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Jiang, W. (2012). Measurements of development in L2 written production: the case of L2 Chinese. Applied Linguistics, 34(1), 1–24. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Jin, H. (2007). Syntactic maturity in second language writings: a case of Chinese as a foreign language (CFL). Journal of Chinese Language Teachers Association,
42
(1).![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Larsen-Freeman, D., & Strom, V. (1977). The construction of a second language acquisition index of development. Language Learning, 27(1), 123–134. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lu, X., & Wu, J. (2022). Noun-phrase complexity measures in Chinese and their relationship to L2 Chinese writing quality: a comparison with topic–comment-unit-based measures. The Modern Language Journal, 106(1), 267–283. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2018). Foundations of machine learning. The MIT Press.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555–578. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24(4), 492–518. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Perkins, K. (1980). Using objective methods of attained writing proficiency to discriminate among holistic evaluations. TESOL Quarterly, 14(1), 61. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Scott, C. M. (1988). Spoken and written syntax. In M. A. Nippold (Ed.), Later Language Development: Ages Nine through Nineteen, 49–95. College-Hill.![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Varoquaux, G., Buitinck, L., Louppe, G., Grisel, O., Pedregosa, F., & Mueller, A. (2015). Scikit-learn. GetMobile: Mobile Computing and Communications, 19(1), 29–33. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Witte, S. P., & Davis, A. J. (1980). The stability of t-unit length: a preliminary investigation. Research in the Teaching of English, 14(1), 5–17. ![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)