Article published In:
Chinese as a Second Language (漢語教學研究—美國中文教師學會學報): Online-First Articles

Exploring the Chinese AWCF Platform’s value in improving CSL learners’ writing performance in a ChatGPT context

ORCID logo | University of Pennsylvania
 | University of Pennsylvania
This study evaluates the efficacy of Chinese Automated Writing Correction Feedback (AWCF) platforms in enhancing writing performance among Chinese as a Second Language (CSL) learners against the backdrop of emerging generative AI tools like ChatGPT. Employing data from an intermediate CSL course at an Ivy League university, the paper scrutinizes selected Chinese AWCF platforms, such as Meta XiezuoCat, for their accuracy, effectiveness, and instructional potential. The analysis juxtaposes AWCF feedback withChatGPT responses and instructor assessments, revealing both the strengths and limitations of these platforms. The findings indicate that existing Chinese AWCF platforms exhibit noticeable limitations in CFL writing support but hold considerable potential for future development. Despite limitations in current corpus and accuracy compared with Generative AI such as ChatGPT, AWCF’s one-stop service model in error correction significantly reduces the cost of prompt engineering, while its feature of allowing “noticing” can help enhance students’ metalinguistic awareness. This finding also highlights the critical need for AWCF platforms to improve their utility by incorporating expansive and authentic CSL learner data.
Keywords: Chinese as a Second Language, automated writing correction feedback (AWCF), generative AI, Computer Assisted Language Learning (CALL), metalinguistic awareness
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1Historical development of AWCF platforms
    • 2.2AWCF in Second Language Education
    • 2.3Chinese AWCF as an under-researched tool
  • 3.Research questions
  • 4.Methodology
  • 5.Findings
    • 5.1Review of Chinese AWCF platforms’ functions
    • 5.2Comparing the accuracy of meta XiezuoCat and ChatGPT’s feedback
    • 5.3Comparing effectiveness: Between critical error identification and over-correction
  • 6.Discussions and implications
  • 7.Limitation and future studies
  • 8.Conclusion
  • References
Published online: 12 November 2024
DOI logo
https://doi.org/10.1075/csl.24008.yan
References (23)
References
Aijiaodui. (2022). Product capabilities. [URL]
Barrot, J. S. (2023a). Using CHATGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57 1, 100745. DOI logoGoogle Scholar
(2023b). Using automated written corrective feedback in the writing classrooms: Effects on L2 writing accuracy. Computer Assisted Language Learning, 36 (4), 584–607. DOI logoGoogle Scholar
Fan, N. (2023). Exploring the effects of automated written corrective feedback on EFL students’ writing quality: A mixed-methods study. Sage Open, 13(2). DOI logoGoogle Scholar
Google. (2023). What is generative AI and what are its applications? Google Cloud. Retrieved February 25, 2024, from [URL]
Grammarly. (2024). About Us. [URL]
Gray, R. (2004). Grammar correction in ESL/EFL writing classes may not be effective. The TESL Journal, X (11).Google Scholar
Guo, Q., Feng, R., & Hua, Y. (2021). How effectively can EFL students use automated written corrective feedback (AWCF) in research writing? Computer Assisted Language Learning, 35 (9), 2312–2331. DOI logoGoogle Scholar
Koltovskaia, S. (2020). Student engagement with automated written corrective feedback (AWCF) provided by Grammarly: A multiple case study. Assessing Writing, 44 1, 100450. DOI logoGoogle Scholar
Lee, I. (2003). L2 writing teachers’ perspectives, practices and problems regarding error feedback. Assessing Writing, 8 (3), 216–237. DOI logoGoogle Scholar
Li, J., Link, S., & Hegelheimer, V. (2015). Rethinking the role of Automated Writing Evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27 1, 1–18. DOI logoGoogle Scholar
Mantello, M. (1997). Error correction in the L2 classroom. Canadian Modern Language Review, 54 (1), 127–132. DOI logoGoogle Scholar
Milewski, G. (2022). Ahead of the curve: How pegTM has led automated scoring for years. Educational Records Bureau. [URL]
Page, E. B. (2003). Project Essay Grade: PEG. In M. D. Shermis & J. Burstein (Eds.), Automated essay scoring: A cross-disciplinary perspective (pp. 43–54). Lawrence Erlbaum Associates Publishers.Google Scholar
Ranalli, J. (2018). Automated written corrective feedback: How well can students make use of it? Computer Assisted Language Learning, 31 (7), 653–674. DOI logoGoogle Scholar
Schmidt, R. W. (1990). The role of consciousness in second language learning. Applied Linguistics, 11 (2), 129–158. DOI logoGoogle Scholar
Truscott, J. (1999). What’s wrong with oral grammar correction? Canadian Modern Language Review, 55 (4), 453–454. DOI logoGoogle Scholar
Warschauer, M., & Ware, P. (2006). Automated writing evaluation: Defining the classroom research agenda. Language Teaching Research, 10 (2), 157–180. DOI logoGoogle Scholar
Wharton School. (2023, July 31). Practical AI for instructors and students part 1: Introduction to AI for teachers and students [Video]. YouTube. [URL]
Wucuozi. (2021, March 23). Checking 12 categories of errors. [URL]
Xiezuomao. (2024). Introduction. [URL]
Yan, D. (2023). Impact of CHATGPT on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies, 28 (11), 13943–13967. DOI logoGoogle Scholar
Zhou, J., Li, C., Liu, H., Bao, Z., Xu, G., & Li, L. (2018). Chinese grammatical error correction using statistical and neural models. In Natural Language Processing and Chinese Computing, 7th CCF International Conference, NLPCC 2018, Hohhot, China, August 26–30, 2018 Proceedings, Part II (pp. 117–128). DOI logoGoogle Scholar