Chapter 5
L2 writing and its internal correlates
A meta-analysis
This study examined the overall average correlation between second/foreign language (L2) writing performance and frequently investigated features of writing (i.e., writing-internal correlates). The correlates of L2 writing performance included objective measures of text features (syntactic complexity, lexical complexity, accuracy, fluency, and cohesion) as well as subjective measures (content, argument, organization, cohesion, coherence, grammar, vocabulary, language use, mechanics, and accuracy). A series of moderator analyses were also carried out for each type of objective measure to examine the effects of participants’ age, L2 proficiency, learning context, first language (L1) and L2 distance, task type, writing scoring method, and some characteristics of objective measures. By doing so, the present study aimed to identify key correlates of L2 writing performance and compare their relative importance. To this end, a total of 103 retrieved studies contributed 1,045 effect sizes based on 15,537 independent participants. The results for objective measures demonstrated that fluency had the strongest mean correlation with L2 writing performance (r = .570), followed by accuracy (r = .477), lexical complexity (r = .295), syntactic complexity (r = .271), and cohesion (r = .198). All subjective measure components had strong mean correlations with L2 writing performance (r = .668 to .927), but content and language use features had the strongest effects and cohesion and coherence features showed the least effects. Participants’ age, learning context, L1–L2 distance, writing scoring method, and some measurement characteristics were found to be significant moderators for certain components. The findings of this study have implications for L2 instruction suggesting that fluency and language accuracy of L2 writing should be promoted across the various developmental stages of L2 learners in various conditions, whereas lexical and syntactic competence should be more focused upon when instructing child/adolescent or low/intermediate L2 writers.
Article outline
- 1.Introduction
- 2.Background to the meta-analysis
- 2.1Review of objective measures
- 2.1.1Syntactic complexity
- 2.1.2Lexical complexity
- 2.1.3Accuracy
- 2.1.4Fluency
- 2.1.5Cohesion
- 2.2Review of subjective measures
- 2.3Review of moderator variables
- 2.3.1Age
- 2.3.2L2 proficiency
- 2.3.3Learning context
- 2.3.4L1–L2 distance
- 2.3.5Task type
- 2.3.6Writing scoring method
- 2.3.7Measurement types of complexity and fluency
- 3.Research questions for the meta-analysis
- 4.Method
- 4.1Literature search and inclusion criteria
- 4.2Acceptable measures of L2 writing performance and internal correlates
- 4.3Coding the primary studies
- 4.4Research synthesis
- 5.Results
- 6.Discussion
- 6.1Objective measures
- 6.1.1Syntactic complexity
- 6.1.2Lexical complexity
- 6.1.3Accuracy
- 6.1.4Fluency
- 6.1.5Cohesion
- 6.2Subjective measures
- 7.Conclusion
-
References
-
Appendix
References
Bacha, N.
(
2001)
Writing evaluation: What can analytic versus holistic essay scoring tell us? System,
29
(3), 371–383.
Barkaoui, K.
(
2010a)
Do ESL essay raters’ evaluation criteria change with experience? A mixed-methods, cross-sectional study.
TESOL Quarterly,
44
(1), 31–57.
Barkaoui, K.
(
2010b)
Explaining ESL essay holistic scores: A multilevel modeling approach.
Language Testing,
27
(4), 515–535.
Berman, R., & Verhoeven, L.
Biber, D., Gray, B., & Poonpon, K.
(
2011)
Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? TESOL Quarterly,
45
(1), 5–35.
Biber, D., Gray, B., & Staples, S.
(
2016)
Predicting patterns of grammatical complexity across language exam task types and proficiency levels.
Applied Linguistics,
37
(5), 639–668.
Bouwer, R., Béguin, A., Sanders, T., & Van den Bergh, H.
(
2015)
Effect of genre on the generalizability of writing scores.
Language Testing,
32
(1), 83–100.
Brodkey, D., & Young, R.
(
1981)
Composition correctness scores.
TESOL Quarterly,
15
(2), 159–167.
Bulté, B., & Housen, A.
(
2014)
Conceptualizing and measuring short-term changes in L2 writing complexity.
Journal of Second Language Writing,
26
, 42–65.
Cooper, H.
(
2010)
Research synthesis and meta-analysis: A step-by-step approach (Vol. 2). Sage.
Crossley, S. A., Kyle, K., Allen, L. K., Guo, L., & McNamara, D. S.
(
2014)
Linguistic microfeatures to predict L2 writing proficiency: A case study in automated writing evaluation.
The Journal of Writing Assessment,
7
(1), 1–34.
Crossley, S. A., Kyle, K., & McNamara, D. S.
(
2016a)
The development and use of cohesive devices in L2 writing and their relations to judgments of essay quality.
Journal of Second Language Writing,
32
, 1–16.
Crossley, S. A., Kyle, K., & McNamara, D. S.
(
2016b)
The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion.
Behavior Research Methods,
48
(4), 1227–1237.
Crossley, S. A., & McNamara, D. S.
(
2012)
Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication.
Journal of Research in Reading,
35
(2), 115–135.
Crossley, S. A., Salsbury, T., & McNamara, D. S.
(
2012)
Predicting the proficiency level of language learners using lexical indices.
Language Testing,
29
(2), 243–263.
Crossley, S. A., Salsbury, T., & McNamara, D. S.
(
2015)
Assessing lexical proficiency using analytic ratings: A case for collocation accuracy.
Applied Linguistics,
36
(5), 570–590.
De Jong, N. H., Steinel, M. P., Florijn, A. F., Schoonen, R., & Hulstijn, J. H.
(
2012)
Facets of speaking proficiency.
Studies in Second Language Acquisition,
34
(1), 5–34.
Engber, C. A.
(
1992)
A study of lexis and the relationship to quality in written texts of second language learners of English (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Global. (303986307)
Engber, C. A.
(
1995)
The relationship of lexical proficiency to the quality of ESL compositions.
Journal of Second Language Writing,
4
(2), 139–155.
Evola, J., Mamer, E., & Lentz, B.
(
1980)
Discrete-point versus global scoring for cohesive devices. In
J. W. Oiler &
K. Perkins (Eds.),
Research and language testing (pp. 177–181). Newbury House.
Flahive, D. E., & Snow, B. G.
(
1980)
Measures of syntactic complexity in evaluating ESL compositions. In
J. W. Oiler &
K. Perkins (Eds.),
Research and language testing (pp. 171–176). Newbury House.
Grabe, W., & Kaplan, R. B.
(
1996)
Theory and practice of writing: An applied linguistic perspective. Longman.
Guo, L., Crossley, S. A., & McNamara, D. S.
(
2013)
Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study.
Assessing Writing,
18
(3), 218–238.
Halliday, M. A. K., & Hasan, R.
(
1976)
Cohesion in English. Longman.
Hinkel, E.
(
2002)
Second language writers’ text: Linguistic and rhetorical features. Routledge.
Housen, A., Kuiken, F., & Vedder, I.
Hunt, K. W.
(
1965)
Grammatical structures written at three grade levels (Vol. 3). National Council of Teachers of English.
Jacobs, H. L., Zinkgraf, S. A., Wormuth, D. R., Hartfiel, V. F., & Hughey, J. B.
(
1981)
Testing ESL composition: A practical approach. Newbury House.
Jak, S.
(
2015)
Meta-analytic structural equation modelling. Springer.
Jarvis, S.
(
2002)
Short texts, best-fitting curves and new measures of lexical diversity.
Language Testing,
19
(1), 57–84.
Jarvis, S.
(
2013)
Capturing the diversity in lexical diversity.
Language Learning,
63
(S1), 87–106.
Jeon, E.-H., & Yamashita, Y.
(
2014)
L2 reading comprehension and its correlates: A meta-analysis.
Language Learning,
64
(1), 160–212.
Kaczmarek, C. M.
(
1980)
Scoring and rating essay tasks. In
J. W. Oiler &
K. Perkins (Eds.),
Research and language testing (pp. 151–159). Newbury House.
Kim, M., & Crossley, S. A.
(
2018)
Modeling second language writing quality: A structural equation investigation of lexical, syntactic, and cohesive features in source-based and independent writing.
Assessing Writing,
37
, 39–56.
Kim, M., Crossley, S. A. & Kyle, K.
(
2018)
Lexical sophistication as a multidimensional phenomenon: Relations to second language lexical proficiency, development, and writing quality.
The Modern Language Journal,
102
(1), 120–141.
Kim, Y. A., & Na, Y. H.
(
2009)
Cohesive devices and quality of argumentative writing produced by Korean EFL learners.
Studies in English Education,
14
(2), 1–29.
Koda, K.
(
2005)
Insights into second language reading: A cross-linguistic approach. Cambridge University Press.
Koda, K., & Yamashita, J.
(
2019)
Reading to learn in a foreign language: An integrated approach to foreign language instruction and assessment. Routledge.
Kojima, M.
(
2020)
A study synthesis on the relationship between second language writing performance and text features: Focusing on text-based measures and study features.
Learner Corpus Studies in Asia and the World,
5
, 1–24.
Kojima, M., & Kaneta, T.
(
2020)
Raitingu hyoka to gengoteki shihyo no kankei: Meta bunseki ni yoru kenkyu seika no togo [The relationship between writing performance and linguistic indices: A meta-analysis]. In
Y. Ishii &
Y. Kondo (Eds.),
Eigo Kyoiku ni okeru jido saiten: Genjo to kadai [
Automated scoring in English language education: Its current situation and issues] (pp. 33–72). Hituzi Shobo.
Kroll, B.
(
1982)
Levels of error in ESL composition (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Global. (303230742)
Kroll, B.
(
1990)
What does time buy? ESL student performance on home versus class compositions. In
B. Kroll (Ed.),
Second language writing: Research insights for the classroom (pp. 140–154). Cambridge University Press.
Kyle, K., & Crossley, S.
(
2016)
The relationship between lexical sophistication and independent and source-based writing.
Journal of Second Language Writing,
34
, 12–24.
Kyle, K., & Crossley, S.
(
2017)
Assessing syntactic sophistication in L2 writing: A usage-based approach.
Language Testing,
34
(4), 513–535.
Kyle, K., & Crossley, S. A.
(
2018)
Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices.
The Modern Language Journal,
102
(2), 333–349.
Larsen-Freeman, D.
(
2009)
Adjusting expectations: The study of complexity, accuracy, and fluency in second language acquisition.
Applied Linguistics,
30
(4), 579–589.
Laufer, B., & Nation, P.
(
1995)
Vocabulary size and use: Lexical richness in L2 written production.
Applied Linguistics,
16
(3), 307–322.
Lee, Y.-W., Gentile, C., & Kantor, R.
(
2008)
Analytical scoring of TOEFL CBT essays: Scores by humans and e-rater,
TOEFL Research Report,
81
,
ETS RR-08-01. Educational Testing Service.
Lee, Y.-W., Gentile, C., & Kantor, R.
(
2010)
Toward automated multi-trait scoring of essays: Investigating links among holistic, analytic, and text feature scores.
Applied Linguistics,
31
(3), 391–417.
Leki, I., Cumming, A., & Silva, T.
(
2008)
A synthesis of research on second language writing in English. Routledge.
Linnarud, M.
(
1986)
Lexis in composition: A performance analysis of Swedish learners’ written English. CWK Gleerup.
Liu, M., & Braine, G.
(
2005)
Cohesive features in argumentative writing produced by Chinese undergraduates.
System,
33
(4), 623–636.
Lu, X., & Ai, H.
(
2015)
Syntactic complexity in college-level English writing: Differences among writers with diverse L1 backgrounds.
Journal of Second Language Writing,
29
, 16–27.
Luk, Z. P. S., & Shirai, Y.
(
2009)
Is the acquisition order of grammatical morphemes impervious to L1 knowledge? Evidence from the acquisition of plural -s, articles, and possessive’s.
Language Learning,
59
(4), 721–754.
Malvern, D., & Richards, B.
(
2002)
Investigating accommodation in language proficiency interviews using a new measure of lexical diversity.
Language Testing,
19
(1), 85–104.
Martínez, A. C. L.
(
2018)
Analysis of syntactic complexity in secondary education EFL writers at different proficiency levels.
Assessing Writing,
35
, 1–11.
McCutchen, D., & Perfetti, C. A.
(
1982)
Coherence and connectedness in the development of discourse production.
Text-Interdisciplinary Journal for the Study of Discourse,
2
(1–3), 113–140.
Meara, P., & Bell, H.
(
2001)
P_Lex: A simple and effective way of describing the lexical characteristics of short L2 texts.
Prospect,
16
(3), 5–19.
Murakami, A., & Alexopoulou, T.
(
2016)
L1 influence on the acquisition order of English grammatical morphemes: A learner corpus study.
Studies in Second Language Acquisition,
38
(3), 365–401.
Norris, J. M., & Ortega, L.
(
2000)
Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis.
Language Learning,
50
(3), 417–528.
Norris, J. M., & Ortega, L.
(
2009)
Towards an organic approach to investigating CAF in instructed SLA: The case of complexity.
Applied Linguistics,
30
(4), 555–578.
Oh, E., Lee, C. M., & Moon, Y. I.
(
2015)
The contributions of planning, L2 linguistic knowledge and individual differences to L2 writing.
The Journal of Asia TEFL,
12
(2), 45–85.
Oh, S.
(
2006)
Investigating the relationship between fluency measures and second language writing placement test decisions (Unpublished master’s thesis). University of Hawai’i. Retrieved from
[URL]
Ortega, L.
(
2003)
Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing.
Applied Linguistics,
24
(4), 492–518.
Ortega, L.
(
2015)
Syntactic complexity in L2 writing: Progress and expansion.
Journal of Second Language Writing,
29
, 82–94.
Pallotti, G.
(
2009)
CAF: Defining, refining and differentiating constructs.
Applied Linguistics,
30
(4), 590–601.
Park, S. K.
(
2013)
Lexical analysis of Korean university students’ narrative and argumentative essays.
English Teaching (
영어교육),
68
(3), 131–157.
Perkins, K.
(
1980)
Using objective methods of attained writing proficiency to discriminate among holistic evaluations.
TESOL Quarterly,
14
(1), 61–69.
Perkins, K.
(
1983)
On the use of composition scoring techniques, objective measures, and objective tests to evaluate ESL writing ability.
TESOL Quarterly,
17
(4), 651–671.
Plonsky, L., & Oswald, F. L.
(
2014)
How big is “big”? Interpreting effect sizes in L2 research.
Language Learning,
64
(4), 878–912.
Read, J.
(
2000)
Assessing vocabulary. Cambridge University Press.
Roca de Larios, J., Manchón, R., Murphy, L., & Marín, J.
(
2008)
The foreign language writer’s strategic behaviour in the allocation of time to writing processes.
Journal of Second Language Writing,
17
(1), 30–47.
Rosenthal, R.
(
1979)
The “file drawer problem” and tolerance for null results.
Psychological Bulletin,
86
, 638–641.
Sasaki, M.
(
2009)
Changes in English as a foreign language students’ writing over 3.5 years: A sociocognitive account. In
R. M. Manchón (Ed.),
Writing in foreign language contexts: Learning, teaching, and research (pp. 49–76). Multilingual Matters.
Schoonen, R., Van Gelderen, A., De Glopper, K., Hulstijn, J., Simis, A., Snellings, P., & Stevenson, M.
(
2003)
First language and second language writing: The role of linguistic knowledge, speed of processing, and metacognitive knowledge.
Language Learning,
53
(1), 165–202.
Schoonen, R., Van Gelderen, A., Stoel, R. D., Hulstijn, J., & De Glopper, K.
(
2011)
Modeling the development of L1 and EFL writing proficiency of secondary school students.
Language Learning,
61
(1), 31–79.
Skehan, P.
(
1998)
A cognitive approach to language learning. Oxford University Press.
Skehan, P.
(
2009)
Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis.
Applied Linguistics,
30
(4), 510–532.
Stoller, F. L.
(
2004)
Content-based instruction: Perspectives on curriculum planning.
Annual Review of Applied Linguistics,
24
, 261–283.
Thewissen, J.
(
2013)
Capturing L2 accuracy developmental patterns: Insights from an error-tagged EFL learner corpus.
The Modern Language Journal,
97
(S1), 77–101.
Verspoor, M., Schmid, M. S., & Xu, X.
(
2012)
A dynamic usage based perspective on L2 writing.
Journal of Second Language Writing,
21
(3), 239–263.
Viechtbauer, W.
(
2010)
Conducting meta-analyses in R with the metafor package.
Journal of Statistical Software,
36
(3), 1–48.
Weigle, S. C.
(
2002)
Assessing Writing. Cambridge University Press.
White, E. M.
(
1984)
Holisticism.
College Composition and Communication,
35
(4), 400–409.
Wolfe-Quintero, Y., Inagaki, S., & Kim, H.-Y.
(
1998)
Second language development in writing: Measures of fluency, accuracy, & complexity. University of Hawai’i Press.
Yang, W., Lu, X., & Weigle, S. C.
(
2015)
Different topics, different discourse: Relationships among writing topic, measures of syntactic complexity, and judgments of writing quality.
Journal of Second Language Writing,
28
, 53–67.
Yde, P., & Spoelders, M.
(
1985)
Text cohesion: An exploratory study with beginning writers.
Applied Psycholinguistics,
6
(4), 407–415.
Yu, G.
(
2010)
Lexical diversity in writing and speaking task performances.
Applied Linguistics,
31
(2), 236–259.
Yun, Y.
(
2005)
Factors explaining EFL learners’ performance in a timed essay writing test: A structural equation modeling approach (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Global. (3199191)
Cited by
Cited by 2 other publications
Mavrou, Irini & Javier Chao
2023.
What Does Linguistic Distance Predict When It Comes to L2 Writing of Adult Immigrant Learners of Spanish?.
Written Communication 40:3
► pp. 943 ff.
Monteiro, Kátia, Scott Crossley, Robert-Mihai Botarleanu & Mihai Dascălu
2023.
L2 and L1 semantic context indices as automated measures of lexical sophistication.
Language Testing 40:3
► pp. 576 ff.
This list is based on CrossRef data as of 28 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.