Chapter 10
L2 speaking and its internal correlates
A meta-analysis
The current meta-analysis examines relationships between second-language (L2) speaking (assessed using global ratings) and its internal features, such as fluency and accuracy (assessed using analytic ratings or measures) that were derived by analyzing speaking performance during the same speaking tasks. A synthesis of 39 studies (284 correlations) suggests that internal features are strongly correlated to L2 speaking in general (r = .649) and that the strength of the correlations varies according to oral features (e.g., r = .713 to .888 for fluency, delivery, grammar, vocabulary, pronunciation, and content). The results highlight the relative importance of various internal features in L2 speaking and suggest areas in need of further research.
Article outline
- 1.Introduction
- 2.Literature review
- 2.1Fluency
- 2.2Accuracy, grammar, and vocabulary
- 2.3Grammatical complexity and lexical complexity
- 2.4Pronunciation, comprehensibility, delivery, content, and coherence
- 2.5Measuring L2 speaking and its internal features
- 2.6Relative strengths of relationships between L2 speaking and internal features
- 3.Current study and research questions
- 4.Method
- 4.1Literature search
- 4.2Study inclusion criteria
- 4.3Coding
- 4.5Analyses
- 5.Results
- 6.Discussion
- 6.1Fluency
- Accuracy, grammar, and vocabulary
- 6.2Grammatical complexity and lexical complexity
- 6.3Pronunciation, comprehensibility, delivery, content, and coherence
- 6.4Relative strengths of relationships between L2 speaking and internal features
- 7.Conclusion
-
Notes
-
References
-
Appendix
References
Adams, M. L.
(
1980)
Five coocurring factors in speaking proficiency. In
J. R. Firth (Ed.),
Measuring spoken language proficiency (pp. 1–6). Georgetown University Press.
Bachman, L., & Palmer, A.
(
2010)
Language assessment in practice. Oxford University Press.
Borenstein, M.
(
2019)
Common mistakes in meta-analysis and how to avoid them. Biostat.
Cao, H.
(
2014)
Disentangling fluency, comprehensibility and coherence: Toward a better understanding of oral proficiency profiles (Doctoral dissertation). Retrieved on 12 January 2022 from
[URL]
Clark, J. L. D., & Swinton, S. S.
(
1980)
The Test of Spoken English as a measure of communicative ability in English-medium instructional settings (
TOEFL Research Report, RR 80–33).
Cucchiarini, C., Strik, H., & Boves, L.
(
2000)
Quantitative assessment of second language learners’ fluency by means of automatic speech recognition technology.
Journal of the Acoustical Society of America,
107
(2), 989–999.
De Jong, N.
(
2018)
Fluency in second language testing: Insights from different disciplines.
Language Assessment Quarterly,
15
(3), 237–254.
De Jong, N. H., Steinel, M. P., Florijn, A., Schoonen, R., & Hulstijn, J. H.
(
2013)
Linguistic skills and speaking fluency in a second language.
Applied Psycholinguistics,
34
(5), 893–916.
Ellis, R., & Yuan, F.
(
2004)
The effects of planning on fluency, complexity and accuracy in second language narrative writing.
Studies in Second Language Acquisition, 26(1), 59–84.
Farnsworth, T. L.
(
2013)
An investigation into the validity of the TOEFL iBT Speaking Test for international teaching assistant certification.
Language Assessment Quarterly,
10
(3), 274–291.
Fisher, Z., & Tipton, E.
(
2015)
Robust variance meta-regression (
Version 2.0) [Software]. Retrieved on 12 January 2022 from
[URL]
Foster, P.
(
2020)
Oral fluency in a second language: A research agenda for the next ten years.
Language Teaching,
53
(4), 446–461.
Foster, P., Tonkyn, A., & Wigglesworth, G.
(
2000)
Measuring spoken language: A unit for all reasons.
Applied Linguistics,
21
(3), 354–375.
Foster, P., & Wigglesworth, G.
(
2016)
Capturing accuracy in second language performance: The case for a weighted clause ratio.
Annual Review of Applied Linguistics,
36
, 98–116.
Fulcher, G.
(
2003)
Testing second language speaking. Pearson Education.
Gan, Z.
(
2008)
Extroversion and group oral performance: A mixed quantitative and discourse analysis approach.
Prospect,
23
(3), 24–42.
Gan, Z.
(
2012)
Complexity measures, task type, and analytic evaluations of speaking proficiency in a school-based assessment context.
Language Assessment Quarterly,
9
(2), 133–151.
Housen, A., Kuiken, F., & Vedder, I.
(
2012)
Complexity, accuracy and fluency. In
A. Housen,
F. Kuiken, &
I. Vedder (Eds.),
Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp. 1–20). John Benjamins.
Iwashita, N., Brown, A., McNamara, T., & O’Hagan, S.
(
2008)
Assessed levels of second language speaking proficiency: How distinct? Applied Linguistics,
29
(1), 24–49.
Jak, S.
(
2015)
Meta-analytic structural equation modelling. Springer.
Jin, T., & Mak, B.
(
2013)
Distinguishing features in scoring L2 Chinese speaking performance: How do they work? Language Testing,
30
(1), 23–47.
Koizumi, R.
(
2005b)
Speaking performance measures of fluency, accuracy, syntactic complexity, and lexical complexity.
JABAET (Japan-Britain Association for English Teaching) Journal,
9
, 5–33.
Koizumi, R.
(
2013)
Vocabulary and speaking. In
C. A. Chapelle (Ed.),
The encyclopedia of applied linguistics [online edition]. John Wiley and Sons.
Koizumi, R., & In’nami, Y.
(
2012)
Effects of text length on lexical diversity measures: Using short texts with less than 200 tokens.
System,
40
, 554–564.
Koizumi, R., & Kurizaki, I.
(
2002)
Nihonjin chugakusei no monorogu niokeru supikingu no tokucho [Speaking characteristics of monologues given by Japanese junior high school students].
Bulletin of the Kanto-Koshin-Etsu English Language Education Society,
16
, 17–28.
Koizumi, R., & Yamanouchi, I.
(
2003)
Nihonjin chugakusei no supikingu noryoku no hattatsu [Development in speaking ability among Japanese junior high school students: Using self-introduction task].
Bulletin of the Kanto-Koshin-Etsu English Language Education Society,
17
, 33–44.
Kormos, J.
(
2006)
Speech production and second language acquisition. Lawrence Erlbaum Associates.
Kormos, J., & Dénes, M.
(
2004)
Exploring measures and perceptions of fluency in the speech of second language learners.
System,
32
(2), 145–164.
Kyle, K., Crossley, S. A., & Jarvis, S.
(
2021)
Assessing the validity of lexical diversity indices using direct judgements.
Language Assessment Quarterly,
18
(2), 154–170.
Levelt, W. J. M.
(
1989)
Speaking: From intention to articulation. The MIT Press.
Li, S.
(
2016)
The construct validity of language aptitude: A meta-analysis.
Studies in Second Language Acquisition,
38
(4), 801–842.
Lu, X.
(
2012)
The relationship of lexical richness to the quality of ESL learners’ oral narratives.
The Modern Language Journal,
96
(2), 190–208.
Luoma, S.
(
2004)
Assessing speaking. Cambridge University Press.
Malvern, D., & Richards, B.
(
2002)
Investigating accommodation in language proficiency interviews using a new measure of lexical diversity.
Language Testing,
19
(1), 85–104.
Milton, J., Wade, J., & Hopkins, N.
(
2010)
Aural word recognition and oral competence in a foreign language. In
R. Chacón-Beltrán,
C. Abello-Contesse, &
M. Torreblanca-López (Eds.),
Further insights into non-native vocabulary teaching and learning (pp. 83–98). Multilingual Matters.
Norris, J. M., & Ortega, L.
(
2009)
Towards an organic approach to investigating CAF in instructed SLA: The case of complexity.
Applied Linguistics,
30
(4), 555–578.
Ockey, G. J., Koyama, D., Setoguchi, E., & Sun, A.
(
2015)
The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students.
Language Testing,
32
(1), 39–62.
Orwin, R.
(
1983)
A fail-safe N for effect size in meta-analysis.
Journal of Educational Statistics,
8
(2), 157–159.
Pietilë, P.
(
1999)
L2 speech: Oral proficiency of students of English at university level.
Anglicana Turkuensia,
19
, 1–80.
Plonsky, L., & Oswald, F. L.
(
2014)
How big is “big”? Interpreting effect sizes in L2 research.
Language Learning,
64
(4), 878–912.
Révész, A., Ekiert, M., & Torgersen, E. N.
(
2016)
The effects of complexity, accuracy, and fluency on communicative adequacy in oral task performance.
Applied Linguistics,
37
(6), 828–848.
Rosenthal, R.
(
1979)
The “file drawer problem” and tolerance for null results.
Psychological Bulletin,
86
(3), 638–641.
Saito, K., Ilkan, M., Magne, V., Tran, M., & Suzuki, S.
(
2018)
Acoustic characteristics and learner profiles of low, mid and high-level second language fluency.
Applied Psycholinguistics,
39
(3), 593–617.
Saito, K., Trofimovich, P., & Isaacs, T.
(
2017)
Using listener judgements to investigate linguistic influences on L2 comprehensibility and accentedness: A validation and generalization study.
Applied Linguistics,
38
(4), 439–462.
Saito, K., Webb, S., Trofimovich, P., & Isaacs, T.
(
2016)
Lexical correlates of comprehensibility versus accentedness in second language speech.
Bilingualism: Language and Cognition,
19
(3), 597–609.
Sato, T.
(
2012)
The contribution of test-takers’ speech content to scores on an English oral proficiency test.
Language Testing,
29
(2), 223–241.
Segalowitz, N.
(
2010)
Cognitive bases of second language fluency. Routledge.
Segalowitz, N., & Freed, B. F.
(
2004)
Context, contact, and cognition in oral fluency acquisition: Learning Spanish in at home and study abroad contexts.
Studies in Second Language Acquisition,
26
(2), 173–199.
Skehan, P.
(
2009)
Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis.
Applied Linguistics,
30
(4), 510–532.
Suzuki, S., Kormos, J., & Uchihara, T.
(
2021)
The relationship between utterance and perceived fluency: A meta-analysis of correlational studies.
The Modern Language Journal,
105
(2), 435–463.
Tavakoli, P., Nakatsuhara, F., & Hunter, A.-M.
(
2020)
Aspects of fluency across assessed levels of speaking proficiency.
The Modern Language Journal,
104
(1), 169–191.
Tavakoli, P., & Skehan, P.
Tavakoli, P., & Wright, C.
(
2020)
Second language speech fluency. Cambridge University Press.
Ushigusa, S.
(
2008)
The relationships between oral fluency, multiword units, and proficiency scores (Doctoral dissertation). Retrieved from UMI. (
Order No. 3344157)
von Hippel, P. T.
(
2015)
The heterogeneity statistic I
2 can be biased in small meta-analyses.
BMC Medical Research Methodology,
15
(35), 1–8.
[URL].
Xi, X., & Mollaun, P.
(
2006)
Investigating the utility of analytic scoring for the TOEFL Academic Speaking Test (TAST). (
TOEFL iBT Research Report, RR-06-07).
[URL].
Yan, X., Kim, H. R., & Kim, J. Y.
(
2018)
Complexity, accuracy and fluency (CAF) features of speaking performances on Aptis across different levels on the Common European Framework of Reference (CEFR).
ARAGs Research Reports online. British Council. Retrieved on 12 January from
[URL]
Zenker, F., & Kyle, K.
(
2021)
Investigating minimum text lengths for lexical diversity indices.
Assessing Writing,
47
, 1–15.
Cited by
Cited by 2 other publications
Handley, Zoe L. & Haiping Wang
2024.
What Do the Measures of Utterance Fluency Employed in Automatic Speech Evaluation (ASE) Tell Us About Oral Proficiency?.
Language Assessment Quarterly 21:1
► pp. 3 ff.
Kyle, Kristopher, Hakyung Sung, Masaki Eguchi & Fred Zenker
2024.
Evaluating evidence for the reliability and validity of lexical diversity indices in L2 oral task responses.
Studies in Second Language Acquisition 46:1
► pp. 278 ff.
This list is based on CrossRef data as of 28 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.