Article published in:
Journal of Second Language Studies
Vol. 1:2 (2018) ► pp. 231–253

Meta-analysis in second language acquisition research

A critical appraisal

| Curtin University
Meta-analysis has become increasingly popular in second language acquisition research (SLA) and has provided valuable summative information about a number of key areas. There are, however, dangers. This article examines a number of key issues that need to be considered in conducting a meta-analysis – inclusiveness, the heterogeneity of language learners, the definition of the independent and dependent variables, the need to consider alternative explanations of observed effects, the importance of examining the quality of the studies included in the analysis, and the apples and oranges problem. These issues are illustrated in a discussion of number of SLA meta-analyses (e.g. Norris and Ortega, 2000; Plonsky, 2011; Qureshi, 2016; Spada and Tomita, 2010). The article concludes by suggesting a number of factors that need to be considered in deciding whether to conduct a meta-analysis and when carrying one out. I argue the need for systematic reviews but suggest that these can often best present their findings in narrative form rather than statistically. I also suggest that a preliminary narrative account of the findings of s systematic review can indicate whether a meta-analysis is appropriate.
Keywords: SLA, meta-analysis, limitations, narrative accounts
Article outline
  • Introduction
    • Issue 1.Inclusiveness
    • Issue 2.Heterogeneity of language learners
    • Issue 3.Defining the independent and dependent variables
    • Issue 4.Explaining an observed effect
    • Issue 5.Quality of studies
    • Issue 6.Apples and oranges
    • Issue 7.Effects of grouping
  • Is there another way?
  • 3.Conclusion
  • Notes
  • References
Published online: 27 August 2018
https://doi.org/10.1075/jsls.00002.ell
References
Berns, M.
(1990) ‘Second’ and ‘foreign’ in second language acquisition? Foreign language learning: A sociolinguistic perspectives. In B. VanPatten & J. Lee (Eds), Second language acquisition – foreign language learning (pp. 3–11). Clevedon: Multilingual Matters.Google Scholar
Bialystok, E., & Hakuta, K.
(1999) Confounded age: Linguistic and cognitive factors in age differences for second language acquisition. In D. Birdsong (Ed.), Second language acquisition and the critical period hypothesis. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Birdsong, D.
(2006) Age and second language acquisition and processing: A selective overview. In M. Gullberg & P. Indefrey (Eds.), The cognitive neuroscience of second language acquisition (pp. 9–49). Malden, MA: Blackwell.Google Scholar
Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H.
(2006) Comprehensive meta-analysis (Version 2.2.027) [Computer software]. Englewood, NJ: Biostat.Google Scholar
(2009) Introduction to meta-analysis. Hoboken, NJ: John Wiley and Sons. CrossrefGoogle Scholar
Cohen, J.
(1988) Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Cooper, H.
(2010) Research synthesis and meta-analysis. Los Angeles, CA: Sage.Google Scholar
DeKeyser, R.
(2003) Implicit and explicit learning. In C. Doughty & M. Long (Eds.), The handbook of second language acquisition (pp. 313–348). Malden, MA: Blackwell. CrossrefGoogle Scholar
Ellis, N.
(2006) Meta-analysis, human cognition, and language learning. In J. Norris & L. Ortega (Eds.), Synthesizing research on language teaching and learning (pp. 301–322). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Ellis, R.
(1991) Grammar teaching – practice or consciousness-raising. In R. Ellis (Ed.), Second language acquisition and second language pedagogy (pp. 232–241). Clevedon: Multilingual Matters.Google Scholar
(1997) SLA research and language teaching. Oxford: Oxford University Press.Google Scholar
(2007) The differential effects of corrective feedback on two grammatical structures. In A. Mackey (Ed.), Conversational interaction and second language acquisition: A series of empirical studies (pp. 339–360). Oxford: Oxford University Press.Google Scholar
(2012) Second language research and language pedagogy. Malden, MA: Wiley Blackwell.Google Scholar
(2015) Understanding second language acquisition (2nd ed.). Oxford: Oxford University Press.Google Scholar
(2015b) Introduction: Complementarity in research syntheses. Applied Linguistics, 36(3), 258–259. CrossrefGoogle Scholar
Eysenck, H.
(1984) Meta-analysis: An abuse of research integration. Journal of Special Education, 151, 9–11.Google Scholar
(1994) Systematic reviews: Meta-analysis and its problems. British Medication Journal, 3091, 789–792. CrossrefGoogle Scholar
Glass, G.
(1976) Primary, secondary and meta-analysis of Research. Educational Researcher, 51, 3–8. CrossrefGoogle Scholar
Glass, S., & Smith, M.
(1979) Meta-analysis on class size and achievement. Educational Evolution and Policy Analysis, 11, 2–16. CrossrefGoogle Scholar
Goo, J., Granena, G., Novella, M., & Yilmaz, M.
(2009) Implicit and explicit instruction in L2 learning: Norris and Ortega (2000) revisited. Paper presented at the Second Language Research Forum. East Lansing, MI.
Jeon, E., & Kaya, T.
(2006) Effects of L2 instruction on interlanguage pragmatic development: A meta-analysis. In J. Norris & L. Ortega (Eds.), Synthesizing research on language teaching and learning (pp. 165–212). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Johnson, J., & Newport, E.
(1989) Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language. Cognitive Psychology, 211, 60–99. CrossrefGoogle Scholar
Khan, S., Kunz, R., Kleijnen, M., & Antes, G.
(2003) Five steps to conducing a systematic review. Journal of the Royal Society of Medicine, 961, 118–121. CrossrefGoogle Scholar
Li, S.
(2010) The effectiveness of corrective feedback in SLA; A meta-analysis. Language Learning, 601, 309–365. CrossrefGoogle Scholar
Li, S., Shitani, N., & Ellis, R.
(2013) Doing meta-analysis in SLA: Practices, choices and standards. Contemporary Foreign Language Studies, 3841, 1–17.Google Scholar
Lipsey, M., & Wilson, D.
(2001) Practical meta-analysis. Thousand Oaks, CA: Sage.Google Scholar
Long, M.
(1988) Maturational constraints on language development. University of Hawai’i Working Papers in ESL, 7(1), 1–53.Google Scholar
(1991) Focus on form: A design feature in language teaching methodology. In K. de Bot, R. Ginsberg, & C. Kramsch (Eds.), Foreign language research in cross-cultural perspective (pp. 39–52). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Lyster, R., & Saito, K.
(2010) Oral feedback in SLA: A meta-analysis. Studies in Second Language Acquisition, 321, 265–302. CrossrefGoogle Scholar
Mackey, A., & Goo, J.
(2007) Interaction research in SLA: A meta-analysis and research (pp. 407–449). In. A. Mackey (Eds.), Conversational interaction in second language acquisition: A collection of empirical studies (pp. 407–452). Oxford: Oxford University Press.Google Scholar
Norris, J., & Ortega, L.
(2000) Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 501, 417–528. CrossrefGoogle Scholar
(2006) The value and practice of research synthesis for language learning and teaching. In J. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 1–50). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Oswald, L., & Plonsky, L.
(2010) Meta-analysis in second language research: Choices and challenges. Annual Review of Applied Linguistics, 301, 85–110. CrossrefGoogle Scholar
Plonsky, L.
(2011) The effectiveness of second language strategy instruction: A meta-analysis. Language Learning, 611, 993–1038. CrossrefGoogle Scholar
Personal website downloaded on 31/7/17 from https://​lukeplonsky​.wordpress​.com​/bibliographies​/meta​-analysis/
Plonsky, L., & Oswald, F.
(2012) How to do a meta-analysis. In A. Mackey & S. Gass (Eds.), Research methods in second language acquisition: A practical introduction (pp. 275–295). Malden, MA: Blackwell. CrossrefGoogle Scholar
Qureshi, M.
(2016) A meta-analysis: Age and second language grammar acquisition. System, 601, 147–160. CrossrefGoogle Scholar
Ross, S.
(1998) Self-assessment in second language testing: A meta-analysis and analysis of experiential factors. Language Testing, 151, 1–20. CrossrefGoogle Scholar
Russo, M.
(2007) How to review a meta-analysis. Gastroenterology and Hepatology, 3(8), 637–642.Google Scholar
Shintani, N., Li, S., & Ellis, R.
(2013) Comprehension-based versus production-based grammar instruction: A meta-analysis of comparative studies. Language Learning, 63(2), 296–329. CrossrefGoogle Scholar
Shintani, N.
(2015) The effectiveness of processing instruction and production-based instruction on L2 grammar acquisition: A meta-analysis. Applied Linguistics, 36(3), 306–325. CrossrefGoogle Scholar
Smith, M., & Glass, G.
(1977) Meta-analysis of psychotherapy outcome studies. American Psychology, 321, 752–60. CrossrefGoogle Scholar
Smith, M., Glass, G., & Miller, T.
(1980) The benefits of psychotherapy. Baltimore, MD: Johns Hopkins Press.Google Scholar
Spada, N., & Lightbown, P.
(2008) Form-focused instruction: Isolated or integrated? TESOL Quarterly, 421, 181–207. CrossrefGoogle Scholar
Spada, N., & Tomita, Y.
(2010) Interactions between type of instruction and type of language feature: A meta-analysis. Language Learning, 601, 268–308. CrossrefGoogle Scholar
Truscott, J.
(2007) The effect of error correction on learners’ ability to write accurately. Journal of Second Language Writing, 161, 255–272. CrossrefGoogle Scholar
VanPatten, B.
(1996) Input processing and grammar instruction in second language acquisition. Norwood, NJ: Ablex.Google Scholar
Walker, E., Hernandez, A., & Katten, M.
(2008) Meta-analysis: Its strength and limitations. Cleveland. Clinic Journal of Medicine, 751, 431–439.Google Scholar
Cited by

Cited by 2 other publications

Bardovi-Harlig, Kathleen & Llorenç Comajoan-Colomé
2020. THE ASPECT HYPOTHESIS AND THE ACQUISITION OF L2 PAST MORPHOLOGY IN THE LAST 20 YEARS. Studies in Second Language Acquisition 42:5 ► pp. 1137 ff. Crossref logo
Yapp, Deborah June, Rick de Graaff & Huub van den Bergh
2021. Improving second language reading comprehension through reading strategies. Journal of Second Language Studies 4:1 ► pp. 154 ff. Crossref logo

This list is based on CrossRef data as of 23 april 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.