Cover not available
Article published In:
Journal of Second Language Studies
Vol. 1:2 (2018) ► pp.231–253

Meta-analysis in second language acquisition research

A critical appraisal

| Curtin University
Meta-analysis has become increasingly popular in second language acquisition research (SLA) and has provided valuable summative information about a number of key areas. There are, however, dangers. This article examines a number of key issues that need to be considered in conducting a meta-analysis – inclusiveness, the heterogeneity of language learners, the definition of the independent and dependent variables, the need to consider alternative explanations of observed effects, the importance of examining the quality of the studies included in the analysis, and the apples and oranges problem. These issues are illustrated in a discussion of number of SLA meta-analyses (e.g. Norris and Ortega, 2000; Plonsky, 2011; Qureshi, 2016; Spada and Tomita, 2010). The article concludes by suggesting a number of factors that need to be considered in deciding whether to conduct a meta-analysis and when carrying one out. I argue the need for systematic reviews but suggest that these can often best present their findings in narrative form rather than statistically. I also suggest that a preliminary narrative account of the findings of s systematic review can indicate whether a meta-analysis is appropriate.
Keywords: SLA, meta-analysis, limitations, narrative accounts
Article outline
  • Introduction
    • Issue 1.Inclusiveness
    • Issue 2.Heterogeneity of language learners
    • Issue 3.Defining the independent and dependent variables
    • Issue 4.Explaining an observed effect
    • Issue 5.Quality of studies
    • Issue 6.Apples and oranges
    • Issue 7.Effects of grouping
  • Is there another way?
  • 3.Conclusion
  • Notes
  • References
Published online: 27 August 2018
DOI logo
https://doi.org/10.1075/jsls.00002.ell
References (49)
References
Berns, M. (1990). ‘Second’ and ‘foreign’ in second language acquisition? Foreign language learning: A sociolinguistic perspectives. In B. VanPatten & J. Lee (Eds), Second language acquisition – foreign language learning (pp. 3–11). Clevedon: Multilingual Matters.Google Scholar
Bialystok, E., & Hakuta, K. (1999). Confounded age: Linguistic and cognitive factors in age differences for second language acquisition. In D. Birdsong (Ed.), Second language acquisition and the critical period hypothesis. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Birdsong, D. (2006). Age and second language acquisition and processing: A selective overview. In M. Gullberg & P. Indefrey (Eds.), The cognitive neuroscience of second language acquisition (pp. 9–49). Malden, MA: Blackwell.Google Scholar
Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (2006). Comprehensive meta-analysis (Version 2.2.027) [Computer software]. Englewood, NJ: Biostat.Google Scholar
(2009). Introduction to meta-analysis. Hoboken, NJ: John Wiley and Sons. DOI logoGoogle Scholar
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Cooper, H. (2010). Research synthesis and meta-analysis. Los Angeles, CA: Sage.Google Scholar
DeKeyser, R. (2003). Implicit and explicit learning. In C. Doughty & M. Long (Eds.), The handbook of second language acquisition (pp. 313–348). Malden, MA: Blackwell. DOI logoGoogle Scholar
Ellis, N. (2006). Meta-analysis, human cognition, and language learning. In J. Norris & L. Ortega (Eds.), Synthesizing research on language teaching and learning (pp. 301–322). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Ellis, R. (1991). Grammar teaching – practice or consciousness-raising. In R. Ellis (Ed.), Second language acquisition and second language pedagogy (pp. 232–241). Clevedon: Multilingual Matters.Google Scholar
(1997). SLA research and language teaching. Oxford: Oxford University Press.Google Scholar
(2007). The differential effects of corrective feedback on two grammatical structures. In A. Mackey (Ed.), Conversational interaction and second language acquisition: A series of empirical studies (pp. 339–360). Oxford: Oxford University Press.Google Scholar
(2012). Second language research and language pedagogy. Malden, MA: Wiley Blackwell.Google Scholar
(2015). Understanding second language acquisition (2nd ed.). Oxford: Oxford University Press.Google Scholar
(2015b). Introduction: Complementarity in research syntheses. Applied Linguistics, 36(3), 258–259. DOI logoGoogle Scholar
Eysenck, H. (1984). Meta-analysis: An abuse of research integration. Journal of Special Education, 151, 9–11.Google Scholar
(1994). Systematic reviews: Meta-analysis and its problems. British Medication Journal, 3091, 789–792. DOI logoGoogle Scholar
Glass, G. (1976). Primary, secondary and meta-analysis of Research. Educational Researcher, 51, 3–8. DOI logoGoogle Scholar
Glass, S., & Smith, M. (1979). Meta-analysis on class size and achievement. Educational Evolution and Policy Analysis, 11, 2–16. DOI logoGoogle Scholar
Goo, J., Granena, G., Novella, M., & Yilmaz, M. (2009). Implicit and explicit instruction in L2 learning: Norris and Ortega (2000) revisited. Paper presented at the Second Language Research Forum. East Lansing, MI.
Jeon, E., & Kaya, T. (2006). Effects of L2 instruction on interlanguage pragmatic development: A meta-analysis. In J. Norris & L. Ortega (Eds.), Synthesizing research on language teaching and learning (pp. 165–212). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Johnson, J., & Newport, E. (1989). Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language. Cognitive Psychology, 211, 60–99. DOI logoGoogle Scholar
Khan, S., Kunz, R., Kleijnen, M., & Antes, G. (2003). Five steps to conducing a systematic review. Journal of the Royal Society of Medicine, 961, 118–121. DOI logoGoogle Scholar
Li, S. (2010). The effectiveness of corrective feedback in SLA; A meta-analysis. Language Learning, 601, 309–365. DOI logoGoogle Scholar
Li, S., Shitani, N., & Ellis, R. (2013). Doing meta-analysis in SLA: Practices, choices and standards. Contemporary Foreign Language Studies, 3841, 1–17.Google Scholar
Lipsey, M., & Wilson, D. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.Google Scholar
Long, M. (1988). Maturational constraints on language development. University of Hawai’i Working Papers in ESL, 7(1), 1–53.Google Scholar
(1991). Focus on form: A design feature in language teaching methodology. In K. de Bot, R. Ginsberg, & C. Kramsch (Eds.), Foreign language research in cross-cultural perspective (pp. 39–52). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Lyster, R., & Saito, K. (2010). Oral feedback in SLA: A meta-analysis. Studies in Second Language Acquisition, 321, 265–302. DOI logoGoogle Scholar
Mackey, A., & Goo, J. (2007). Interaction research in SLA: A meta-analysis and research (pp. 407–449). In. A. Mackey (Eds.), Conversational interaction in second language acquisition: A collection of empirical studies (pp. 407–452). Oxford: Oxford University Press.Google Scholar
Norris, J., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 501, 417–528. DOI logoGoogle Scholar
(2006). The value and practice of research synthesis for language learning and teaching. In J. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 1–50). Amsterdam: John Benjamins. DOI logoGoogle Scholar
Oswald, L., & Plonsky, L. (2010). Meta-analysis in second language research: Choices and challenges. Annual Review of Applied Linguistics, 301, 85–110. DOI logoGoogle Scholar
Plonsky, L. (2011). The effectiveness of second language strategy instruction: A meta-analysis. Language Learning, 611, 993–1038. DOI logoGoogle Scholar
Personal website downloaded on 31/7/17 from <[URL]>
Plonsky, L., & Oswald, F. (2012). How to do a meta-analysis. In A. Mackey & S. Gass (Eds.), Research methods in second language acquisition: A practical introduction (pp. 275–295). Malden, MA: Blackwell. DOI logoGoogle Scholar
Qureshi, M. (2016). A meta-analysis: Age and second language grammar acquisition. System, 601, 147–160. DOI logoGoogle Scholar
Ross, S. (1998). Self-assessment in second language testing: A meta-analysis and analysis of experiential factors. Language Testing, 151, 1–20. DOI logoGoogle Scholar
Russo, M. (2007). How to review a meta-analysis. Gastroenterology and Hepatology, 3(8), 637–642.Google Scholar
Shintani, N., Li, S., & Ellis, R. (2013). Comprehension-based versus production-based grammar instruction: A meta-analysis of comparative studies. Language Learning, 63(2), 296–329. DOI logoGoogle Scholar
Shintani, N. (2015). The effectiveness of processing instruction and production-based instruction on L2 grammar acquisition: A meta-analysis. Applied Linguistics, 36(3), 306–325. DOI logoGoogle Scholar
Smith, M., & Glass, G. (1977). Meta-analysis of psychotherapy outcome studies. American Psychology, 321, 752–60. DOI logoGoogle Scholar
Smith, M., Glass, G., & Miller, T. (1980). The benefits of psychotherapy. Baltimore, MD: Johns Hopkins Press.Google Scholar
Spada, N., & Lightbown, P. (2008). Form-focused instruction: Isolated or integrated? TESOL Quarterly, 421, 181–207. DOI logoGoogle Scholar
Spada, N., & Tomita, Y. (2010). Interactions between type of instruction and type of language feature: A meta-analysis. Language Learning, 601, 268–308. DOI logoGoogle Scholar
The International Research Foundation for English language Education. Meta-analyses in language research: A selected bibliography. Downloaded from <[URL]> (31 July, 2017).
Truscott, J. (2007). The effect of error correction on learners’ ability to write accurately. Journal of Second Language Writing, 161, 255–272. DOI logoGoogle Scholar
VanPatten, B. (1996). Input processing and grammar instruction in second language acquisition. Norwood, NJ: Ablex.Google Scholar
Walker, E., Hernandez, A., & Katten, M. (2008). Meta-analysis: Its strength and limitations. Cleveland. Clinic Journal of Medicine, 751, 431–439.Google Scholar
Cited by (4)

Cited by four other publications

Anani Sarab, Mohammad Reza & Mohammad Amini Farsani
2024. Second-order synthesis of meta-analytic studies in applied linguistics (1998–2021). Quality & Quantity 58:2 ► pp. 1517 ff. DOI logo
Yamashita, Taichi
2024. Effectiveness and inclusiveness of locally adapted human-delivered synchronous written corrective feedback for English referential articles. Computer Assisted Language Learning 37:5-6 ► pp. 1074 ff. DOI logo
Yapp, Deborah June, Rick de Graaff & Huub van den Bergh
2021. Improving second language reading comprehension through reading strategies. Journal of Second Language Studies 4:1 ► pp. 154 ff. DOI logo
Bardovi-Harlig, Kathleen & Llorenç Comajoan-Colomé
2020. THE ASPECT HYPOTHESIS AND THE ACQUISITION OF L2 PAST MORPHOLOGY IN THE LAST 20 YEARS. Studies in Second Language Acquisition 42:5 ► pp. 1137 ff. DOI logo

This list is based on CrossRef data as of 15 september 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.