Cover not available
Part of
Instructed Second Language Acquisition Research Methods
Edited by Laura Gurzynski-Weiss and YouJin Kim
[Research Methods in Applied Linguistics 3] 2022
► pp. 103–122

Chapter 5
Replication research in instructed SLA

| Pennsylvania State University

Replication is a research methodology designed to verify, consolidate, and advance knowledge and understanding within empirical fields of study. By repeating a study’s methodology (with or without change), a replication aims to better understand the nature and generalizability of a previous study’s findings. This chapter introduces readers to the replication research process, beginning with a description of what replication research is, what the most common types of replication research are, and why carrying out replication is important. Close attention is paid to the types of research questions that replication studies are designed to investigate. This is followed by an overview of replication in the field of ISLA, with links to studies and resources. In addition, specific guidelines are provided for carrying out and reporting replication studies. Recommendations for future replications in ISLA are suggested as well as ways in which researchers can integrate replication into future programs of research.

Keywords: replication, ISLA, methodology, design
Article outline
  • 1.What is replication research and why is it important?
  • 2.Research questions in replication research
  • 3.Common options for replication research in instructed SLA
  • 4.Troubleshooting replication research in ISLA
    • 4.1Considerations for variable modification
    • 4.2What statistical options are available in replication studies?
    • 4.3Interpreting a replication’s findings
  • 5.Advice for replication researchers
    • 5.1Carry out a close or approximate replication study
    • 5.2A replication needs a motivation
    • 5.3Comparison
    • 5.4Use the label “replication” in the title and abstract
  • 6.Conclusions
  • 7.Further reading
  • References
Published online: 8 December 2022
DOI logo
https://doi.org/10.1075/rmal.3.05mcm
References
Allen, M., & Preiss, R.
(1993) Replication and meta-analysis: A necessary connection. Journal of Social Behavior and Personality, 8 (6), 9–20.Google Scholar
Andringa, S., & Curcic, M.
(2015) How explicit knowledge affects online L2 processing: Evidence from differential object marking acquisition. Studies in Second Language Acquisition, 37 (2), 237–268. DOI logoGoogle Scholar
Bailey, N., Madden, C., & Krashen, S. D.
(1974) Is there a “natural sequence” in adult second language learning? Language Learning, 24 (2), 235–243. DOI logoGoogle Scholar
Booth, P.
(2013) Vocabulary knowledge in relation to memory and analysis: An approximate replication of Milton’s (2007) study on lexical profiles and learning style. Language Teaching, 46 (3), 335–354. DOI logoGoogle Scholar
Casanave, C. P.
(2012) Heading in the wrong direction? A response to Porte and Richards. Journal of Second Language Writing, 21 (3), 296–297. DOI logoGoogle Scholar
Crossley, S. A., & McNamara, D. S.
(2008) Assessing L2 reading texts at the intermediate level: An approximate replication of Crossley, Louwerse, McCarthy, & McNamara (2007). Language Teaching, 41 (3), 409–429. DOI logoGoogle Scholar
Cumming, G., & Calin-Jageman, R.
(2017) Introduction to the new statistics: Estimation, open science, and beyond. Routledge.Google Scholar
East, M.
(2021) What do beginning teachers make of task-based language teaching? A comparative re-production of East (2014). Language Teaching, 54 (4), 552–566. DOI logoGoogle Scholar
Eckerth, J.
(2009) Negotiated interaction in the L2 classroom. Language Teaching, 42 (1), 109–130. DOI logoGoogle Scholar
Ellis, R.
(1997) SLA research and language teaching. Oxford University Press.Google Scholar
Foster, P.
(1998) A classroom perspective on the negotiation of meaning. Applied Linguistics, 19 (1), 1–23. DOI logoGoogle Scholar
Gass, S. M.
(2017) Input, interaction, and the second language learner (2nd ed.). Routledge. DOI logoGoogle Scholar
Gass, S. M., Loewen, S., & Plonsky, L.
(2021) Coming of age: The past, present, and future of quantitative SLA research. Language Teaching, 54 (2), 245–258. DOI logoGoogle Scholar
Gould, J., & Kolb, W. L.
(Eds.) (1967) A dictionary of the social sciences. Tavistock.Google Scholar
Handley, Z.
(2018) Replication research in computer-assisted language learning: Replication of Neri et al. (2008) and Satar & Özdener (2008). Language Teaching 51 (3), 417–429. DOI logoGoogle Scholar
Hiver, P., & Al-Hoorie, A. H.
(2020) Reexamining the role of vision in second language motivation: A preregistered conceptual replication of You, Dörnyei, and Csizér (2016). Language Learning, 70 (1), 48–102. DOI logoGoogle Scholar
Howard, M.
(2005) Les contextes prototypiques et marqués de l’emploi de l’imparfait par l’apprenant du français langue étrangère. In Nouveaux développements de l’imparfait (pp. 175–197). Rodopi.Google Scholar
In’nami, Y., & Koizumi, R.
(2010) Can structural equation models in second language testing and learning research be successfully replicated? International Journal of Testing, 10 (3), 262–273. DOI logoGoogle Scholar
Johnson, M. D., & Nicodemus, C. L.
(2016) Testing a threshold: An approximate replication of Johnson, Mercado & Acevedo 2012. Language Teaching, 49 (2), 251–274. DOI logoGoogle Scholar
Kessler, M., Ma, W., & Solheim, I.
(2021) The effects of topic familiarity on text quality, complexity, accuracy, and fluency: A conceptual replication. TESOL Quarterly, 1–28. DOI logoGoogle Scholar
Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., Bahník, Š., Bernstein, M. J., Bocian, K., Brandt, M. J., Brooks, B., Brumbaugh, C. C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W. E., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E. M., … Nosek, B. A.
(2014) Investigating variation in replicability: A “many labs” replication project. Social Psychology, 45 (3), 142–152. DOI logoGoogle Scholar
Larson-Hall, J., & Plonsky, L.
(2015) Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. Language Learning, 65 (S1), 127–159. DOI logoGoogle Scholar
Loewen, S.
(2015) Introduction to instructed second language acquisition. Routledge.Google Scholar
Mackey, A., & Gass, S. M.
(2016) Second language research: Methodology and design (2nd ed.). Routledge.Google Scholar
Markee, N.
(2017) Are replication studies possible in qualitative second/foreign language classroom research? A call for comparative re-production research. Language Teaching, 50 (3), 367–383. DOI logoGoogle Scholar
Marsden, E., & Chen, H.-Y.
(2011) The roles of structured input activities in processing instruction and the kinds of knowledge they promote: Roles of structured input activities in processing instruction. Language Learning, 61 (4), 1058–1098. DOI logoGoogle Scholar
Matsuda, P. K.
(2012) On the nature of second language writing: Replication in a postmodern field. Journal of Second Language Writing, 21 (3), 300–302. DOI logoGoogle Scholar
McManus, K.
(2019) Awareness of L1 form-meaning mappings can reduce crosslinguistic effects in L2 grammatical learning. Language Awareness, 28 (2), 114–138. DOI logoGoogle Scholar
(2020) Crosslinguistic influence and L2 grammar learning: Proposed replications of Ellis and Sagarra (2011) and Tolentino and Tokowicz (2014). Language Teaching, 1–9. DOI logoGoogle Scholar
(2021a) Crosslinguistic influence and second language learning. Routledge. DOI logoGoogle Scholar
(2021b) Examining the effectiveness of language-switching practice for reducing cross-language competition in L2 grammatical processing. Bilingualism: Language and Cognition, 24 (1), 167–184. DOI logoGoogle Scholar
(2022) Are replication studies infrequent because of negative attitudes? Insights from a survey of attitudes and practices in second language research. Studies in Second Language Acquisition. Advance online publication. DOI logoGoogle Scholar
McManus, K., & Marsden, E.
(2017) L1 explicit instruction can improve L2 online and offline performance. Studies in Second Language Acquisition, 39 (3), 459–492. DOI logoGoogle Scholar
(2018) Online and offline effects of L1 practice in L2 grammar learning: A partial replication. Studies in Second Language Acquisition, 40 (2), 459–475. DOI logoGoogle Scholar
(2019a) Signatures of automaticity during practice: Explicit instruction about L1 processing routines can improve L2 grammatical processing. Applied Psycholinguistics, 40 (1), 205–234. DOI logoGoogle Scholar
(2019b) Using explicit instruction about L1 to reduce crosslinguistic effects in L2 grammar learning: Evidence from oral production in L2 French. Modern Language Journal, 103 (2), 459–480. DOI logoGoogle Scholar
Morris, S. B., & DeShon, R. P.
(2002) Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7 (1), 105–125. DOI logoGoogle Scholar
Myles, F.
(2010) The development of theories of second language acquisition. Language Teaching, 43 (3), 320–332. DOI logoGoogle Scholar
Norouzian, R.
(2020) Sample size planning in quantitative L2 research: A pragmatic approach. Studies in Second Language Acquisition, 42 (4), 849–870. DOI logoGoogle Scholar
Norris, J. M., Plonsky, L., Ross, S. J., & Schoonen, R.
(2015) Guidelines for reporting quantitative methods and results in primary research: Guidelines for reporting quantitative methods. Language Learning, 65 (2), 470–476. DOI logoGoogle Scholar
Nosek, B. A., & Errington, T. M.
(2020) What is replication? PLOS Biology, 18 (3), e3000691. DOI logoGoogle Scholar
Open Science Collaboration
(2015) Estimating the reproducibility of psychological science. Science, 349 (6251), aac4716. DOI logoGoogle Scholar
Plonsky, L.
(2012) Replication, meta-analysis, and generalizability. In G. K. Porte (Ed.), Replication research in applied linguistics (pp. 116–132). Cambridge University Press.Google Scholar
(2013) Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35 (4), 655–687. DOI logoGoogle Scholar
(2015) Statistical power, p values, descriptive statistics, and effect sizes: A “back-to- basics” approach to advancing quantitative methods in L2 research. In L. Plonsky (Ed.), Advancing quantitative methods in second language research (pp. 23–45). Routledge. DOI logoGoogle Scholar
(2022) Bibliography of research synthesis and meta-analysis in applied linguistics. [URL]
Plonsky, L., & Oswald, F. L.
(2014) How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64 (4), 878–912. DOI logoGoogle Scholar
Porte, G. K.
(Ed.) (2012) Replication research in applied linguistics. Cambridge University Press.Google Scholar
Porte, G., & Richards, K.
(2012) Focus article: Replication in second language writing research. Journal of Second Language Writing, 21 (3), 284–293. DOI logoGoogle Scholar
Révész, A.
(2019) Replication in task-based language teaching research: Kim (2012) and Shintani (2012). Language Teaching, 52 (3), 374–384. DOI logoGoogle Scholar
Rott, S., & Gavin, B.
(2015) Comprehending and learning from Internet sources: A conceptual replication study of Goldman, Braasch, Wiley, Greasser, and Brodowinska (2012). CALICO Journal, 32 (2), 323–354. DOI logoGoogle Scholar
Sanz, C., & Morgan-Short, K.
(2004) Positive evidence versus explicit rule presentation and explicit negative feedback: A computer-assisted study. Language Learning, 54 (1), 35–78. DOI logoGoogle Scholar
Schmidt, S.
(2009) Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology, 13 (2), 90–100. DOI logoGoogle Scholar
Waring, R.
(1997) The negative effects of learning words in semantic sets: A replication. System, 25 (2), 261–274. DOI logoGoogle Scholar
Wu, S.-L., & Ortega, L.
(2013) Measuring global oral proficiency in SLA research: A new elicited imitation test of L2 Chinese. Foreign Language Annals, 46 (4), 680–704. DOI logoGoogle Scholar