Article published In:
Review of Cognitive Linguistics: Online-First Articles

Raising the bar

Enhancing study design and validity in L2 idiom research

ORCID logo | Kansai University

Empirical testing is a challenging aspect of L2 classroom-based research, especially when exploring the multifaceted nature of figurative language, such as metaphorical idioms. Typically, studies in applied linguistics involve language learners from convenience samples of intact classes. This approach can pose problems as these classes represent non-random, often small, samples of participants. Despite these challenges, appropriate precautions and considerations, such as addressing overlooked idiom-inherent variables, contemplating counterbalancing, managing time-on-task, and making well-informed treatment and data collection design choices can minimize confounding variables and enhance a study’s design and resultant validity. The author of this article offers a reflective commentary based on a previous study (Ramonda, 2022) to expound on these considerations and provide modest proposals for improving future study quality in this domain.

Keywords: metaphorical idioms, confounding variables, study design, validity
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1Challenges in L2 classroom-based research
    • 2.2Figurative language and L2 learning
    • 3.Method
    • 3.1Treatment preparation
    • 3.2Participants
    • 3.3Procedure
      • 3.3.1Study design
      • 3.3.2Biographical information and vocabulary pre-tests
      • 3.3.3PowerPoint treatment
      • 3.3.4Immediate and delayed post-treatment assessments
  • 4.Results
  • 5.Discussion
    • 5.1Confounding variables
      • 5.1.1Semantic transparency
      • 5.1.2Imageability
      • 5.1.3Lexical features
      • 5.1.4L1 equivalents
      • 5.1.5Complexity of meaning
    • 5.2Counterbalancing and time-on-task
    • 5.3Measuring the durable learning of idioms
    • 5.4Participant-related variables
  • 6.Conclusion
  • References
Published online: 29 April 2024
DOI logo
https://doi.org/10.1075/rcl.00185.ram
References
Boers, F.
(2011) Cognitive Semantic ways of teaching figurative phrases: An assessment. Review of Cognitive Linguistics, 9 (1), 227–261. DOI logoGoogle Scholar
(2013) Cognitive Linguistic approaches to teaching vocabulary: Assessment and integration. Language Teaching, 46 (2), 208–224. DOI logoGoogle Scholar
Boers, F., & Demecheleer, M.
(2001) Measuring the impact of cross-cultural differences on learners’ comprehension of imageable idioms. ELT Journal, 55 (3), 255–262. DOI logoGoogle Scholar
Boers, F., Lindstromberg, S., Littlemore, J., Stengers, H., & Eyckmans, J.
(2008) Variables in the mnemonic effectiveness of pictorial elucidation. In F. Boers & S. Lindstromberg (Eds.), Cognitive Linguistic approaches to teaching vocabulary and phraseology (pp. 189–216). Berlin: Mouton de Gruyter. DOI logoGoogle Scholar
Boers, F., Piquer-Píriz, A. M., Stengers, H., & Eyckmans, J.
(2009) Does pictorial elucidation foster recollection of idioms? Language Teaching Research, 13 (4), 367–382. DOI logoGoogle Scholar
Boers, F., Webb, S.
(2015) Gauging the semantic transparency of idioms: Do natives and learners see eye to eye? In R. R. Heredia & A. B. Cieślicka (Eds.), Bilingual figurative language processing (pp. 368–392). Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Bortfeld, H.
(2002) What native and non-native speakers’ images for idioms tell us about figurative language. In R. R. Heredia & J. Altarriba (Eds.), Advances in psychology: Bilingual sentence processing (pp. 275–295). Amsterdam, Netherlands: Elsevier Press. DOI logoGoogle Scholar
Cacciari, C., & Glucksberg, S.
(1991) Understanding idiomatic expressions: The contribution of word meanings. In G. B. Simpson (Ed.), Understanding word and sentence (pp. 217–240). Amsterdam: Elsevier. DOI logoGoogle Scholar
Costa, A., Santesteban, M., & Caño, A.
(2005) On the facilitatory effects of cognate words in bilingual speech production. Brain and Language, 94 (1), 94–103. DOI logoGoogle Scholar
Dörnyei, Z.
(2007) Research methods in applied linguistics. New York: Oxford University Press.Google Scholar
Farley, A., Ramonda, K., & Liu, X.
(2012) The concreteness effect and the bilingual lexicon: The impact of visual stimuli attachment on meaning recall of abstract L2 words. Language Teaching Research, 16 (4), 449–466. DOI logoGoogle Scholar
Gass, S., Loewen, S., & Plonsky, L.
(2021) Coming of age: The past, present, and future of quantitative SLA research. Language Teaching, 54 (2), 245–258. DOI logoGoogle Scholar
Gibbs, R. W., Nayak, N. P., & Cutting, C.
(1989) How to kick the bucket and not decompose: Analyzability and idiom processing. Journal of Memory and Language, 28 (5), 576–593. DOI logoGoogle Scholar
Grant, L. E., & Bauer, L.
(2004) Criteria for re-defining idioms: Are we barking up the wrong tree? Applied Linguistics, 25 (1), 38–61. DOI logoGoogle Scholar
In’nami, Y., Mizumoto, A., Plonsky, L., & Koizumi, R.
(2022) Promoting computationally reproducible research in applied linguistics: Recommended practices and considerations. Research Methods in Applied Linguistics, 1 (3), 100030. DOI logoGoogle Scholar
Ioannidis, J. P. A.
(2005) Why most published research findings are false. PLOS Medicine, 2 (8), e124. DOI logoGoogle Scholar
Irujo, S.
(1986) Don’t put your leg in your mouth: Transfer in the acquisition of idioms in a second language. TESOL Quarterly, 20 (2), 287–304. DOI logoGoogle Scholar
Isbell, D. R., Brown, D., Chen, M., Derrick, D. J., Ghanem, R., Gutiérrez Arvizu, M. N., Schnur, E., Zhang, M., & Plonsky, L.
(2022) Misconduct and questionable research practices: The ethics of quantitative data handling and reporting in applied linguistics. The Modern Language Journal, 106 (1), 172–195. DOI logoGoogle Scholar
Katz, A. N.
(1983) What does it mean to be a high imager? In J. C. Yuille (Ed.), Imagery, Memory and Cognition (pp. 39–63). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.Google Scholar
Kövecses, Z., & Szabcó, P.
(1996) Idioms: A view from cognitive semantics. Applied Linguistics, 17 (3), 326–355. DOI logoGoogle Scholar
Larsson, T., Plonsky, L., Sterling, S., Kytö, M., Yaw, K., & Wood, M.
(2023) On the frequency, prevalence, and perceived severity of questionable research practices. Research Methods in Applied Linguistics, 2 (3), 100064. DOI logoGoogle Scholar
Laufer, B., & Goldstein, Z.
(2004) Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning, 54 (3), 399–436. DOI logoGoogle Scholar
Liu, D.
(2008) Idioms: Description, comprehension, acquisition, and pedagogy. London, UK: Routledge.Google Scholar
Moon, R.
(1998) Fixed expressions and idioms in English: A corpus-based approach. Oxford, UK: Oxford University Press. DOI logoGoogle Scholar
Morris, C. D., Bransford, J. D., & Franks, J. J.
(1977) Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16 (5), 519–533. DOI logoGoogle Scholar
Nunberg, G., Sag, I. A., & Wasow, T.
(1994) Idioms. Language, 70 (3), 491–538. DOI logoGoogle Scholar
Paivio, A.
(1986) Mental representations. Oxford, UK: Oxford University Press.Google Scholar
Paivio, A., & Desrochers, A.
(1980) A dual-coding approach to bilingual memory. Canadian Journal of Psychology, 34 (4), 388–399. DOI logoGoogle Scholar
Paivio, A., & Harshman, R.
(1983) Factor analysis of a questionnaire on imagery and verbal habits and skills. Canadian Journal of Psychology, 37 (4), 461–483. DOI logoGoogle Scholar
Pashler, H., & Harris, C. R.
(2012) Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7 (6), 531–536. DOI logoGoogle Scholar
Ramonda, K.
(2019) The role of encyclopedic world knowledge in semantic transparency intuitions of idioms. English Language & Linguistics, 23 (1), 31–53. DOI logoGoogle Scholar
(2022) A double-edged sword: Metaphor and metonymy through pictures for learning idioms. International Review of Applied Linguistics in Language Teaching, 60 (3), 523–561. DOI logoGoogle Scholar
Riding, R. J.
(1991) Cognitive styles analysis. Birmingham, England: Learning and Training Technology.Google Scholar
Riding, R. J., & Cheema, I.
(1991) Cognitive Styles—An overview and integration. Educational Psychology, 11 1, 193–215. DOI logoGoogle Scholar
Schmitt, N.
(2008) Review article: Instructed second language vocabulary learning. Language Teaching Research, 12 (3), 329–363. DOI logoGoogle Scholar
Sinclair, J.
(2012) Collins COBUILD idioms dictionary (3rd ed.). London, UK: Collins.Google Scholar
Skoufaki, S.
(2009) Investigating the source of idiom transparency intuitions. Metaphor and Symbol, 24 (1), 20–41. DOI logoGoogle Scholar
Tonzar, C., Lotto, L., & Job, R.
(2009) L2 vocabulary acquisition in children: Effects of learning method and cognate status. Language Learning, 59 (3), 623–646. DOI logoGoogle Scholar
Yu, X., & Boers, F.
(2023) Inferring the meaning of idioms: Does accuracy matter for retention in memory? RELC Journal. Epub ahead of print. DOI logoGoogle Scholar
Verspoor, M., & Lowie, W.
(2003) Making sense of polysemous words. Language Learning, 53 (3), 547–586. DOI logoGoogle Scholar