Cover not available
Article published In:
Translation, Cognition & Behavior
Vol. 5:1 (2022) ► pp.1–26

Considerations of ecological validity in cognitive translation and interpreting studies

ORCID logo | University of North Carolina at Charlotte
ORCID logo | Butler University
Renewed interest in research methods used in cognitive translation and interpreting studies (CTIS) has led to increased reflection on the nature of research and experimental design, internal and external validity, and the type and nature of experimental tasks. Of particular concern is the extent to which valid generalization can be made from empirical studies of translation, interpreting, and their associated cognitive behaviors as objects of study. This article traces the definition and history of ecological validity in the extant literature on research methods and philosophy of science, emphasizing three considerations: the experimental setting, the stimuli under investigation, and the behavioral response of participants. Subsequently, we discuss potential misunderstandings or misapplications of appeals to ecological validity, including mundane realism, naturalistic tasks, and various data collection methods. A final section describes key points to consider for research in CTIS. Throughout, the argument considers tradeoffs among various categories of validity as well as the importance of aligning experimental design with research goals. This extensive engagement with a singular aspect of validity is provided to encourage deeper reflection and better communication around the topic of ecological validity.
Keywords: ecological validity, generalizability, mundane realism, experimental control, naturalistic tasks, data collection methods
Article outline
  • 1.Introduction
  • 2.Definitions of ecological validity
    • 2.1Alternative conception of ecological validity
  • 3.Balancing risks to ecological validity
    • 3.1Naturalistic tasks
    • 3.2Mundane realism
    • 3.3Data collection methods
  • 4.Establishing ecological validity
    • 4.1Define the relevant environment for experimentation and generalization
    • 4.2Align the stimuli, variables, and measurement with the research purpose
    • 4.3Consider influences and constraints on participant responses
  • 5.Conclusion
  • References
Published online: 8 July 2022
DOI logo
https://doi.org/10.1075/tcb.00061.mel
References
Alves, Fabio
2015 “Translation Process Research at the Interface: Paradigmatic, theoretical, and methodological issues in dialogue with cognitive science, expertise studies, and psycholinguistics.” In Psycholinguistic and Cognitive Inquiries into Translation and Interpreting. Edited by A. Ferreira, and J. W. Schwieter, 17–40. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Araújo, Duarte, Keith Davids, and Pedro Passos
2007 “Ecological Validity, Representative Design, and Correspondence between Experimental Task Constraints and Behavioral Setting: Comment on Rogers, Kadar, and Costall (2005).” Ecological Psychology 19 (1), 69–78. DOI logoGoogle Scholar
Banaji, Mahzarin R., and Robert G. Crowder
1989 “The Bankruptcy of Everyday Memory.” American Psychologist 44 (9), 1185–1193. DOI logoGoogle Scholar
Baraldi, Claudio, and Christopher D. Mellinger
2016 “Observations.” In Researching Translation and Interpreting. Edited by C. V. Angelelli, and B. J. Baer, 257–268. New York: Routledge.Google Scholar
Berkowitz, Leonard, and Edward Donnerstein
1982 “External Validity is More than Skin Deep: Some answers to criticisms of laboratory experiments.” American Psychologist 37 (3), 245–257. DOI logoGoogle Scholar
Blanco-Elorrieta, Esti, and Liina Pylkkänen
2018 “Ecological Validity in Bilingualism Research and the Bilingual Advantage.” Trends in Cognitive Sciences 22 (12), 1117–1126. DOI logoGoogle Scholar
Bolaños Medina, Alicia
2016 “Translation Psychology within the Framework of Translator Studies: New research perspectives.” In From the Lab to the Classroom and Back Again: Perspectives on Translation and Interpreting Training. Edited by C. Martín de León, and V. González-Ruíz, 59–100. Peter Lang.Google Scholar
Bracht, Glenn H., and Gene V. Glass
1968 “The External Validity of Experiments.” American Educational Research Journal 5 (4), 437–474. DOI logoGoogle Scholar
Bronfenbrenner, Urie
1977 “Toward an Experimental Ecology of Human Development.” American Psychologist 32 (7), 513–531. DOI logoGoogle Scholar
Brunswik, Egon
1943 “Organismic Achievement and Environmental Probability.” Psychological Review 50 (3), 255–272. DOI logoGoogle Scholar
1949 “Remarks on Functionalism in Perception.” Journal of Personality 18 (1), 56–65. DOI logoGoogle Scholar
1955 “Representative Design and Probabilistic Theory in a Functional Psychology.” Psychological Review 62 (3), 193–217. DOI logoGoogle Scholar
Calder, Bobby J., Lynn W. Phillips, and Alice M. Tybout
1981 “Designing Research for Application.” Journal of Consumer Research 81, 197–207. DOI logoGoogle Scholar
1983 “Beyond External Validity.” Journal of Consumer Research 10 (1), 112–114. DOI logoGoogle Scholar
Campbell, Donald T., and Julian C. Stanely
1963 Experimental and Quasi-experimental Designs for Research. Cengage.Google Scholar
Carlsmith, J. Merrill, Phoebe C. Ellsworth, and Eliot Aronson
1976 Methods of Research in Social Psychology. Addison-Wesley.Google Scholar
Chaytor, Naomi, and Maureen Schmitter-Edgecombe
2003 “The Ecological Validity of Neuropsychological Tests: A review of the literature on everyday cognitive skills.” Neuropsychology Review 13 (4), 181–197. DOI logoGoogle Scholar
Chow, Siu L.
1987 “Science, Ecological Validity, and Experimentation.” Journal for the Theory of Social Behavior 17 (2), 181–194. DOI logoGoogle Scholar
Cook, Thomas D., and Donald T. Campbell
1979 Quasi-experimentation: Design & analysis issues for field settings. Houghton Mifflin.Google Scholar
Dipboye, Robert L., and Michael F. Flanagan
1980 “Reply to Willems and Howard.” American Psychologist 351, 388–390. DOI logoGoogle Scholar
Dobbins, Gregory H., Irving M. Lane, and Dirk D. Steiner
1988 “A Note on the Role of Laboratory Methodologies in Applied Behavioural Research: Don’t throw out the baby with the bath water.” Journal of Organizational Behavior 9 (3), 281–286. DOI logoGoogle Scholar
Downie, Jonathan
2021 “Interpreting is Interpreting: Why we need to leave behind interpreting settings to discover comparative interpreting studies.” Translation and Interpreting Studies 16 (3), 325–346. DOI logoGoogle Scholar
Dunlosky, John, Sara Bottiroli, and Marissa Hartwig
2009 “Sins Committed in the Name of Ecological Validity: A call for representative design in education science.” In Handbook of Metacognition in Education. Edited by D. J. Hacker, J. Dunlosky, and A. C. Graesser, 430–440. New York: Routledge.Google Scholar
Ehrensberger-Dow, Maureen, and Gary Massey
2020 “Translation Workplace-based Research.” In The Routledge Handbook of Translation and Technology. Edited by M. O’Hagan, 354–369. New York: Routledge. DOI logoGoogle Scholar
Franzen, Michael D., and Karen L. Wilhelm
1996 “Conceptual Foundations of Ecological Validity in Neuropsychological Assessment.” In Ecological Validity of Neuropsychological Testing. Edited by R. J. Sbordone, and C. J. Long, 91–112. New York: St. Lucie Press.Google Scholar
García, Adolfo M.
2019 The Neurocognition of Translation and Interpreting. John Benjamins. DOI logoGoogle Scholar
Gerber, Leah, Jim Hlavac, Irwyn Shepherd, Paul McIntosh, Alex Avella Archila, and Hyein Cho
2021 “Stepping into the Future: Virtual reality training for community interpreters working in the area of family violence.” Journal of Specialised Translation 361, 252–275.Google Scholar
Gibson, James J.
1957 “Survival in a World of Probable Objects. Review of perception and the representative design of psychological experiment by E. Brunswik.” Contemporary Psychology 21, 33–35. DOI logoGoogle Scholar
1986 The Ecological Approach to Visual Perception. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
Gile, Daniel
2001 “Consecutive vs. Simultaneous: Which is more accurate?” Interpretation Studies 11, 8–20.Google Scholar
2009 “Interpreting Studies: A critical view from within.” MonTI 11, 135–155. DOI logoGoogle Scholar
2016 “Experimental Research.” In Researching Translation and Interpreting. Edited by C. V. Angelelli, and B. J. Baer, 220–228. New York: Routledge.Google Scholar
2018 “Research into Translation as a Specialism: An analysis and recommendations.” Journal of Specialised Translation 301, 23–39.Google Scholar
Hale, Sandra, Jane Goodman-Delahunty, Natalie Martschuk, and Stephen Doherty
2021 “The Effects of Mode on Interpreting Performance in a Simulated Police Interview.” Translation and Interpreting Studies. DOI logoGoogle Scholar
Hammond, Kenneth R.
1998 “ Ecological Validity: Then and Now .” Available at [URL]
Hammond, Kenneth R., and Thomas R. Stewart
2001 “Introduction.” In The Essential Brunswik: Beginnings, Explications, Applications. Edited by K. R. Hammond, and T. R. Stewart, 3–11. New York: Oxford University Press. DOI logoGoogle Scholar
Heilmann, Arndt, Tatiana Serbina, Daniel Couto Vale, and Stella Neumann
2019 “Shorter than a Text, Longer than a Sentence: Source text length for ecologically valid translation experiments.” Target 31 (1), 98–124. DOI logoGoogle Scholar
Highhouse, Scott
2009 “Designing Experiments that Generalize.” Organizational Research Methods 12 (3), 554–566. DOI logoGoogle Scholar
Hoc, Jean-Michel
2001 “Towards Ecological Validity of Research in Cognitive Ergonomics.” Theoretical Issues in Ergonomic Science 2 (3), 278–288. DOI logoGoogle Scholar
Holleman, Gijs A., Ignace T. C. Hooge, Chantal Kemner, and Roy S. Hessels
2020 “The “Real-World Approach” and its Problems: A critique of the term ecological validity.” Frontiers in Psychology 111, 721. DOI logoGoogle Scholar
House, Juliane
2013 “Towards a New Linguistic-Cognitive Orientation in Translation Studies.” Target 25 (1), 46–60. DOI logoGoogle Scholar
Jääskeläinen, Riitta
2012 “Translation Psychology.” In Handbook of Translation Studies. Edited by Y. Gambier, and L. van Doorslaer, 191–197. Amsterdam: John Benjamins. DOI logoGoogle Scholar
2017 “Verbal Reports.” In Handbook of Translation and Cognition. Edited by J. W. Schwieter, and A. Ferreira, 213–231. Hoboken, NJ: Wiley. DOI logoGoogle Scholar
Jakobsen, Arnt L.
2020 “Translation Technology Research with Eye Tracking.” In The Routledge Handbook of Translation and Technology. Edited by M. O’Hagan, 398–416. New York: Routledge. DOI logoGoogle Scholar
Kaiser, Henry F.
1960 “Directional Statistical Hypotheses.” Psychological Review 67 (3), 160–167. DOI logoGoogle Scholar
Kihlstrom, John F.
2021 “Ecological Validity and ‘Ecological Validity.’” Perspectives on Psychological Science 16 (2), 466–471. DOI logoGoogle Scholar
Korpal, Pawel
2015 “Eye-tracking in Translation and Interpreting Studies: The growing popularity and methodological problems.” In Accessing Audiovisual Translation. Edited by L. Bogucki, and M. Deckert, 199–212. Frankfurt: Peter Lang.Google Scholar
Kothgassner, Oswald D., and Anna Felnhofer
2020 “Does Virtual Reality Help to Cut the Gordian Knot between Ecological Validity and Experimental Control?” Annals of the International Communication Association 44 (3), 210–218. DOI logoGoogle Scholar
Kvavilashvili, Lia, and Judi A. Ellis
2004 “Ecological Validity and the Real-life/Laboratory Controversy in Memory Research: A critical (and historical) review.” History and Philosophy of Psychology 6 (1), 59–80.Google Scholar
Ladouce, Simon, David I. Donaldson, Paul A. Dudchenko, and Magdalena Ietswaart
2017 “Understanding Minds in Real-world Environments: Toward a mobile cognition approach.” Frontiers in Human Neuroscience 101, 694. DOI logoGoogle Scholar
Lewkowicz, David J.
2001 “The Concept of Ecological Validity: What are its limitations and is it bad to be invalid?” Infancy 2 (4), 437–450. DOI logoGoogle Scholar
Lucas, Jeffrey W.
2003 “Theory-testing, Generalization, and the Problem of External Validity.” Sociological Theory 21 (3), 236–253. DOI logoGoogle Scholar
Lund, Thorleif
2021 “A Revision of the Campbellian Validity System.” Scandinavian Journal of Educational Research 65 (3), 523–535. DOI logoGoogle Scholar
Lynch, John G., Jr.
1999 “Theory and External Validity.” Journal of the Academy of Marketing Science 27 (3), 367–376. DOI logoGoogle Scholar
Malamatidou, Sofia
2018 Corpus Triangulation: Combining Data and Methods in Corpus-based Translation Studies. London: Routledge.Google Scholar
Marín García, Álvaro
2019 “The Opportunities of Epistemic Pluralism for Cognitive Translation Studies.” Translation, Cognition & Behavior 2 (2), 165–185. DOI logoGoogle Scholar
Matt, G. E., A. Brewer, and M. Sklar
2010 “External validity.” In International Encyclopedia of Education, 3rd Ed. Edited by P. Peterson, E. Baker, and B. McGaw, 521–527. Elsevier Science. DOI logoGoogle Scholar
McGrath, Joseph E., and David Brinberg
1983 “External Validity and the Research Process: A comment on the Calder/Lynch dialogue.” Journal of Consumer Research 10 (1), 115–124. DOI logoGoogle Scholar
Mellinger, Christopher D.
2015 “On the Applicability of Internet-mediated Research Methods to Investigate Translators’ Cognitive Behavior.” Translation & Interpreting 7 (1), 59–71.Google Scholar
2020 “Positionality in Public Service Interpreting Research.” FITISPos International Journal 71, 92–109. DOI logoGoogle Scholar
Mellinger, Christopher D., and Thomas A. Hanson
2017 Quantitative Research Methods in Translation and Interpreting Studies. New York: Routledge.Google Scholar
2020 “Methodological Considerations for Survey Research: Validity, Reliability, and Quantitative Analysis.” Linguistica Antverpiensia, New Series – Themes in Translation Studies 191, 172–190. DOI logoGoogle Scholar
Mellinger, Christopher D., Nicoletta Spinolo, Maureen Ehrensberger-Dow, and Sharon O’Brien
Forthcoming. “Designing Naturalistic Tasks.” In Innovative Data Collection Methods in Cognitive Translation and Interpreting Studies Edited by A. Rojo and R. Muñoz Martín Amsterdam John Benjamins
Mook, Douglas G.
1983 “In Defense of External Invalidity.” American Psychologist 38 (4), 379–387. DOI logoGoogle Scholar
Muñoz Martín, Ricardo
2010 “On Paradigms and Cognitive Translatology.” In Translation and Cognition. Edited by G. M. Shreve and E. Angelone, 169–188. Amsterdam: John Benjamins. DOI logoGoogle Scholar
2016 “Of Minds and Men–Computers and Translators.” Poznań Studies in Contemporary Linguistics 52 (2), 351–381. DOI logoGoogle Scholar
2017 “Looking Toward the Future of Cognitive Translation Studies.” In The Handbook of Translation and Cognition. Edited by J. W. Schwieter, and A. Ferreira, 554–572. Hoboken, NJ: Wiley. DOI logoGoogle Scholar
Neguţ, Alexandra, Silviu-Andrei Matu, Florin Alin Sava, and Daniel David
2016 “Virtual Reality Measures in Neuropsychological Assessment: A meta-analytic review.” The Clinical Neuropsychologist 30 (2), 165–184. DOI logoGoogle Scholar
O’Brien, Sharon
2013 “The Borrowers: researching the cognitive aspects of translation.” Target 25 (1), 5–17. DOI logoGoogle Scholar
Orne, Martin T.
1962 “On the Social Psychology of the Psychological Experiment: With particular reference to demand characteristics and their implications.” American Psychologist 17 (11), 776–783. DOI logoGoogle Scholar
Parsons, Thomas D.
2016 Clinical Neuropsychology and Technology: What’s New and How We Can Use It. Cham, Switzerland: Springer. DOI logoGoogle Scholar
Peecher, Mark E., and Ira Solomon
2001 “Theory and Experimentation in Studies of Audit Judgments and Decisions: Avoiding common research traps.” International Journal of Auditing 5 (3), 193–203. DOI logoGoogle Scholar
Risku, Hanna, Regina Rogl, and Jelena Milošević
2020 “Researching Workplaces.” In The Bloomsbury Companion to Language Industry Studies. Edited by E. Angelone, M. Ehrensberger-Dow, and G. Massey, 37–62. London: Bloomsbury. DOI logoGoogle Scholar
Rojo López, Ana M., and Pawel Korpal
2020 “Through your Skin to your Heart and Brain: A Critical Evaluation of Physiological Methods in Cognitive Translation and Interpreting Studies.” Linguistica Antverpiensia, New Series: Themes in Translation Studies 191, 191–217. DOI logoGoogle Scholar
Saldanha, Gabriela, and Sharon O’Brien
2013 Research Methodologies in Translation Studies. London: Routledge.Google Scholar
Schmuckler, Mark A.
2001 “What is Ecological Validity? A dimensional analysis.” Infancy, 2 (4), 419–436. DOI logoGoogle Scholar
Shadish, William R., Thomas D. Cook, and Donald T. Campbell
2002 Experimental and Quasi-experimental Designs for Generalized Causal Inference. Cengage.Google Scholar
Shreve, Gregory M., and Erik Angelone
2010 “Translation and Cognition: Recent developments.” In Translation and Cognition. Edited by G. M. Shreve, and E. Angelone, 1–13. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Spooner, Donna M., and Nancy A. Pachana
2006 “Ecological Validity in Neuropsychological assessment: A case for greater consideration in research with neurologically intact populations.” Archives of Clinical Neuropsychology 211, 327–337. DOI logoGoogle Scholar
Steiner, Erich
2021 “Translation, Equivalence, and Cognition.” In The Routledge Handbook of Translation and Cognition. Edited by F. Alves, and A. L. Jakobsen, 344–359. New York: Routledge.Google Scholar
Stoffregen, Thomas A.
1993 “ ‘Natural,’ ‘Real,’ and the Use of Non-physical Displays in Perception-Action Research.” ISEP Newsletter 61, 4–9.Google Scholar
Stoffregen, Thomas A., Benoit Bardy, L. J. Smart, and Randy J. Pagulayan
2003 “On the Nature and Evaluation of Fidelity in Virtual Environments.” In Virtual and Adaptive Environments: Applications, Implications, and Human Performance Issues. Edited by L. J. Hettinger, and M. W. Haas, 111–128. Mahwah, NJ: Lawrence Erlbaum. DOI logoGoogle Scholar
Sun, Sanjun, Tian Li, and Xiaoyan Zhou
2020 “Effects of Thinking Aloud on Cognitive Effort in Translation.” Linguistica Antverpiensia, New Series – Themes in Translation Studies 191, 132–151. DOI logoGoogle Scholar
Tiselius, Elisabet
2021 “Informed Consent: An Overlooked Part of Ethical Research in Interpreting Studies.” InContext 1 (1), 83–100. DOI logoGoogle Scholar
Wachtel, Paul L.
1980 “Investigation and its Discontents: Some Constraints on Progress in Psychological Research.” American Psychologist 35 (5), 399–408. DOI logoGoogle Scholar
Weng, Yu, and Binghan Zheng
2020 “A Multi-Methodological Approach to Studying Time-Pressure in Written Translation: Manipulation and measurement.” Linguistica Antverpiensia, New Series: Themes in Translation Studies 191, 218–236. DOI logoGoogle Scholar
Winer, Russell S.
1999 “Experimentation in the 21st Century: The importance of external validity.” Journal of the Academy of Marketing Science 27 (3), 349–358. DOI logoGoogle Scholar
Xiao, Kairong, and Ricardo Muñoz Martín
2020 “Cognitive Translation Studies: Models and methods at the cutting edge.” Linguistica Antverpiensia, New Series: Themes in Translation Studies 191, 1–24. DOI logoGoogle Scholar
Ziegler, Klaus, and Sebastiano Gigliobianco
2018 “Present? Remote? Remotely Present! New technological approaches to remote simultaneous conference interpreting.” In Interpreting and Technology. Edited by C. Fantinuoli, 119–139. Berlin: Language Science Press.Google Scholar