Cover not available
Article published In:
Translation and Interpreting Studies
Vol. 18:3 (2023) ► pp.421–447

Relating utterance fluency to perceived fluency of interpreting

A partial replication and a mini meta-analysis

ORCID logo | Xiamen University
ORCID logo | Xiamen University
The prospect of automated scoring for interpreting fluency has prompted investigations into the predictability of human raters’ perceived fluency based on acoustically measured utterance fluency. Recently, Han, Chen, Fu and Fan (2020) correlated ten utterance fluency measures with raters’ perceived fluency ratings. To verify previous correlational patterns, the present study partially replicated Han et al. (2020). Our analysis shows that most of the correlations observed in Han et al. (2020) were successfully replicated. To produce overall interim estimates of the true relationships, we conducted a mini meta-analysis of correlation coefficients reported in six relevant studies, informed by the “continuously cumulating meta-analysis” approach (Braver et al. 2014). We found that phonation time ratio, mean length of run, and speech rate had relatively strong correlations with perceived fluency. We discuss these findings in light of automated fluency assessment and the need for replication and meta-analysis in translation and interpreting studies.
Keywords: utterance fluency, perceived fluency, consecutive interpreting, replication, meta-analysis
Article outline
  • Introduction
  • Literature review
    • Relationship between utterance fluency and perceived fluency of interpreting
    • The target study for replication: Han et al. (2020)
    • Replication research
    • Meta-analysis
  • Methods
    • The replication study
    • Data source
    • Calculation of utterance fluency measures
    • Measurement of perceived fluency
    • Data analysis
    • Mini meta-analysis
  • Results
    • The replication results
    • Replication analysis
    • Mini meta-analysis results
  • Discussion
  • Conclusion
  • Notes
  • References
Published online: 15 November 2022
DOI logo
https://doi.org/10.1075/tis.20091.han
References (43)
References
Allen, Mike and Raymond Preiss. 1993. “Replication and meta-analysis: A necessary connection.” Journal of Social Behavior & Personality 8(6): 9–20.Google Scholar
Asendorpf, Jens B., et al. 2013. “Recommendations for increasing replicability in psychology.” European Journal of Personality 27(2): 108–119. DOI logoGoogle Scholar
Borenstein, Michael, Larry V. Hedges, Julian Higgins, and Hannah R. Rothstein. 2009. Introduction to meta-analysis. Chichester: Wiley. DOI logoGoogle Scholar
Brandt, Mark J. et al. 2014. “The Replication Recipe: What makes for a convincing replication?” Journal of Experimental Social Psychology 501: 217–224. DOI logoGoogle Scholar
Braver, Sanford L., Felix J. Thoemmes, and Robert Rosenthal. 2014. “Continuously cumulating meta-analysis and replicability.” Perspectives on Psychological Science 9(3): 333–342. DOI logoGoogle Scholar
Cecot, Michela. 2001. “Pauses in simultaneous interpretation: A contrastive analysis of professional interpreters’ performances.” The Interpreters’ Newsletter 111: 63–85.Google Scholar
Collard, Camille and Bart Defrancq. 2020. Disfluencies in simultaneous interpreting a corpus-based study with special reference to sex. In New empirical perspectives on translation and interpreting, ed. by Lore Vandevoorde, Joke Daems and Bart Defrancq, 264–299. New York: Routledge. DOI logoGoogle Scholar
Corey, David M., William P. Dunlap, and Michael J. Burke. 1998. “Averaging correlations: Expected values and bias in combined Pearson rs and Fisher’s z transformations.” Journal of General Psychology 125(3): 245–262. DOI logoGoogle Scholar
Crandall, Christian S. and Jeffrey W. Sherman. 2016. “On the scientific superiority of conceptual replications for scientific progress.” Journal of Experimental Social Psychology 661: 93–99. DOI logoGoogle Scholar
Easley, Richard W., Charles S. Madden, and Mark G. Dunn. 2000. “Conducting marketing science: The role of replication in the research process.” Journal of Business Research 48(1): 83–92. DOI logoGoogle Scholar
Gile, Daniel. 2009. “Interpreting studies: a critical view from within.” MonTI 11: 135–155. DOI logoGoogle Scholar
Goh, Jin X., Judith A. Hall, and Robert Rosenthal. 2016. “Mini meta-analysis of your own studies: Some arguments on why and a primer on how.” Social and Personality Psychology Compass 10(10): 535–549. DOI logoGoogle Scholar
Han, Chao and Ke-rui An. 2021. “Using unfilled pauses to measure (dis)fluency in English-Chinese consecutive interpreting: in search of an optimal pause threshold(s).” Perspectives 29(6): 917–933. DOI logoGoogle Scholar
Han, Chao, Sijia Chen, Rongbo Fu, and Qing Fan. 2020. “Modeling the relationship between utterance fluency and raters’ perceived fluency of consecutive interpreting.” Interpreting 22(2): 211–237. DOI logoGoogle Scholar
Han, Chao. 2015. “(Para)linguistic correlates of perceived fluency in English-to-Chinese simultaneous interpretation.” International Journal of Comparative Literature and Translation Studies 3(4): 32–37. DOI logoGoogle Scholar
Hoeppner, Sven. 2019. “A note on replication analysis.” International Review of Law and Economics 591: 98–102. DOI logoGoogle Scholar
Hubbard, Raymond and J. Scott Armstrong. 1994. “Replications and extensions in marketing: Rarely published but quite contrary.” International Journal of Research in Marketing 11(3): 233–248. DOI logoGoogle Scholar
Lee, Jieun. 2008. “Rating scales for interpreting performance assessment.” The Interpreter and Translator Trainer 2(2): 165–184. DOI logoGoogle Scholar
Mead, Peter. 2000. “Control of pauses by trainee interpreters in their A and B languages.” The Interpreters’ Newsletter 101: 89–102.Google Scholar
. 2005. “Methodological issues in the study of interpreters’ fluency.” The Interpreters’ Newsletter 131: 39–63.Google Scholar
Mellinger, Christopher D. and Thomas A. Hanson. 2017. Quantitative Research Methods in Translation and Interpreting Studies. New York: Routledge. DOI logoGoogle Scholar
. 2019. “Meta-analyses of simultaneous interpreting and working memory.” Interpreting 21(2): 165–195. DOI logoGoogle Scholar
. 2020. “Meta-analysis and replication in interpreting studies.” Interpreting 22(1): 140–149. DOI logoGoogle Scholar
Park, Jung Hwan, Olesya Venger, Doo Yeon Park, and Leonard N. Reid. 2015. “Replication in advertising research, 1980–2012: A longitudinal analysis of leading advertising journals.” Journal of Current Issues and Research in Advertising 36(2): 115–135. DOI logoGoogle Scholar
Pöchhacker, Franz. 2011. “Replication in research on quality in conference interpreting.” T&I Review 11: 35–58.Google Scholar
Pöchhacker, Franz and Cornelia Zwischenberger. 2010. “Survey on quality and role: Conference interpreters’ expectations and self-perceptions.” aiic.org [URL] Last accessed 20 January 2022.
Porte, Graeme and Keith Richards. 2012. “Replication in second language writing research.” Journal of Second Language Writing 21 (3): 284–293. DOI logoGoogle Scholar
Porte, Graeme. 2013. “Who needs replication?” Computer assisted language instruction Consortium Journal 30 (1): 10–15. [URL]. DOI logo
Rennert, Sylvi. 2010. “The impact of fluency on the subjective assessment of interpreting quality.” The Interpreters’ Newsletter 151: 101–115.Google Scholar
Rosenthal, Robert and Ralph L. Rosnow. 1984. “Applying Hamlet’s question to the ethical conduct of research: A conceptual addendum.” American Psychologist 39(5): 561–563. DOI logoGoogle Scholar
Saldanha, Gabriela and Sharon O’Brien. 2013. Research methodologies in translation studies. London: Routledge. DOI logoGoogle Scholar
Segalowitz, Norman. 2010. Cognitive bases of second language fluency. New York: Routledge. DOI logoGoogle Scholar
Song, Shuxian. 2020. Fluency in simultaneous interpreting of trainee interpreters: The perspectives of cognitive, utterance and perceived fluency. Unpublished PhD dissertation. The Hong Kong Polytechnic University. [URL]
Stanley, David J. and Jeffrey R. Spence. 2014. “Expectations for replications: Are yours realistic?” Perspectives on Psychological Science 9(3): 305–318. DOI logoGoogle Scholar
Tavakoli, Parveneh and Peter Skehan. 2005. Strategic planning, task structure and performance testing. In Planning and task performance in a second language, ed. by Rod Ellis, 239–273. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Valentine, Jeffrey C. et al. 2011. “Replication in prevention science.” Prevention Science 12, 103 (2011). DOI logoGoogle Scholar
Van Aert, Bobbie C. M. and Marcel A. L. M. van Assen. 2018. “Examining reproducibility in psychology: A hybrid method for combing a statistically significant original study and a replication.” Behavior Research Methods 501: 1515–1539. DOI logoGoogle Scholar
Wang, Binhua and Li Tao. 2015. “An empirical study of pauses in Chinese-English simultaneous interpreting.” Perspectives 23(1): 124–142. DOI logoGoogle Scholar
Wen, Hao and Yanping Dong. 2019. “How does interpreting experience enhance working memory and short-term memory: A meta-analysis.” Journal of Cognitive Psychology 31(8): 769–784. DOI logoGoogle Scholar
Wright, Ben D. and John M. Linacre. 1994. “Reasonable mean-square fit values.” Rasch Measurement Transactions 8(3): 370. [URL]
Yang, Liu-yan. 2015. “An exploratory study of fluency in English output of Chinese consecutive interpreting learners.” Journal of Zhejiang International Studies University 11, 60–68.Google Scholar
Cited by (1)

Cited by one other publication

Han, Chao
2023. Interrogating the predictive validity of aptitude testing for interpreting: a systematic methodological review. The Interpreter and Translator Trainer 17:1 ► pp. 7 ff. DOI logo

This list is based on CrossRef data as of 5 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.