Article published In:
Translation and Interpreting Studies: Online-First Articles

Relating utterance fluency to perceived fluency of interpreting

A partial replication and a mini meta-analysis

ORCID logo | Xiamen University
ORCID logo | Xiamen University

The prospect of automated scoring for interpreting fluency has prompted investigations into the predictability of human raters’ perceived fluency based on acoustically measured utterance fluency. Recently, Han, Chen, Fu and Fan (2020) correlated ten utterance fluency measures with raters’ perceived fluency ratings. To verify previous correlational patterns, the present study partially replicated Han et al. (2020). Our analysis shows that most of the correlations observed in Han et al. (2020) were successfully replicated. To produce overall interim estimates of the true relationships, we conducted a mini meta-analysis of correlation coefficients reported in six relevant studies, informed by the “continuously cumulating meta-analysis” approach (Braver et al. 2014). We found that phonation time ratio, mean length of run, and speech rate had relatively strong correlations with perceived fluency. We discuss these findings in light of automated fluency assessment and the need for replication and meta-analysis in translation and interpreting studies.

Keywords: utterance fluency, perceived fluency, consecutive interpreting, replication, meta-analysis
Article outline
  • Introduction
  • Literature review
    • Relationship between utterance fluency and perceived fluency of interpreting
    • The target study for replication: Han et al. (2020)
    • Replication research
    • Meta-analysis
  • Methods
    • The replication study
    • Data source
    • Calculation of utterance fluency measures
    • Measurement of perceived fluency
    • Data analysis
    • Mini meta-analysis
  • Results
    • The replication results
    • Replication analysis
    • Mini meta-analysis results
  • Discussion
  • Conclusion
  • Notes
  • References
Published online: 15 November 2022
DOI logo
https://doi.org/10.1075/tis.20091.han
References
Allen, Mike and Raymond Preiss
1993 “Replication and meta-analysis: A necessary connection.” Journal of Social Behavior & Personality 8(6): 9–20.Google Scholar
Asendorpf, Jens B., et al
2013 “Recommendations for increasing replicability in psychology.” European Journal of Personality 27(2): 108–119. DOI logoGoogle Scholar
Borenstein, Michael, Larry V. Hedges, Julian Higgins, and Hannah R. Rothstein
2009 Introduction to meta-analysis. Chichester: Wiley. DOI logoGoogle Scholar
Brandt, Mark J. et al.
2014 “The Replication Recipe: What makes for a convincing replication?” Journal of Experimental Social Psychology 501: 217–224. DOI logoGoogle Scholar
Braver, Sanford L., Felix J. Thoemmes, and Robert Rosenthal
2014 “Continuously cumulating meta-analysis and replicability.” Perspectives on Psychological Science 9(3): 333–342. DOI logoGoogle Scholar
Cecot, Michela
2001 “Pauses in simultaneous interpretation: A contrastive analysis of professional interpreters’ performances.” The Interpreters’ Newsletter 111: 63–85.Google Scholar
Collard, Camille and Bart Defrancq
2020 Disfluencies in simultaneous interpreting a corpus-based study with special reference to sex. In New empirical perspectives on translation and interpreting, ed. by Lore Vandevoorde, Joke Daems and Bart Defrancq, 264–299. New York: Routledge. DOI logoGoogle Scholar
Corey, David M., William P. Dunlap, and Michael J. Burke
1998 “Averaging correlations: Expected values and bias in combined Pearson rs and Fisher’s z transformations.” Journal of General Psychology 125(3): 245–262. DOI logoGoogle Scholar
Crandall, Christian S. and Jeffrey W. Sherman
2016 “On the scientific superiority of conceptual replications for scientific progress.” Journal of Experimental Social Psychology 661: 93–99. DOI logoGoogle Scholar
Easley, Richard W., Charles S. Madden, and Mark G. Dunn
2000 “Conducting marketing science: The role of replication in the research process.” Journal of Business Research 48(1): 83–92. DOI logoGoogle Scholar
Gile, Daniel
2009 “Interpreting studies: a critical view from within.” MonTI 11: 135–155. DOI logoGoogle Scholar
Goh, Jin X., Judith A. Hall, and Robert Rosenthal
2016 “Mini meta-analysis of your own studies: Some arguments on why and a primer on how.” Social and Personality Psychology Compass 10(10): 535–549. DOI logoGoogle Scholar
Han, Chao and Ke-rui An
2021 “Using unfilled pauses to measure (dis)fluency in English-Chinese consecutive interpreting: in search of an optimal pause threshold(s).” Perspectives 29(6): 917–933. DOI logoGoogle Scholar
Han, Chao, Sijia Chen, Rongbo Fu, and Qing Fan
2020 “Modeling the relationship between utterance fluency and raters’ perceived fluency of consecutive interpreting.” Interpreting 22(2): 211–237. DOI logoGoogle Scholar
Han, Chao
2015 “(Para)linguistic correlates of perceived fluency in English-to-Chinese simultaneous interpretation.” International Journal of Comparative Literature and Translation Studies 3(4): 32–37. DOI logoGoogle Scholar
Hoeppner, Sven
2019 “A note on replication analysis.” International Review of Law and Economics 591: 98–102. DOI logoGoogle Scholar
Hubbard, Raymond and J. Scott Armstrong
1994 “Replications and extensions in marketing: Rarely published but quite contrary.” International Journal of Research in Marketing 11(3): 233–248. DOI logoGoogle Scholar
Lee, Jieun
2008 “Rating scales for interpreting performance assessment.” The Interpreter and Translator Trainer 2(2): 165–184. DOI logoGoogle Scholar
Mead, Peter
2000 “Control of pauses by trainee interpreters in their A and B languages.” The Interpreters’ Newsletter 101: 89–102.Google Scholar
2005 “Methodological issues in the study of interpreters’ fluency.” The Interpreters’ Newsletter 131: 39–63.Google Scholar
Mellinger, Christopher D. and Thomas A. Hanson
2017 Quantitative Research Methods in Translation and Interpreting Studies. New York: Routledge. DOI logoGoogle Scholar
2019 “Meta-analyses of simultaneous interpreting and working memory.” Interpreting 21(2): 165–195. DOI logoGoogle Scholar
2020 “Meta-analysis and replication in interpreting studies.” Interpreting 22(1): 140–149. DOI logoGoogle Scholar
Park, Jung Hwan, Olesya Venger, Doo Yeon Park, and Leonard N. Reid
2015 “Replication in advertising research, 1980–2012: A longitudinal analysis of leading advertising journals.” Journal of Current Issues and Research in Advertising 36(2): 115–135. DOI logoGoogle Scholar
Pöchhacker, Franz
2011 “Replication in research on quality in conference interpreting.” T&I Review 11: 35–58.Google Scholar
Pöchhacker, Franz and Cornelia Zwischenberger
2010 “Survey on quality and role: Conference interpreters’ expectations and self-perceptions.” aiic.org [URL] Last accessed 20 January 2022.
Porte, Graeme and Keith Richards
2012 “Replication in second language writing research.” Journal of Second Language Writing 21 (3): 284–293. DOI logoGoogle Scholar
Porte, Graeme
2013 “Who needs replication?” Computer assisted language instruction Consortium Journal 30 (1): 10–15. [URL]. DOI logo
Rennert, Sylvi
2010 “The impact of fluency on the subjective assessment of interpreting quality.” The Interpreters’ Newsletter 151: 101–115.Google Scholar
Rosenthal, Robert and Ralph L. Rosnow
1984 “Applying Hamlet’s question to the ethical conduct of research: A conceptual addendum.” American Psychologist 39(5): 561–563. DOI logoGoogle Scholar
Saldanha, Gabriela and Sharon O’Brien
2013 Research methodologies in translation studies. London: Routledge. DOI logoGoogle Scholar
Segalowitz, Norman
2010 Cognitive bases of second language fluency. New York: Routledge. DOI logoGoogle Scholar
Song, Shuxian
2020 Fluency in simultaneous interpreting of trainee interpreters: The perspectives of cognitive, utterance and perceived fluency. Unpublished PhD dissertation. The Hong Kong Polytechnic University. [URL]
Stanley, David J. and Jeffrey R. Spence
2014 “Expectations for replications: Are yours realistic?” Perspectives on Psychological Science 9(3): 305–318. DOI logoGoogle Scholar
Tavakoli, Parveneh and Peter Skehan
2005 Strategic planning, task structure and performance testing. In Planning and task performance in a second language, ed. by Rod Ellis, 239–273. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Valentine, Jeffrey C. et al.
2011 “Replication in prevention science.” Prevention Science 12, 103 (2011) DOI logoGoogle Scholar
Van Aert, Bobbie C. M. and Marcel A. L. M. van Assen
2018 “Examining reproducibility in psychology: A hybrid method for combing a statistically significant original study and a replication.” Behavior Research Methods 501: 1515–1539. DOI logoGoogle Scholar
Wang, Binhua and Li Tao
2015 “An empirical study of pauses in Chinese-English simultaneous interpreting.” Perspectives 23(1): 124–142. DOI logoGoogle Scholar
Wen, Hao and Yanping Dong
2019 “How does interpreting experience enhance working memory and short-term memory: A meta-analysis.” Journal of Cognitive Psychology 31(8): 769–784. DOI logoGoogle Scholar
Wright, Ben D. and John M. Linacre
1994 “Reasonable mean-square fit values.” Rasch Measurement Transactions 8(3): 370. [URL]
Yang, Liu-yan
2015 “An exploratory study of fluency in English output of Chinese consecutive interpreting learners.” Journal of Zhejiang International Studies University 11, 60–68.Google Scholar