Article published In:
InterpretingVol. 19:1 (2017) ► pp.47–68
Predicting judged fluency of consecutive interpreting from acoustic measures
Potential for automatic assessment and pedagogic implications
This experimental study of consecutive interpreting investigates whether: (1) there is any correlation between assessments of its fluency and accuracy; (2) judged fluency can be predicted from computer-based measurements like articulation rate. Ten raters judged six criteria of accuracy and fluency in two consecutive interpretations of the same recorded source speech, from Chinese A into English B, by 12 trainee interpreters (seven undergraduates, five MA students). The recorded interpretations were examined with the speech analysis tool PRAAT. From a computerized count of the pauses thus detected, together with disfluencies identified by raters, 12 acoustic measures of fluency were calculated. The advanced students were more fluent than the beginners; both groups were less fluent in the initial interpretation. Statistical analysis shows: (1) a strong positive correlation between judged accuracy and judged fluency; (2) strong correlations between judged fluency and objective fluency variables; (3) the usefulness of effective speech rate (number of syllables, excluding disfluencies, divided by total duration of speech production and pauses) as a predictor of judged fluency. Other important determinants of judged fluency were the number of filled pauses, articulation rate, and mean length of pause. Potential for developing automatic fluency assessment in consecutive interpreting is discussed, as are possible training implications.
This article is currently available as a sample article.
References
AIIC
(2002) Regulation governing admissions and language classification. Geneva: AIIC.
Bortfeld, H., Leon, S. D., Bloom, J. E., Schober, M. F. & Brennan, S. E
(1999) Which speakers are most disfluent in conversation and when? Proceedings ICPhS99 Satellite Meeting on Disfluency in Spontaneous Speech, 7–10.
Brumfit, C
(1984) Communicative methodology in language teaching: The roles of fluency and accuracy. Cambridge: Cambridge University Press.
Bühler, H
(1986) Linguistic (semantic) and extra-linguistic (pragmatic) criteria for the evaluation of conference interpretation and interpreters. Multilingua 5 (4), 231–235. 
Chambers, F
(1997) What do we mean by fluency? System 251, 535–544.
Chiaro, D. & Nocella, G
(2004) Interpreters’ perception of linguistic and non-linguistic factors affecting quality: A survey through the World Wide Web. Meta 49 (2), 278–293.
Collados Aís, A
(1998/2002) Quality assessment in simultaneous interpreting: The importance of nonverbal communication. In F. Pöchhacker & M. Shlesinger (Eds.), The interpreting studies reader. London/New York: Routledge, 327–336.
Collados Aís, A., Pradas Macías, M., Stévaux, E. & García Becerra, O
(Eds.) (2007) La evaluación de la calidad en interpretación simultánea: Parámetros de incidencia. Granada: Comares.
Cucchiarini, C., Strik, H. & Boves, L
(2000) Quantitative assessment of second language learners’ fluency by means of automatic speech recognition technology. Journal of the Acoustical Society of America 1071, 989–999.
(2002) Quantitative assessment of second language learners’ fluency: Comparisons between read and spontaneous speech. Journal of the Acoustical Society of America 1111, 2862–2873.
De Jong, N. H. & Wempe, T
(2009) Praat script to detect syllable nuclei and measure speech rate automatically. Behavior Research Methods 41 (2), 385–390.
Fillmore, C. J
(1979) On fluency. In C. J. Fillmore, D. Kempler & W. S. Wang (Eds.), Individual differences in language ability and language behaviour. New York: Academic Press, 85–101.
Freed, B. F
(1995) What makes us think that students who study abroad become fluent? In B. F. Freed (Ed.), Second language acquisition in a study-abroad context. Amsterdam: John Benjamins, 123–148.
Gile, D
(1995) Basic concepts and models for interpreter and translator training. Amsterdam: John Benjamins.
Goldman-Eisler, F
(1968) Psycholinguistics: Experiments in spontaneous speech. New York: Academic Press.
Grosjean, F
(1980) Temporal variables within and between languages. In H. W. Dechert & M. Raupach (Eds.), Towards a cross-linguistic assessment of speech production. Frankfurt: Lang, 39–53.
Grosjean, F. & Deschamps, A
(1975) Analyse contrastive des variables temporelles de l’Anglais et du Francais: Vitesse de parole et variables composantes, phénomènes d’hésitation. Phonetica 311, 144–184.
Kormos, J. & Dénes, M
(2004) Exploring measures and perceptions of fluency in the speech of second language learners. System 321, 145–164.
Kurz, I
(1993) Conference interpretation: Expectations of different user groups. The Interpreters’ Newsletter 51, 3–16.
(2001) Conference interpreting: Quality in the ears of the user. Meta 46 (2), 394–409.
(2003) Quality from the user perspective. In A. Collados Aís, M. Fernández Sanchez & D. Gile (Eds.), La evaluación de la calidad en interpretación: Investigación. Granada: Comares, 3–22.
Leeson, R
(1975) Fluency and language teaching. London: Longman.
Lennon, P
(1990) Investigating fluency in EFL: A quantitative approach. Language Learning 31, 387–417.
(2000) The lexical element in spoken second language fluency. In H. Riggenbach (Ed.), Perspectives on fluency. Michigan: The University of Michigan Press, 25–42.
Levelt, W. J. M
(1989) Speaking: From intention to articulation. Cambridge, MA: MIT Press.
Mead, P
(2000) Control of pauses by trainee interpreters in their A and B languages. The Interpreters’ Newsletter 101, 89–102.
(2005) Methodological issues in the study of interpreters’ fluency. The Interpreters’ Newsletter 131, 39–63.
Möhle, D
(1984) A comparison of the second language speech production of different native speakers. In H. W. Dechert, D. Möhle & M. Raupach (Eds.), Second language productions. Tübingen: Gunter Narr, 26–49.
Moser, P
Nation, P
(1989) Improving speaking fluency. System 31, 377–384.
Pinget, A., Bosker, H. R., Quené, H., Sanders, T. & De Jong, N. H
(2014) Native speakers’ perceptions of fluency and accent in L2 speech. Language Testing 31 (3), 349–365.
Pradas Macías, E. M
(2003) Repercusión del intraparámetro pausas silenciosas en la fluidez: Influencia en las expectativas y en la evaluación de la calidad en interpretación simultánea. PhD dissertation, University of Granada.
(2007) La incidencia del parámetro fluidez. In A. Collados Aís, M. Pradas Macías, E. Stévaux & O. García Becerra (Eds.), La evaluación de la calidad en interpretación simultánea: Parámetros de incidencia. Granada: Comares, 53–70.
Pöchhacker, F
(1993) On the science of interpretation. The Interpreters’ Newsletter 51, 52–59.
(2012) Interpreting quality: Global professional standards? In W. Ren (Ed.), Interpreting in the age of globalization: Proceedings of the 8th National Conference and International Forum on Interpreting. Beijing: Foreign Language Teaching and Research Press, 305–318.
Rennert, S
(2010) The impact of fluency on the subjective assessment of interpreting quality. The Interpreters’ Newsletter 151, 101–115.
Riggenbach, H
(1991) Toward an understanding of fluency: A microanalysis of non-native speaker conversations. Discourse Processes 141, 423–441.
Schmidt, R
(1992) Psychological mechanisms underlying second language fluency. Studies in Second Language Acquisition 141, 357–385.
Tavakoli, P. & Skehan, P
(2005) Strategic planning, task structure and performance testing. In R. Ellis (Ed.), Planning and task performance in a second language. Amsterdam: John Benjamins, 239–273.
Towell, R., Hawkins, R. & Bazergui, N
(1996) The development of fluency in advanced learners of French. Applied Linguistics 11, 84–119.
Townshend, B., Bernstein, J., Todic, O. & Warren, E
(1998) Estimation of spoken language proficiency. Proceedings of the ESCA Workshop Speech Technology in Language Learning (STiLL 98), 179–182.
Yu, W. T. & van Heuven, V. J
(2013) Effects of immediate repetition at different stages of consecutive interpreting: An experimental study. In: S. Aalberse & A. Auer (Eds.), Linguistics in the Netherlands 2013. Amsterdam: John Benjamins, 201–213.
Zhou, D
(2006) A study on the effects of input frequency and output frequency. Modern Foreign Languages 291, 154–163.
Zwischenberger, C. & Pöchhacker, F
(2010) Survey on quality and role: Conference interpreters’ expectations and self-perceptions. Communicate! [URL] (accessed 21 January 2013).
Cited by
Cited by 24 other publications
Abdel Latif, Muhammad M. M.
2020. Translation and Interpreting Assessment Research. In Translator and Interpreter Education Research [New Frontiers in Translation Studies, ], ► pp. 61 ff. 
Castillo-Rodríguez, Cristina, Cristina Toledo-Báez & Miriam Seghiri
2023. Teaching interpreting in times of Covid: perspectives, experience and satisfaction. Revista de Lingüística y Lenguas Aplicadas 18 ► pp. 19 ff.
Chang, Chia-chien
2018. English Language Needs of Chinese/English Interpreting Students: an Error Analysis of the Chinese-to-English Short Consecutive Interpreting Test. English Teaching & Learning 42:3-4 ► pp. 207 ff.
Chen, Jing & Chao Han
2021. Testing and Assessment of Interpreting in China: An Overview. In Testing and Assessment of Interpreting [New Frontiers in Translation Studies, ], ► pp. 1 ff.
Chen, Sijia & Jan-Louis Kruger
2023. The effectiveness of computer-assisted interpreting. Translation and Interpreting Studies 18:3 ► pp. 399 ff.
Gao, Fei
2023. Getting the message in ‘Sound’ across at conference interpreting: a case study on rendering prosodic emphasis. Perspectives 31:6 ► pp. 1033 ff.
Gieshoff, Anne Catherine
2021. The impact of visible lip movements on silent pauses in simultaneous interpreting. Interpreting. International Journal of Research and Practice in Interpreting 23:2 ► pp. 168 ff.
Gieshoff, Anne Catherine & Andrea Hunziker Heeb
Han, Chao
2022. Interpreting testing and assessment: A state-of-the-art review. Language Testing 39:1 ► pp. 30 ff.
Han, Chao & Kerui An
2021. Using unfilled pauses to measure (dis)fluency in English-Chinese consecutive interpreting: in search of an optimal pause threshold(s). Perspectives 29:6 ► pp. 917 ff.
Han, Chao, Sijia Chen, Rongbo Fu & Qin Fan
2020. Modeling the relationship between utterance fluency and raters’ perceived fluency of consecutive
interpreting. Interpreting. International Journal of Research and Practice in Interpreting 22:2 ► pp. 211 ff.
Han, Chao & Liuyan Yang
2023. Relating utterance fluency to perceived fluency of interpreting. Translation and Interpreting Studies 18:3 ► pp. 421 ff.
Kuang, Huolingxiao & Binghan Zheng
2022. How does interpreting performance correlate with note-taking process, note-taking product and note-reading process? An eye-tracking and pen-recording study. Across Languages and Cultures 23:2 ► pp. 167 ff.
Li, Xinguang, Xiaoning Li, Shuai Chen, Shanxian Ma & Fenfang Xie
2022. Neural-based automatic scoring model for Chinese-English interpretation with a multi-indicator assessment. Connection Science 34:1 ► pp. 1638 ff.
Lin, Yumeng, Qianxi Lv & Junying Liang
2018. Predicting Fluency With Language Proficiency, Working Memory, and Directionality in Simultaneous Interpreting. Frontiers in Psychology 9
Lu, Xiaolei & Chao Han
2023. Automatic assessment of spoken-language interpreting based on machine-translation evaluation metrics. Interpreting. International Journal of Research and Practice in Interpreting 25:1 ► pp. 109 ff.
Sawyer, David B.
2022. Review of Chen & Han (2021): Testing and assessment of interpreting: Recent developments in China. Interpreting. International Journal of Research and Practice in Interpreting 24:1 ► pp. 155 ff.
Shen, Mingxia & Junying Liang
2021. Self-repair in consecutive interpreting: similarities and differences between professional interpreters and student interpreters. Perspectives 29:5 ► pp. 761 ff.
Wang, Xiaoman & Binhua Wang
2022. Identifying fluency parameters for a machine-learning-based automated interpreting assessment system. Perspectives ► pp. 1 ff.
Wang, Xiaoman & Lu Yuan
2023. Machine-learning based automatic assessment of communication in interpreting. Frontiers in Communication 8
Wu, Zhiwei
2021. Chasing the Unicorn? The Feasibility of Automatic Assessment of Interpreting Fluency. In Testing and Assessment of Interpreting [New Frontiers in Translation Studies, ], ► pp. 143 ff.
Yenkimaleki, Mahmood & Vincent J. van Heuven
2023. Effect of pedagogic intervention in enhancing speech fluency by EFL students: A longitudinal study. Language Teaching Research
Yenkimaleki, Mahmood, Vincent J. van Heuven & Mostafa Hosseini
2023. The effect of fluency strategy training on interpreter trainees’ speech fluency: Does content familiarity matter?. Speech Communication 146 ► pp. 1 ff.
Yu, Wenting & Vincent J. van Heuven
2021. Quantitative Correlates as Predictors of Judged Fluency in Consecutive Interpreting: Implications for Automatic Assessment and Pedagogy. In Testing and Assessment of Interpreting [New Frontiers in Translation Studies, ], ► pp. 117 ff.
This list is based on CrossRef data as of 31 march 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.