Article In:
Interpreting: Online-First Articles

Fundamental frequency as an acoustic mirror of interpreters’ cognitive states

| Zhejiang Gongshang University
| Ghent University
The present study proposes the acoustic parameter of fundamental frequency (F0) as an alternative method for gauging interpreters’ cognitive load and fatigue during simultaneous interpreting (SI). The data collected from an English-Chinese SI experiment reported in Shao and Chai (2021) were re-analysed for the purposes of this study. Cognitive load was measured as the number of undelivered information chunks within the ear-voice span (EVS), whereas fatigue was assumed to increase with time. F0 variables were correlated with cognitive load within the EVS of selected sentences and with fatigue in one-minute segments distributed over the entire interpreting performance. The results show that cognitive constructs are associated in various ways with F0-related variables. Higher F0 peaks and a wider F0 range appear to be associated with higher cognitive load in a majority of interpreters. A higher mean F0 and, to a lesser extent, higher F0 peaks are associated with fatigue. The findings suggest that F0 could be used as a promising indicator of cognitive load and fatigue in SI.
Keywords: fundamental frequency, cognitive load, fatigue
Article outline
  • 1.Introduction
  • 2.Fundamental frequency and its associated cognitive variables
  • 3.Cognitive load and fatigue in simultaneous interpreting
    • 3.1Cognitive load
    • 3.2Fatigue
    • 3.3The measurement gap
  • 4.Data and methods
    • 4.1Data
    • 4.2Areas of interest
    • 4.3Statistical analysis
  • 5.Results
    • 5.1Fatigue and F0
    • 5.2Cognitive load and F0
  • 6.Discussion
  • 7.Conclusion
  • Note
  • References
This content is being prepared for publication; it may be subject to changes.
References (94)
References
Aaronson, L. S., Teel, C. S., Cassmeyer, V., Neuberger, G. B., Pallikkathayil, L., Pierce, J., Press, A. N., Williams, P. D. & Wingate, A. (1999). Defining and measuring fatigue. Image – The Journal of Nursing Scholarship 31 (1), 45–50. DOI logoGoogle Scholar
Ackerman, P. L. & Kanfer, R. (2009). Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions. Journal of Experimental Psychology. Applied 15 (2), 163–181. DOI logoGoogle Scholar
AIIC (2023). Schools directory. [URL] (accessed 30 June 2023).
Ayres, P., Lee, J. Y., Paas, F. & Van Merriënboer, J. (2021). The validity of physiological measures to identify differences in intrinsic cognitive load. Frontiers in Psychology 12 1, 702538. DOI logoGoogle Scholar
Behrens, M., Gube, M., Chaabene, H., Prieske, O., Zenon, A., Broscheid, K. C., Schega, L., Husmann, F. & Weippert, M. (2023). Fatigue and human performance: An updated framework. Sports Medicine (Auckland, N.Z.) 53 (1), 7–31. DOI logoGoogle Scholar
Benson, P. (1995). Analysis of the acoustic correlates of stress from an operational aviation emergency. In I. Trancoso & R. Moore (Eds.), Proceedings of the ESCA-NATO Tutorial and Research Workshop on Speech under Stress. Lisbon: INESC, 61–64.Google Scholar
Bjørheim, F., Pavlou, D. G. & Siriwardane, S. C. (2022). Nonlinear fatigue life prediction model based on the theory of the S–N fatigue damage envelope. Fatigue & Fracture of Engineering Material & Structures 45 (5), 1480–1493. DOI logoGoogle Scholar
Boersma, P. & Weenink, D. (2023). Praat: Doing phonetics by computer, version 6.3.09. [URL] (accessed 2 March 2023).
Boos, M., Kobi, M., Elmer, S. & Jäncke, L. (2022). The influence of experience on cognitive load during simultaneous interpretation. Brain and Language 234 1, 105185. DOI logoGoogle Scholar
Bortfeld, H., Leon, S. D., Bloom, J. E., Schober, M. F. & Brennan, S. E. (2001). Disfluency rates in conversation: Effects of age, relationship, topic, role, and gender. Language and Speech 44 (2), 123–147. DOI logoGoogle Scholar
Boyer, S., Paubel, P. V., Ruiz, R., El Yagoubi, R. & Daurat, A. (2018). Human voice as a measure of mental load level. Journal of Speech, Language, and Hearing Research 61 (11), 2722–2734. DOI logoGoogle Scholar
Cañas, J. J., Muñoz-de-Escalona, E. & Morales-Guaman, J. F. (2021). Fundamental frequency as an alternative method for assessing mental fatigue of distance learning teachers. In L. Longo & M. C. Leva (Eds.), Human mental workload: Models and applications. H-WORKLOAD 2021. Cham: Springer, 45–59. DOI logoGoogle Scholar
Carroll, T., Nix, J., Hunter, E., Emerich, K., Titze, I. & Abaza, M. (2006). Objective measurement of vocal fatigue in classical singers: A vocal dosimetry pilot study. Otolaryngology-Head and Neck Surgery 135 (4), 595–602. DOI logoGoogle Scholar
Chen, S. (2017). The construct of cognitive load in interpreting and its measurement. Perspectives 25 (4), 640–657. DOI logoGoogle Scholar
Chevalier, L. & Gile, D. (2015). Interpreting quality: A case study of spontaneous reactions. Forum 13 (1), 1–26. DOI logoGoogle Scholar
Christodoulides, G. (2016). Effects of cognitive load on speech production and perception. PhD dissertation, Université Catholique de Louvain.
Cowan, N. (2000). The magic number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Science 24 1, 87–114. DOI logoGoogle Scholar
De Jong, T. (2010). Cognitive load theory, educational research, and instructional design: Some food for thought. Instructional Science 38 (2), 105–134. DOI logoGoogle Scholar
Evgrafova, K., Evdokimova, V., Skrelin, P. & Chukaeva, T. (2016). Vocal fatigue in voice professionals: Collecting data and acoustic analysis. ExLing 2016: Proceedings of 7th Tutorial and Research Workshop on Experimental Linguistics, 27 June – 2 July 2016, Saint Petersburg, Russia, 59–62.Google Scholar
Gawron, V. J. (2016). Overview of self-reported measures of fatigue. The International Journal of Aviation Psychology 26 (3/4), 120–131. DOI logoGoogle Scholar
Gieshoff, A. C. & Albl-Mikasa, M. (2022). Interpreting accuracy revisited: A refined approach to interpreting performance analysis, Perspectives 32 (2), 210–228. DOI logoGoogle Scholar
Gile, D. (1999). Testing the effort models’ tightrope hypothesis in simultaneous interpreting – a contribution. Hermes 23 1, 153–172.Google Scholar
(2021). The effort models of interpreting as a didactic construct. In R. Muñoz Martín, S. Sun & D. Li (Eds.), Advances in cognitive translation studies. Singapore: Springer, 139–160. DOI logoGoogle Scholar
Grassmann, M., Vlemincx, E., von Leupoldt, A., Mittelstädt, J. M. & Van den Bergh, O. (2016). Respiratory changes in response to cognitive load: A systematic review. Neural Plasticity 2016, 8146809. DOI logoGoogle Scholar
Griffin, G. R. & Williams, C. E. (1987). The effects of different levels of task complexity on three vocal measures. Aviation, Space, and Environmental Medicine 58 1, 1165–1170.Google Scholar
Habay, J., Proost, M., De Wachter, J., Díaz-García, J., De Pauw, K., Meeusen, R., Van Cutsem, J. & Roelands, B. (2021). Mental fatigue-associated decrease in table tennis performance: Is there an electrophysiological signature? International Journal of Environmental Research and Public Health 18 (24), 12906. DOI logoGoogle Scholar
Hart, S. G. & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload, New York: North-Holland, 139–183. DOI logoGoogle Scholar
Hebbar, P. A., Bhattacharya, K., Prabhakar, G., Pashilkar, A. A. & Biswas, P. (2021). Correlation between physiological and performance-based metrics to estimate pilots’ cognitive workload. Frontiers in Psychology 12 1, 555446. DOI logoGoogle Scholar
Hirst, D. J. & De Looze, C. (2021). Measuring speech: Fundamental frequency and pitch. In R. Knight & J. Setter (Eds.), The Cambridge handbook of phonetics. Cambridge: Cambridge University Press, 336–361. DOI logoGoogle Scholar
Hoza, J. (2010). Team interpreting as collaboration and interdependence. Alexandria, VA: RID Press.Google Scholar
Hubscher-Davidson, S. (2020). Ethical stress in the translation and interpreting profession. In K. Koskinen & N. K. Pokorn (Eds.), The Routledge handbook of translation and ethics. Abingdon: Routledge, 415–430. DOI logoGoogle Scholar
Huttunen, K., Keränen, H., Väyrynen, E., Pääkkönen, R. & Leino, T. (2011). Effect of cognitive load on speech prosody in aviation: Evidence from military simulator flights. Applied Ergonomics 42 (2), 348–357. DOI logoGoogle Scholar
Hyönä, J., Tommola, J. & Alaja, A.-M. (1995). Pupil dilation as a measure of processing load in simultaneous interpretation and other language tasks. Quarterly Journal of Experimental Psychology Section A 48 (3), 598–612. DOI logoGoogle Scholar
Jacquet, T., Lepers, R., Poulin-Charronnat, B., Bard, P., Pfister, P. & Pageaux, B. (2021). Mental fatigue induced by prolonged motor imagery increases perception of effort and the activity of motor areas. Neuropsychologia 150 1, 107701. DOI logoGoogle Scholar
Johannes, B., Wittels, P., Enne, R., Eisinger, R., Castro, C. A., Thomas, J. L., Adler, A. B. & Gerzer, R. (2007). Non-linear function model of voice pitch dependency on physical and mental load. European Journal of Applied Physiology 101 1, 267–276. DOI logoGoogle Scholar
Kauffman, B. Y., Garey, L., Nordan, A., Jardin, C., Mayorga, N. A., Robles, Z. & Zvolensky, M. J. (2019). The development and initial validation of the fatigue sensitivity questionnaire. Cognitive Behaviour Therapy 48 (5), 419–429. DOI logoGoogle Scholar
Kitch, J. A. & Oates, J. (1994). The perceptual features of vocal fatigue as self-reported by a group of actors and singers. Journal of Voice 8 (3), 207–214. DOI logoGoogle Scholar
Korpal, P. (2017). Linguistic and psychological indicators of stress in simultaneous interpreting. Poznań: Wydawnictwo Naukowe UAM.Google Scholar
Krejtz, K., Duchowski, A. T., Niedzielska, A., Biele, C. & Krejtz, I. (2018). Eye tracking cognitive load using pupil diameter and microsaccades with fixed gaze. PloS ONE 13 (9), e0203629. DOI logoGoogle Scholar
Kurz, I. (2001). Conference interpreting: Quality in the ears of the user. Meta 46 (2), 394–409. DOI logoGoogle Scholar
(2003). Physiological stress during simultaneous interpreting: A comparison of experts and novices. The Interpreters’ Newsletter 12 1, 51–67.Google Scholar
Lambooij, M. T. M., IJsselsteijn, W. A., Fortuin, M. F. & Heynderickx, I. E. J. (2009). Visual discomfort and visual fatigue in stereoscopic displays: A review. Journal of Imaging Science and Technology 53 (3), 1–14. DOI logoGoogle Scholar
Laukkanen, A.-M., Ilomäki, I., Leppänen, K. & Vilkman, E. (2008). Acoustic measures and self-reports of vocal fatigue by female teachers. Journal of Voice 22 (3), 283–289. DOI logoGoogle Scholar
Lewis, M. (1997). Pedagogical implications of the lexical approach. In J. Coady & T. Huckin (Eds.), Second language vocabulary acquisition: A rationale for pedagogy. Cambridge: Cambridge University Press, 255–270.Google Scholar
Liang, J., Fang, Y., Lv, Q. & Liu, H. (2017). Dependency distance differences across interpreting types: Implications for cognitive demand. Frontiers in Psychology 8 1, 2132. DOI logoGoogle Scholar
Lim, J., Wu, W., Wang, J., Detre, J., Dinges, D. & Rao, H. (2010). Imaging brain fatigue from sustained mental workload: An ASL perfusion study of the time-on-task effect. Neuroimage 49 1, 3426–3425. DOI logoGoogle Scholar
Lively, S. E., Pisoni, D. B., Van Summers, W. & Bernacki, R. H. (1993). Effects of cognitive workload on speech production: Acoustic analyses and perceptual consequences. Journal of the Acoustical Society of America 93 (5), 2962–2973. DOI logoGoogle Scholar
Mehta, R. K. & Parasuraman, R. (2013). Neuroergonomics: A review of applications to physical and cognitive work. Frontiers in Human Neuroscience 7 1, 889. DOI logoGoogle Scholar
Melnicuk, V., Thompson, S., Jennings, P. & Birrell, S. (2021). Effect of cognitive load on drivers’ state and task performance during automated driving: Introducing a novel method for determining stabilisation time following take-over of control. Accident Analysis & Prevention 151 1, 105967. DOI logoGoogle Scholar
Mizuno, A. (2017). Simultaneous interpreting and cognitive constraints. Journal of College of Literature, Aoyama Gakuin University 58 1, 1–28.Google Scholar
Morris, T. L. & Miller, J. C. (1996). Electrooculographic and performance indices of fatigue during simulated flight. Biological Psychology 42 (3), 343–360. DOI logoGoogle Scholar
Moser-Mercer, B. (2003). Remote interpreting: Assessment of human factors and performance parameters. [URL] (accessed 21 June 2021)
Moser-Mercer, B., Künzli, A. & Korac, M. (1998). Prolonged turns in interpreting: Effects on quality, physiological and psychological stress (Pilot study). Interpreting 3 1, 47–64. DOI logoGoogle Scholar
Nattinger, J. R. & DeCarrico, J. S. (1992). Lexical phrases and language teaching. New York: Oxford University Press.Google Scholar
Nilsson, E. J., Bärgman, J., Ljung, Aust M., Matthews, G. & Svanberg, B. (2022). Let complexity bring clarity: A multidimensional assessment of cognitive load using physiological measures. Frontiers in Neuroergonomics 3 1, 787295. DOI logoGoogle Scholar
O’Halpin, R. (2010). The perception and production of stress and intonation by children with cochlear implants. PhD dissertation, University of London.
Ots, N. (2021). Cognitive constraints on advance planning of sentence intonation. PLoS ONE 16 (11), e0259343. DOI logoGoogle Scholar
Paas, F., Renkl, A. & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist 38 1, 1–4. DOI logoGoogle Scholar
Pageaux, B. & Lepers, R. (2018). The effects of mental fatigue on sport-related performance. Progress in Brain Research 240 1, 291–315. DOI logoGoogle Scholar
Petsche, H., Etlinger, S. C. & Filz, O. (1993). Brain electrical mechanisms of bilingual speech management: An initial investigation. Electroencephalography and Clinical Neurophysiology 86 (6), 385–394. DOI logoGoogle Scholar
Plevoets, K. & Defrancq, B. (2016). The effect of informational load on disfluencies in interpreting: A corpus-based regression analysis. Translation and Interpreting Studies 11 (2), 202–224. DOI logoGoogle Scholar
Pilcher, J. J., Ginter, D. R. & Sadowsky, B. (1997). Sleep quality versus sleep quantity: Relationships between sleep and measures of health, well-being and sleepiness in college students. Journal of Psychosomatic Research 42 (6), 583–596. DOI logoGoogle Scholar
Pöchhacker, F. (2001). Quality assessment in conference and community interpreting. Meta 46 1, 410–425. DOI logoGoogle Scholar
Pöchhacker, F. & Zwischenberger, C. (2010). Survey on quality and role: Conference interpreters’ expectations and self-perception. [URL] (accessed 17 June 2024).
Pym, A. (2008). On omission in simultaneous interpreting: Risk analysis of a hidden effort. In G. Hansen, A. Chesterman & H. Gerzymisch-Arbogast (Eds.), Efforts and models in interpreting and translation research: A tribute to Daniel Gile. Amsterdam: John Benjamins, 83–105.Google Scholar
Remacle, A., Garnier, M., Gerber, S., David, C. & Petillon, C. (2018). Vocal change patterns during a teaching day: Inter- and intra-subject variability. Journal of Voice 32 (1), 57–63. DOI logoGoogle Scholar
Russell, A., Oates, J. & Greenwood, K. M. (1998). Prevalence of voice problems in teachers. Journal of Voice 12 (4), 467–479. DOI logoGoogle Scholar
Sáiz-Manzanares, M. C., Marticorena-Sánchez, R., Martín Antón, L. J., González-Díez, R. & Carbonero Martín, M. Á. (2023). Using eye tracking technology to analyse cognitive load in multichannel activities in university students. International Journal of Human-Computer Interaction 40 (12), 3263–3281. DOI logoGoogle Scholar
Schnotz, W. & Kürschner, C. (2007). A reconsideration of cognitive load theory. Educational Psychology Review 19 1, 469–508. DOI logoGoogle Scholar
Seeber, K. G. (2011). Cognitive load in simultaneous interpreting: Existing theories – new models. Interpreting 13 (2), 176–204. DOI logoGoogle Scholar
(2013). Cognitive load in simultaneous interpreting: Measures and methods. Target 25 (1), 18–33. DOI logoGoogle Scholar
Shao, Z. & Chai, M. (2021). The effect of cognitive load on simultaneous interpreting performance: An empirical study at the local level. Perspectives 29 (5), 778–794. DOI logoGoogle Scholar
Smets, E. M., Garssen, B., Bonke, B. & De Haes, J. C. (1995). The Multidimensional Fatigue Inventory (MFI) psychometric qualities of an instrument to assess fatigue. Journal of Psychosomatic Research 39 (3), 315–325. DOI logoGoogle Scholar
Souchet, A. D., Philippe, S., Lourdeaux, D. & Leroy, L. (2022). Measuring visual fatigue and cognitive load via eye tracking while learning with virtual reality head-mounted displays: A review. International Journal of Human-Computer Interaction 38 (9), 801–824. DOI logoGoogle Scholar
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction 4 (4), 295–312. DOI logoGoogle Scholar
Sweller, J., van Merrienboer, J. J. G. & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review 10 (3), 251–296. DOI logoGoogle Scholar
Sweller, J., van Merriënboer, J. J. G. & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review 31 (2), 261–292. DOI logoGoogle Scholar
Tanaka, M., Ishii, A. & Watanabe, Y. (2014). Neural effects of mental fatigue caused by continuous attention load: A magnetoencephalography study. Brain Research 1561 1, 60–66. DOI logoGoogle Scholar
Tolkmitt, F. J. & Scherer, K. R. (1986). Effect of experimentally induced stress on vocal parameters. Journal of Experimental Psychology: Human Perception and Performance 12 (3), 302–313.Google Scholar
Tommola, J. & Niemi, P. (1986). Mental load in simultaneous interpreting: An on-line pilot study. In L. S. Evensen (Ed.), Nordic research in text linguistics and discourse analysis. Trondheim: Tapir. 171–184.Google Scholar
Van Puyvelde, M., Neyt, X., McGlone, F. & Pattyn, N. (2018). Voice stress analysis: A new framework for voice and effort in human performance. Frontiers in Psychology 9 1, 1994. DOI logoGoogle Scholar
Vogel, A. P. (2010). Acoustic analysis of the effects of sustained wakefulness on speech. Journal of the Acoustical Society of America 128 (6), 3747–3756. DOI logoGoogle Scholar
Wang, M. (2013). Correlation between error types and local cognitive load in simultaneous interpreting – taking English-Chinese language pair in Cognitive Load Model as an example. MA Thesis, Shanghai International Studies University.
Wang, Y., Zhai, G., Chen, S., Min, X., Gao, Z. & Song, X. (2019). Assessment of eye fatigue caused by head-mounted displays using eye-tracking. BioMed Engineering OnLine 18 1, 111. DOI logoGoogle Scholar
Welham, N. V. & Maclagan, M. A. (2003). Vocal fatigue: Current knowledge and future directions. Journal of Voice 17 (1), 21–30. DOI logoGoogle Scholar
Xu, R., Zhang, C., He, F., Zhao, X., Qi, H., Zhou, P., Zhang, L. & Ming, D. (2018). How physical activities affect mental fatigue based on EEG energy, connectivity, and complexity. Frontiers in Neurology 9 1, 915. DOI logoGoogle Scholar
Yamada, Y. & Kobayashi, M. (2018). Detecting mental fatigue from eye-tracking data gathered while watching video: Evaluation in younger and older adults. Artificial Intelligence in Medicine 91 1, 39–48. DOI logoGoogle Scholar
Yap, T. F., Epps, J., Ambikairajah, E. & Choi, E. H. C. (2011). Formant frequencies under cognitive load: Effects and classification. EURASIP Journal on Advances in Signal Processing 2011, 219253. DOI logoGoogle Scholar
Yoo, G., Kim, H. & Hong, S. (2023). Prediction of cognitive load from electroencephalography signals using long short-term memory network. Bioengineering 10 (3), 361. DOI logoGoogle Scholar
Zheng, R. Z. & Greenberg, K. (2017). The boundary of different approaches in cognitive load measurement strengths and limitations. In R. Z. Zheng & K. Grenberg (Eds.), Cognitive load measurement and application. New York: Routledge, 45–56.Google Scholar