Cover not available
Article published In:
Language, Context and Text
Vol. 4:2 (2022) ► pp.335–374

Emotion in speech

A systemic functional semiotic approach to the vocalisation of affect

ORCID logo | University of Wollongong, Australia | Universidad Nacional de La Pampa
ORCID logo | University of Wollongong, Australia
ORCID logo | University of Wollongong, Australia
This paper explores how emotion is conveyed in spoken language based on a sample of three stories for children read aloud by a trained storyteller. It draws on both Martin & White’s appraisal framework (2005) and a systematic account of vocal features. Interpreting and profiling emotion is a challenge both for researchers interested in spoken language and for English-as-an-additional-language teachers, when working to improve students’ spoken expression of emotion. Underpinning this challenge is the absence of a unified theoretical stance and a consequent lack of shared analytical tools for describing how speakers use semiotic resources to convey emotion in their speech. A systemic functional semiotic approach can make visible the co-patterning of wordings and vocalisations that express emotion.
Keywords: systemic functional semiotics (SFS), appraisal analysis, spoken language, vocal features, co-patterning of wordings and vocalisations, emotion in speech, interpreting and profiling emotion, multi-stratal analysis, multisemiotic interaction, storytelling, children’s stories, English as additional language (EAL)
Article outline
  • 1.Introduction
  • 2.The study
    • 2.1Methodology
      • 2.1.1Data collection and data preparation
    • 2.2Theoretical framework for the analysis
      • 2.2.1Emotion in speech: SFS perspective
      • 2.2.2Analysis of evaluative language
      • 2.2.3Analysis of vocal features
        • 2.2.3.1Suprasegmental features
        • 2.2.3.2Paralinguistic features: Definition and analytical technique
  • 3.Results and discussion
    • 3.1Preliminary vocal profiles of inscribed affect
      • 3.1.1Vocal profiles for inscribed affect and vocal emotion profiles outside the SFS tradition
      • 3.1.2Prediction value of profiles and teaching implications
      • 3.1.3Association between affect categories and paralinguistic features
      • 3.1.4Additional understandings gained from associating affect categories and vocal profiles
      • 3.1.5Distinguishing affect types: The example of ‘unhappy’
  • 4.Conclusions and pedagogical implications
  • Acknowledgements
  • Notes
  • References
Published online: 29 September 2022
DOI logo
https://doi.org/10.1075/langct.21012.ari
References (60)
References
Ariztimuño, Lilian I. 2017. The expression of emotion in fairy tales: A multimodal approach to improve EFL students’ oral renderings. Phonetics Teaching and Learning Conference (PTLC2017) Proceedings, 11–15.Google Scholar
Arnfield, Simon, Peter Roach, Jane Setter, Peter Greasley & Dave Horton. 1995. Emotional stress and speech tempo variation. ESCA-NATO Tutorial and research workshop on speech under stress, 13–15.Google Scholar
Bednarek, Monika. 2008. Emotion talk in corpora. Basingstoke/New York: Palgrave Macmillan. DOI logoGoogle Scholar
Boersma, Paul & David Weenink. 2021. Praat: Doing phonetics by computer [Computer program]. Version 6.0. 381. [URL]
Bombelli, Griselda, Lidia R. Soler & María S. Waasaf. 2013. The language of Evaluation: Paralinguistic features as a phonological domain for Appraisal. DELTA Documentação de Estudos em Lingüística Teórica e Aplicada 29 (2). 267–280. DOI logoGoogle Scholar
Caldwell, David. 2014. The interpersonal voice: Applying appraisal to the rap and sung voice. Social Semiotics 24 (1). 40–55. DOI logoGoogle Scholar
Cruttenden, Alan. 2008. Gimson’s pronunciation of English (7th edition). London: Hodder Education.Google Scholar
Crystal, David & Randolph Quirk. 1964. Systems of prosodic and paralinguistic features in English. The Hague: De Gruyter Mouton. DOI logoGoogle Scholar
Douglas-Cowie, Ellen, Nick Campbell, Roddy Cowie & Peter Roach. 2003. Emotional speech: Towards a new generation of databases. Speech Communication 40 (1). 33–60. DOI logoGoogle Scholar
Eggins, Suzanne & Diana Slade. 2005. Analysing casual conversation. London: Equinox.Google Scholar
Ekman, Paul. 1999. Basic emotions. In Tim Dalgleish & Mick Power (eds.), Handbook of cognition and emotion. Chichester: John Wiley & Sons. DOI logoGoogle Scholar
Germani, Miriam P. & Lucía I. Rivas. 2017. A genre approach to prosody: Teaching intonation from a discourse perspective. Phonetics Teaching and Learning Conference (PTLC2017) Proceedings, 49–53.Google Scholar
Halliday, Michael A. K. 1970. A course in spoken English: Intonation. London: OUP.Google Scholar
1978. Language as social semiotic: The social interpretation of language and meaning. London: Hodder Education.Google Scholar
2003. On language and linguistics, volume 3 in the collected works of M.A.K Halliday. Edited by Jonathan J. Webster. London: Continuum.Google Scholar
Halliday, Michael A. K. & William S. Greaves. 2008. Intonation in the grammar of English. Equinox.Google Scholar
Halliday, Michael A. K. & Ruqaiya Hasan. 1985. Language, context, and text: Aspects of language in a social-semiotic perspective. Waurn Ponds, Victoria: Deakin University Press.Google Scholar
Halliday, Michael A. K. & Christian M. I. M. Matthiessen. 2014. Halliday’s introduction to functional grammar. London: Routledge. DOI logoGoogle Scholar
Hasan, Ruqaiya. 1984. The nursery tale as a genre. Nottingham Linguistic Circular, 71–102.Google Scholar
. 1985. Linguistics, language, and verbal art. Oxford: Oxford University Press.Google Scholar
Humphrey, Sally & Shoshana Dreyfus. 2012. Exploring the interpretive genre in applied linguistics. Indonesian Journal of SFL 1 (2). 156–174.Google Scholar
Jewitt, Carey, Jeff Bezemer & Kay O’Halloran. 2016. Introducing multimodality. London/New York: Routledge. DOI logoGoogle Scholar
Johnstone, Tom, Van Reekum, Carien M. & Klaus R. Scherer. 2001. Vocal expression correlates of appraisal processes. In Klaus R. Scherer, Angela Schorr & Tom Johnstone (eds.), Appraisal processes in emotion: Theory, methods, research, 271–284. New York: Oxford University Press.Google Scholar
Juslin, Patrik N. & Klaus R. Scherer. 2005. Vocal expression of affect. In Jinni A. Harrigan, Robert Rosenthal & Klaus R. Scherer (eds.), The new handbook of methods in nonverbal behavior research. Oxford: Oxford University Press. DOI logoGoogle Scholar
Kress, Gunther R. 2010. Multimodality: A social semiotic approach to contemporary communication. New York: Routledge.Google Scholar
Laukka, Petri, Hillary Anger Elfenbein, Nutankumar S. Thingujam, Thomas Rockstuhl, Frederick. K. Iraki, Wanda Chui & Jean Althoff. 2016. The expression and recognition of emotions in the voice across five nations: A lens model analysis based on acoustic features. Journal of Personality and Social Psychology 111 (5). 686–705. DOI logoGoogle Scholar
Lorette, Parnelle & Jean-Marc Dewaele. 2019. The relationship between bi/multilingualism, nativeness, proficiency and multimodal emotion recognition ability. International Journal of Bilingualism 23 (6). 1502–1516. DOI logoGoogle Scholar
Macken-Horarik, Mary. 2003. Appraisal and the special instructiveness of narrative. Text & Talk 23 (2). 285–312. DOI logoGoogle Scholar
Macken-Horarik, Mary & Anne Isaac. 2014. Appraising appraisal. In Geoff Thompson & Laura Alba-Juez (eds.), Evaluation in context. Amsterdam/Philadelphia: John Benjamins. DOI logoGoogle Scholar
Mackenzie, J. Lachlan & Laura Alba-Juez. 2019. Emotion in discourse. Amsterdam/Philadelphia: John Benjamins. DOI logoGoogle Scholar
Martin, James. R. 2017. The discourse semantics of attitudinal relations: Continuing the study of lexis. Russian Journal of Linguistics 21 (1). 22–47. DOI logoGoogle Scholar
Martin, James R. & David Rose. 2008. Genre relations: Mapping culture. London/Oakville: Equinox.Google Scholar
Martin, James R. & Peter R. R. White. 2005. The language of evaluation. Appraisal in English. Houndmills, UK: Palgrave Macmillan. DOI logoGoogle Scholar
Martin, James R. & Michele Zappavigna. 2019. Embodied meaning: A systemic functional perspective on paralanguage. Functional Linguistics 6 (1). 1–33. DOI logoGoogle Scholar
Matthiessen, Christian M. I. M. 2009. Multisemiosis and context-based register typology. In Eija Ventola & Arsenio J. M. Guijarro (eds.), The world told and the world shown: Multisemiotic issues. 11–38. London: Palgrave Macmillan.Google Scholar
Mills, Kathy A. & Len Unsworth. 2016. Multimodal Literacy: Oxford Research Encyclopedia of Education. DOI logoGoogle Scholar
. 2017. iPad animations: Powerful multimodal practices for adolescent literacy and emotional language. Journal of Adolescent & Adult Literacy 61 (6). 609–620. DOI logoGoogle Scholar
Noad, Betty & Georgina Barton. 2018. Emotion resonance and divergence: A semiotic analysis of music and sound in “The Lost Thing” an animated short film and “Elizabeth” a film trailer. Social Semiotics 30 (2). 206–224. DOI logoGoogle Scholar
O’Grady, Gerard. 2010. A grammar of spoken English: The intonation of increments. London: Continuum.Google Scholar
. 2020. Is there a role for prosody within register studies? And if so what and how? Language, Context and Text 2 (1). 59–90. DOI logoGoogle Scholar
Painter, Clare, James. R. Martin & Len Unsworth. 2013. Reading visual narratives: Image analysis of children’s picture books. Sheffield: Equinox.Google Scholar
Pavlenko, Aneta. 2007. Emotions and multilingualism. Cambridge: Cambridge University Press.Google Scholar
Rintell, Ellen. M. 1984. But how did you feel about that? The learner’s perception of emotion in speech. Applied Linguistics 5 (3). 255–264. DOI logoGoogle Scholar
Roach, Peter. 2000. Techniques for the phonetic description of emotional speech. In ISCA Tutorial and Research Workshop (ITRW) on Speech and Emotion. 53–59.Google Scholar
. 2009. English phonetics and phonology: A practical course. Cambridge: Cambridge University Press.Google Scholar
Roach, Peter, Richard Stibbard, Jane Osborne, Simon Arnfield & Jane Setter. 1998. Transcription of prosodic and paralinguistic features of emotional speech. Journal of the International Phonetic Association 28 (1–2). 83–94. DOI logoGoogle Scholar
Royce, Terry D. 2007. Multimodal communicative competence in second language contexts. In Terry D. Royce & Wendy L. Bowcher (eds.), New directions in the analysis of multimodal discourse. 361–390. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
Scherer, Klaus R. 2003. Vocal communication of emotion: A review of research paradigms. Speech Communication 40 (1–2). 227–256. DOI logoGoogle Scholar
2019. Acoustic patterning of emotion vocalizations. In Sascha Fruhholz & Pascal Belin (eds.), The Oxford handbook of voice perception. 61–91. Oxford: OUP.Google Scholar
Scherer, Klaus R. & Heiner Ellgring. 2007. Multimodal expression of emotion: Affect programs or componential appraisal patterns? Emotion 7 (1). 158–171. DOI logoGoogle Scholar
Schuller, Björn, Anton Batliner, Stefan Steidl & Dino Seppi. 2011. Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge. Speech Communication 53 (9–10). 1062–1087. DOI logoGoogle Scholar
Smith, Bradley A. 2012. Speech and writing: Intonation within multimodal studies. In Kay. L. O’Halloran & Bradley A. Smith (eds.), Multimodal studies: Exploring issues and domains. 291–313. London: Routledge.Google Scholar
Smith, Bradley A. & William S. Greaves. 2015. Intonation. In Jonathan J. Webster (ed.), The Bloomsbury companion to MAK Halliday. 291–313. London: Bloomsbury.Google Scholar
Storynory. 2013. King Thrushbeard, The elves and the shoemaker, Rumpelstiltskin. Available at [URL]. (17 July, 2013.)
Styler, Will. 2015. Using Praat for linguistic research. University of Colorado at Boulder Phonetics Laboratory. [URL]
Tench, Paul. 1996. The intonation systems of English. London: Cassell.Google Scholar
. 2014. A systemic functional model of the intonation of clauses in English. Readings in English Phonetics and Phonology (1). 261–296.Google Scholar
Unsworth, Len & Kathy A. Mills. 2020. English language teaching of attitude and emotion in digital multimodal composition. Journal of Second Language Writing (47). 100712. DOI logoGoogle Scholar
van Leeuwen, Theo. 1999. Speech, music, sound. Basingstoke: Macmillan. DOI logoGoogle Scholar
Zappavigna, Michele, Chris Cléirigh, Paul Dwyer & James R. Martin. 2010. Visualising appraisal prosody. In Ahmar Mahboob & Naomi K. Knight (eds.), Appliable Linguistics: Texts, Contexts, and Meanings, 150–167. London & New York: Continuum.Google Scholar
Cited by (1)

Cited by one other publication

Fryer, Daniel Lees
2024. Feeling, thinking, acting. Language, Context and Text. The Social Semiotics Forum 6:1 ► pp. 122 ff. DOI logo

This list is based on CrossRef data as of 16 september 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.