Cover not available
Article published In:
Revista Española de Lingüística Aplicada/Spanish Journal of Applied Linguistics
Vol. 37:2 (2024) ► pp.486–517

A protocol for the annotation of evaluative stance and metaphor across four discourse genres

ORCID logo | Universidad Autónoma de Madrid
ORCID logo | Universidad de La Rioja
ORCID logo | Universidad de Valladolid
ORCID logo | Universidad Complutense de Madrid
ORCID logo | Universidad Complutense de Madrid
ORCID logo | Universidad Complutense de Madrid
ORCID logo | Universidad de Cantabria
The present article contributes to research on evaluation by addressing two complementary objectives: first, we present a protocol for the identification and annotation of evaluation in English discourse and, second, we show the results of the implementation of the protocol in the annotation of evaluation in a sample of a corpus of four genres. We first describe the protocol by discussing the theoretical and methodological grounding of the annotation scheme, the criteria, the categories, the steps for the implementation of the protocol and an illustrative example of the application of the protocol to a short extract. We subsequently provide the preliminary results of a pilot study with the frequency of evaluative expressions across the four genres. Results show that while adjectives and non-metaphoric evaluative expressions are overall more frequent, there are differences regarding the preference for positive or negative value and regarding the frequency of function.
Keywords: evaluation, stance, evaluative metaphor, evaluation annotation, evaluation in genres
Article outline
  • 1.Introduction
  • 2.A protocol for the identification and annotation of evaluation in discourse
    • 2.1Evaluation and stance: Theoretical grounding and methodological issues
    • 2.2Motivations for the design of the protocol
    • 2.3Criteria for identifying evaluative stance
    • 2.4Categories
      • 2.4.1Part of speech
      • 2.4.2Function
      • 2.4.3Metaphoricity
      • 2.4.4Value
    • 2.5Annotation procedure
    • 2.6Stages in the development and implementation of the protocol
  • 3.Implementing the protocol: An example
  • 4.Variation of evaluation across genres: A pilot study
    • 4.1Data
    • 4.2Method
    • 4.3Results and discussion
    • 4.4Examples from the different genres: Preliminary results from case studies
      • 4.4.1Newspaper opinion discourse
      • 4.4.2Political discourse
      • 4.4.3Press popularization articles
      • 4.4.4Fora
  • 5.Conclusions
  • Notes
  • References
Published online: 12 March 2024
DOI logo
https://doi.org/10.1075/resla.21058.hid
References (35)
References
Alba-Juez, L. & G. Thompson. (2014). The many faces and phases of evaluation. In G. Thompson & L. Alba-Juez (Eds.) Evaluation in context (pp. 3–24). John Benjamins. DOI logoGoogle Scholar
Bednarek, M. (2009). Language patterns and ATTITUDE. Functions of Language, 16 (2), 165–192. DOI logoGoogle Scholar
Bednarek, M., & Caple, H. (2014). Why do news values matter? Towards a new methodological framework for analysing news discourse in Critical Discourse Analysis and beyond. Discourse & Society, 25 (2), 135–158. DOI logoGoogle Scholar
Biber, D., Johansson, S., Leech, G., Conrad, S., & Finegan, E. (1999). Longman grammar of spoken and written English. Longman Pearson.Google Scholar
Deignan, A. (2010). The evaluative properties of metaphors. In G. Low, Z. Todd, A. Deignan, & L. Cameron (Eds.) Researching and applying metaphor in the real world (pp. 357–374). John Benjamins. DOI logoGoogle Scholar
Du Bois, J. W. (2007). The stance triangle. In R. Englebretson (Ed.) Stancetaking in discourse (pp. 139–182). John Benjamins. DOI logoGoogle Scholar
Englebretson, R. (Ed.). (2007). Stancetaking in discourse. John Benjamins. DOI logoGoogle Scholar
Fuoli, M. (2018). A stepwise method for annotating Appraisal. Functions of Language, 25 (2), 229–258. DOI logoGoogle Scholar
Fuoli, M., & Hommerberg, C. (2015). Optimising transparency, reliability and replicability: Annotation principles and inter-coder agreement in the quantification of evaluative expressions. Corpora, 10 (3), 315–349. DOI logoGoogle Scholar
Fuoli, M., Littlemore, J., & Turner, S. (2021). Sunken ships and screaming banshees: Metaphor and evaluation in film reviews. English Language and Linguistics, 26 (1), 1–29. DOI logoGoogle Scholar
Hidalgo-Downing, L. (2016). Grammar and evaluation. In C. Chapelle (Ed.) The Applied Linguistics Encyclopedia. Online publication. [Accessed 2 October 2021].Google Scholar
Hidalgo-Downing, L., & Pérez-Sobrino, P. (2022). Developing an annotation protocol for evaluative stance and metaphor in discourse: Theoretical and methodological considerations. Text and Talk. Advanced online publication. DOI logoGoogle Scholar
(2023). ‘Pushing Britain off the precipice’: A CDA approach to negative evaluative stance in opinion articles on Brexit. In J. Marín-Arrese, L. Hidalgo-Downing, & J. R. Zamorano Mansilla (Eds.) Stance, inter/subjectivity and identity in discourse (pp. 201–226). Peter Lang.Google Scholar
Hidalgo Tenorio, E., & Benítez Castro, M. Á. (2020). The language of evaluation in the narratives by the Magdalene laundries survivors: The discourse of female victimhood. Applied Linguistics, 42 (2), 315–341. DOI logoGoogle Scholar
Hunston, S. (2007). Using a corpus to investigate stance quantitatively and qualitatively. In R. Englebretson (Ed.) Stancetaking in discourse (pp. 27–48). John Benjamins. DOI logoGoogle Scholar
(2011). Corpus Approaches to Evaluation. Phraseology and Evaluative Language. Routledge.Google Scholar
Hunston, S., & Su, H. (2019). Patterns, constructions, and local grammar: A case study of evaluation. Applied Linguistics, 40 (4), 567–593. DOI logoGoogle Scholar
Hunston, S., & Thompson, G. (2000). Evaluation in text. Oxford University Press. DOI logoGoogle Scholar
Jaki, S., De Smedt, T., Gwóźdź, M., Panchal, R., Rossa, A., & De Pauw, G. (2019). Online hatred of women in the Incels. me forum: Linguistic analysis and automatic detection. Journal of Language Aggression and Conflict, 7 (2), 240–268. DOI logoGoogle Scholar
Lorenzo-Dus, N., & Nouri, L. (2020). The discourse of the US alt-right online–a case study of the Traditionalist Worker Party blog. Critical Discourse Studies, 18 (4), 1–19.Google Scholar
Maíz-Arévalo, C., & Sánchez-Moya, A. (2023). “Histrionic, appalling, a major turkey”: The expression of evaluative stance in the discourse of online forums. In J. Marín Arrese, L. Hidalgo-Downing, & J. R. Zamorano Mansilla (Eds.) Stance, inter/subjectivity and identity in discourse (pp. 249–269). Peter Lang.Google Scholar
Martin, J. R., & White, P. R. R. (2005). The language of evaluation. Appraisal in English. Macmillan. DOI logoGoogle Scholar
Núñez-Perucha, B., & Filardo-Llamas, L. (2023). From “roaring lion” to “chlorinated chicken”: Evaluative stance and ideological positioning in a corpus of British political discourse. In J. Marín-Arrese, L. Hidalgo-Downing, & J. R. Zamorano Mansilla (Eds.) Stance, inter/subjectivity and identity in discourse (pp. 227–248). Peter Lang.Google Scholar
Pragglejazz Group (2007). A method for identifying metaphorically used words in discourse. Metaphor and Symbol, 22 (1), 1–39. DOI logoGoogle Scholar
Prażmo, E. (2020). Foids are worse than animals. A cognitive linguistics analysis of dehumanizing metaphors in online discourse. Topics in Linguistics, 21 (2), 16–27. DOI logoGoogle Scholar
Read, J., & Carroll, J. (2012). Annotating expressions of Appraisal in English. Language Resources and Evaluation, 46 (3), 421–447. DOI logoGoogle Scholar
Simaki, V., Paradis, C., & Kerren, A. (2018). Evaluating stance-annotated sentences from the Brexit Blog Corpus: A quantitative linguistic analysis. ICAME journal, 42 (1), 133–166. DOI logoGoogle Scholar
Simaki, V., Paradis, C., Skeppstedt, M., Sahlgren, M., Kucher, K., & Kerren, A. (2020). Annotating speaker stance in discourse: The Brexit Blog Corpus. Corpus Linguistics and Linguistic Theory, 16 (2), 215–248.Google Scholar
Steen, G., Dorst, Aletta G., Herrmann, J. B., Kaal, A. A., & Krennmayr, T. (2010). Metaphor in usage. Cognitive Linguistics, 21 (4), 765–796. DOI logoGoogle Scholar
Taboada, M., & Carretero, M. (2010). Contrastive analyses of evaluation in text: Key issues in the design of an annotation system for attitude applicable to consumer reviews in English and Spanish. Linguistics and the Human Sciences, 6 (1–3), 275–295. DOI logoGoogle Scholar
Thompson, G. (2014). Affect and emotion, target-value mismatches, and Russian dolls: Refining the appraisal model. In L. Alba-Juez, & G. Thompson (Eds.) Evaluation in context (pp. 47–66). John Benjamins. DOI logoGoogle Scholar
Van Dijk, T. A. (1995). Power and the news media. In D. L. Paletz (Ed.) Political communication and action (pp. 9–36). Hampton Press.Google Scholar
van Leeuwen, T. (2008). Discourse and Practice: New Tools for Critical Discourse Analysis. Oxford University Press. DOI logoGoogle Scholar
Williams Camus, J. T. (2023). Evaluative stance in science popularisations in the English press. In J. Marín-Arrese, L. Hidalgo-Downing, & J. R. Zamorano Mansilla (eds.) Stance, inter/subjectivity and identity in discourse (pp. 271–293). Peter Lang.Google Scholar
Wodak, R. & M. Meyer (Eds.) (2015). Methods of Critical Discourse Studies. Sage.Google Scholar