Cover not available
Article published In:
Functions of Language
Vol. 22:3 (2015) ► pp.297–331

Adjectives, communities, and taxonomies of evaluative meaning

| University of Tsukuba
| University of Birmingham
From a corpus consisting of free text comments submitted to the website RateMyProfessors.com, adjectives describing people have been identified and clustered using Principal Components Analysis. This follows the Meaning Extraction method used by Chung & Pennebaker (2007). The outcome is a set of seven factors, each of which is interpretable as representing a dimension of meaning along which individual professors have been evaluated. These dimensions are in turn discussed using Martin & White’s (2005) parameters of Judgement and Appreciation, and Coffin’s (2002) concept of evaluative voices. It is argued that contributors to RateMyProfessors.com have available three distinct voices: ‘novice intellectual’, ‘consumer’, and ‘subordinate’. The paper demonstrates how a bottom-up, statistical technique may be used to provide the initial data for identifying evaluative parameters. It raises the possibility that such parameters may be specific to individual discourse communities. It therefore offers a complementary and problematizing alternative to top-down, researcher-driven taxonomies of evaluative meaning.
Published online: 10 December 2015
DOI logo
https://doi.org/10.1075/fol.22.3.01mil
References (66)
Argamon, Shlomo, Moshe Koppel, James W. Pennebaker & Jonathan Schler. 2007. Mining the blogosphere: Age, gender and the varieties of self-expression. First Monday 121. Available at: <[URL]>. Date accessed: 12 August 2014. DOI logoGoogle Scholar
Bednarek, Monika. 2006. Evaluation in media discourse. London: Continuum.Google Scholar
. 2008a. ‘An increasingly familiar tragedy’: Evaluative collocation and conflation. Functions of Language 151. 7–34. DOI logoGoogle Scholar
. 2008b. Emotion talk across corpora. London: Palgrave. DOI logoGoogle Scholar
Biber, Douglas. 1988. Variation across speech and writing. Cambridge: CUP. DOI logoGoogle Scholar
. 1993. The multi-dimensional approach to linguistic analyses of genre variation: An overview of methodology and findings. Computers and the Humanities 261. 331–345.Google Scholar
. 2006. Stance in spoken and written university registers. Journal of English for Academic Purposes 51. 97–116. DOI logoGoogle Scholar
Biber, Douglas & Susan Conrad. 2009. Register, genre, and style. Cambridge: CUP. DOI logoGoogle Scholar
Biber, Douglas & Edward Finegan. 1988. Adverbial stance types in English. Discourse Processes 111. 1–34. DOI logoGoogle Scholar
Brown, Michael J., Michelle Baillie & Shawndel Fraser. 2009. Rating RateMyProfessors.com: A comparison of online and official student evaluations of teaching. College Teaching 57(2). 89–92. DOI logoGoogle Scholar
Charles, Maggie. 2006. The construction of stance in reporting clauses: A cross-disciplinary study of theses. Applied Linguistics 271. 492–518. DOI logoGoogle Scholar
. 2007. Argument or evidence? Disciplinary variation in the use of the noun-that pattern in stance construction. English for Specific Purposes 261. 203–218. DOI logoGoogle Scholar
Chen, Yining & Leon B. Hoshower. 2003. Student evaluation of teaching effectiveness: An assessment of student perception and motivation. Assessment and Evaluation in Higher Education 281. 71–88. DOI logoGoogle Scholar
Chung, Cindy K. & James W. Pennebaker. 2008. Revealing dimensions of thinking in open-ended self-descriptions: An automated meaning extraction method for natural language. Journal of Research in Personality 42(1). 96–132. DOI logoGoogle Scholar
Coffin, Caroline. 2002. The voices of history: Theorising the interpersonal semantics of historical discourses. Text 22(4). 503–528. DOI logoGoogle Scholar
Coffin, Caroline & Kieran O’Halloran. 2006. The role of appraisal and corpora in detecting covert evaluation. Functions of Language 131. 77–110. DOI logoGoogle Scholar
Conrad, Susan & Douglas Biber. 2000. Adverbial marking of stance in speech and writing. In Susan Hunston & Geoff Thompson (eds.), 56–73.Google Scholar
Crawley, Michael J. 2007. The R book. New Jersey, NJ: Wiley. DOI logoGoogle Scholar
Davison, Elizabeth & Jammie Price. 2009. How do we rate? An evaluation of online student evaluations. Assessment & Evaluation in Higher Education 34(1). 51–65. DOI logoGoogle Scholar
Drasovean, Anda & Caroline Tagg. Submitted. Evaluation and its community-building role.
Dunbar, George. 2012. Adaptive Resonance Theory as a model of polysemy and vagueness in the cognitive lexicon. Cognitive Linguistics 23(3). 507–537. DOI logoGoogle Scholar
Eggins, Suzanne & Diane Slade. 1997. Analysing casual conversation. London: Cassell.Google Scholar
Felton, James, Peter T. Koper, John Mitchell & Michael Stinson. 2008. Attractiveness, easiness and other issues: Student evaluations of professors on Ratemyprofessors.com. Assessment & Evaluation in Higher Education 33(1). 45–61. DOI logoGoogle Scholar
Felton, James, John Mitchell & Michael Stinson. 2004. Web-based student evaluations of professors : The relations between perceived quality, easiness and sexiness. Assessment & Evaluation in Higher Education 29(1). 91–108. DOI logoGoogle Scholar
Francis, Gill. 1995. Corpus-driven grammar and its relevance to the learning of English in a cross-cultural situation. Unpublished manuscript.
Fuoli, Matteo. 2012. Assessing social responsibility: A quantitative analysis of Appraisal in BP’s and IKEA’s social reports. Discourse and Communication 6(1). 55–81. DOI logoGoogle Scholar
Garside, Roger & Nicholas Smith. 1997. A hybrid-grammatical tagger: CLAWS 4. In Roger Garside, Geoffrey Leech & Anthony McEnery (eds.), Corpus annotation: Linguistic information from computer text corpora, 102–121. Harlow: Longman.Google Scholar
Gee, James Paul. 2005. Semiotic social spaces and affinity spaces: From The Age of Mythology to today’s schools. In David Barton & Karin Tusting (eds.), Beyond communities of practice: Language power and social context, 214–232. Cambridge: CUP. DOI logoGoogle Scholar
Halliday, M.A.K. 1978. Language as social semiotic: The social interpretation of language and meaning. London: Arnold.Google Scholar
. 1994. An introduction to functional grammar, 2nd edition. London: Arnold.Google Scholar
Hardy, Jack A. & Ute Römer. 2013. Revealing disciplinary variation in student writing: A multi-dimensional analysis of the Michigan Corpus of Upper-level Student papers (MICUSP). Corpora 81. 183–207. DOI logoGoogle Scholar
Hiltunen, Turo. 2010. Grammar and disciplinary culture: A corpus-based study. Helsinki: University of Helsinki.Google Scholar
Hunston, Susan. 1993. Evaluation and ideology in scientific writing. In M. Ghadessy (ed.), Register analysis: Theory and practice, 57–74. London: Pinter.Google Scholar
. 2011. Corpus approaches to evaluation: Phraseology and evaluative meaning. London: Routledge.Google Scholar
Hunston, Susan & Geoff Thompson (eds.). 2000. Evaluation in text: Authorial stance and the construction of discourse. Oxford: OUP.Google Scholar
Hyland, Ken. 2005. Metadiscourse. London: Continuum.Google Scholar
. 2009. Academic discourse. London: Continuum.Google Scholar
. 2012. Disciplinary identities: Individuality and community in academic discourse. Cambridge: CUP.Google Scholar
Iedema, Rick, Susan Feez & Peter R.R. White. 1994. Media literacy. Sydney: Disadvantaged Schools Program, NSW Department of School Education.Google Scholar
Knight, Naomi. 2008. ‘Still cool…and american too!’: An SFL analysis of deferred bonds in internet messaging humour. In Nina Nørgaard (ed.), Systemic Functional Linguistics in use, Odense Working Papers in Language and Communication, 481–502. Retrieved 3rd January 2014 from [URL].Google Scholar
Kramer, Adam D.I. & Cindy K. Chung. 2011. Dimensions of self-expression in Facebook status updates. Proceedings of the 2008 International Conference on Weblogs and Social Media (ICWSM2011), Palo Alto, CA, July 2011. 169–176.
Kristiansen, Gitte & Dirk Geeraerts. 2013. Contexts and usage in Cognitive Sociolinguistics. Journal of Pragmatics 521. 1–4. DOI logoGoogle Scholar
Lemke, Jay. 1998. Resources for attitudinal meaning: Evaluative orientations in text semantics. Functions of Language 5(1). 33–56. DOI logoGoogle Scholar
Liu Bing. 2010. Sentiment analysis and subjectivity. In Nitin Indurkhya & Fred J. Damerau (eds.), Handbook of Natural Language Processing, 2nd edn, 627–665. Boca Raton, FL: Chapman & Hall / CRC.Google Scholar
Liu Dilin. 2013. Salience and construal in the use of synonymy: A study of two sets of near-synonymous nouns. Cognitive Linguistics 24(1). 67–113. DOI logoGoogle Scholar
Marsh, Herbert W. 2007. Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In Raymond P. Perry & John C. Smart (eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective, 319–383. Berlin: Springer. DOI logoGoogle Scholar
Martin, James R. 2000. Beyond exchange: Appraisal systems in English. In Susan Hunston & Geoff Thompson (eds.), 142–175.Google Scholar
Martin, James R. & Peter R.R. White. 2005. The language of evaluation: Appraisal in English. London: Palgrave. DOI logoGoogle Scholar
Norman, Geoffrey R. & David L. Streiner. 1994. Biostatistics: The bare essentials. St Louis: Mosby.Google Scholar
Polanyi, Livia. 1985. Telling the American story: From the structure of linguistic texts to the grammar of a culture. Norwood, NJ: Ablex.Google Scholar
Pütz, Martin, Justyna A. Robinson & Monika Reif (eds.). 2014. Cognitive sociolinguistics: Social and cultural variation in cognition and language use. Amsterdam: Benjamins. DOI logoGoogle Scholar
Ramírez-Esparza, Nairan, Cindy K. Chung, Ewa Kacewicz & James W. Pennebaker. 2008. The psychology of word use in depression forums in English and in Spanish. Testing two text analytic approaches. Proceedings of the 2008 International Conference on Weblogs and Social Media (ICWSM2008) , Menlo Park, CA, April 2008. 102–108.
RateMyProfessors.com. 2013. About us. Retrieved August 6, 2013, from [URL]
Revelle, William. 2013. Psych. Procedures for personality and psychological research, Northwestern University, Evanston, Illinois, USA, [URL]. Version = 1.3.2Google Scholar
Sanders, Shane, Bhavneet Walia, Joel Potter & Kenneth W. Linna. 2011. Do more online instructional ratings lead to better prediction of instructor quality? Practical Assessment, Research & Evaluation 16(2). 1–6.Google Scholar
Schiffrin, Deborah. 1984. How a story says what it means and does. Text 41. 313–346. DOI logoGoogle Scholar
Schönefeld, Doris. 2013. ‘It is…quite common for theoretical predictions to go untested (BNC_CMH)’. A register-specific analysis of the English go un-V-ed construction. Journal of Pragmatics 521: 17–33. DOI logoGoogle Scholar
Shevlin, Mark, Philip Banyard, Mark Davies & Mark Griffiths. 2000. The validity of student evaluation of teaching in higher education: Love me, love my lectures? Assessment and Evaluation in Higher Education 251. 397–405. DOI logoGoogle Scholar
Stenglin, Maree K. 2004. Packaging curiosities: Towards a grammar of three-dimensional space. Sydney: University of Sydney PhD thesis.Google Scholar
Su, Hang. In preparation. Adjectives of judgement in biographical discourse: A corpus study. Birmingham: University of Birmingham PhD thesis.
Teubert, Wolfgang. 2010. Meaning, discourse and society. Cambridge: CUP. DOI logoGoogle Scholar
Thompson, Geoff & Susan Hunston. 2000. Evaluation: An introduction. In Susan Hunston & Geoff Thompson (eds.), 1–27.Google Scholar
Tuggy, David. 1993. Ambiguity, polysemy and vagueness. Cognitive Linguistics 41. 273–290. DOI logoGoogle Scholar
Turney, Peter D. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , Philadelphia, July 2002. 417–424.
Wolf, Markus, Cindy K. Chung & Hans Kordy. 2010. Inpatient treatment to online aftercare: E-mailing themes as a function of therapeutic outcomes. Psychotherapy research: Journal of the Society for Psychotherapy Research 20(1). 71–85. DOI logoGoogle Scholar
Zappavigna, Michelle. 2011. Ambient affiliation: A linguistic perspective on Twitter. New Media & Society 13(5). 788–806. DOI logoGoogle Scholar
Cited by (18)

Cited by 18 other publications

Etaywe, Awni
2024. Discursive pragmatics of justification in terrorist threat texts: Victim-blaming, denying, discrediting, legitimating, manipulating, and retaliation. Discourse & Society DOI logo
Fetterman, Adam K., Nicholas D. Evans, Eriksen P. Ravey, Perla R. Henderson, Bao Han L. Tran & Ryan L. Boyd
2024. The Topics of Nostalgic Recall: The Benefits of Nostalgia Depend on the Topics That One Recalls. Social Psychological and Personality Science DOI logo
Cheng, Shi
2023. A review of interpersonal metafunction studies in systemic functional linguistics (2012–2022). Journal of World Languages 0:0 DOI logo
Etaywe, Awni & Michele Zappavigna
2022. Identity, ideology and threatening communication. Journal of Language Aggression and Conflict 10:2 ► pp. 315 ff. DOI logo
Etaywe, Awni & Michele Zappavigna
2023. The role of social affiliation in incitement: A social semiotic approach to far-right terrorists’ incitement to violence. Language in Society ► pp. 1 ff. DOI logo
Ho, Mei-ching
2022. A corpus-pragmatics approach to evaluation in professor reviews. Concentric. Studies in Linguistics 48:1 ► pp. 114 ff. DOI logo
Foxman, Maxwell, David M Markowitz & Donna Z Davis
2021. Defining empathy: Interconnected discourses of virtual reality’s prosocial impact. New Media & Society 23:8 ► pp. 2167 ff. DOI logo
Markowitz, David M.
2021. The Meaning Extraction Method: An Approach to Evaluate Content Patterns From Large-Scale Language Data. Frontiers in Communication 6 DOI logo
Li, Jing, Lei Lei & Le Cheng
2020. Mapping Evaluation, Appraisal and Stance in Discourse (2000–2015): A Bibliometric Analysis. Glottotheory 10:1-2 ► pp. 31 ff. DOI logo
Markowitz, David M. & Paul Slovic
2020. Social, psychological, and demographic characteristics of dehumanization toward immigrants. Proceedings of the National Academy of Sciences 117:17 ► pp. 9260 ff. DOI logo
Goddard, Cliff, Maite Taboada & Radoslava Trnavac
2019. The semantics of evaluational adjectives. Functions of Language 26:3 ► pp. 308 ff. DOI logo
Ikizer, Elif G., Nairán Ramírez-Esparza & Ryan L. Boyd
2019. #sendeanlat (#tellyourstory): Text Analyses of Tweets About Sexual Assault Experiences. Sexuality Research and Social Policy 16:4 ► pp. 463 ff. DOI logo
Su, Hang & Susan Hunston
2019. Adjective complementation patterns and judgement: Aligning lexical-grammatical and discourse-semantic approaches in appraisal research. Text & Talk 39:3 ► pp. 415 ff. DOI logo
Su, Hang & Susan Hunston
2019. Language patterns and attitude revisited. Functions of Language 26:3 ► pp. 343 ff. DOI logo
Blackburn, Kate G., Gamze Yilmaz & Ryan L. Boyd
2018. Food for thought: Exploring how people think and talk about food online. Appetite 123 ► pp. 390 ff. DOI logo
Huntley, Selene J., Michaela Mahlberg, Viola Wiegand, Yves van Gennip, Hui Yang, Rachel S. Dean & Marnie L. Brennan
2018. Analysing the opinions of UK veterinarians on practice-based research using corpus linguistic and mathematical methods. Preventive Veterinary Medicine 150 ► pp. 60 ff. DOI logo
Su, Hang
2016. How products are evaluated? Evaluation in customer review texts. Language Resources and Evaluation 50:3 ► pp. 475 ff. DOI logo
[no author supplied]
2021. The Sociopragmatics of Stance [Pragmatics & Beyond New Series, 329], DOI logo

This list is based on CrossRef data as of 2 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.