Adjectives, communities, and taxonomies of evaluative meaning
From a corpus consisting of free text comments submitted to the website RateMyProfessors.com, adjectives describing people have been identified and clustered using Principal Components Analysis. This follows the Meaning Extraction method used by Chung & Pennebaker (2007). The outcome is a set of seven factors, each of which is interpretable as representing a dimension of meaning along which individual professors have been evaluated. These dimensions are in turn discussed using Martin & White’s (2005) parameters of Judgement and Appreciation, and Coffin’s (2002) concept of evaluative voices. It is argued that contributors to RateMyProfessors.com have available three distinct voices: ‘novice intellectual’, ‘consumer’, and ‘subordinate’. The paper demonstrates how a bottom-up, statistical technique may be used to provide the initial data for identifying evaluative parameters. It raises the possibility that such parameters may be specific to individual discourse communities. It therefore offers a complementary and problematizing alternative to top-down, researcher-driven taxonomies of evaluative meaning.
References
Argamon, Shlomo, Moshe Koppel, James W. Pennebaker & Jonathan Schler
2007 Mining the blogosphere: Age, gender and the varieties of self-expression.
First Monday 121. Available at:
[URL]. Date accessed: 12 August 2014.


Bednarek, Monika
2006 Evaluation in media discourse. London: Continuum.

Bednarek, Monika
2008b Emotion talk across corpora. London: Palgrave.


Biber, Douglas
1988 Variation across speech and writing. Cambridge: CUP.


Biber, Douglas
1993 The multi-dimensional approach to linguistic analyses of genre variation: An overview of methodology and findings.
Computers and the Humanities 261. 331–345.

Biber, Douglas
2006 Stance in spoken and written university registers.
Journal of English for Academic Purposes 51. 97–116.


Biber, Douglas & Susan Conrad
2009 Register, genre, and style. Cambridge: CUP.


Biber, Douglas & Edward Finegan
1988 Adverbial stance types in English.
Discourse Processes 111. 1–34.


Brown, Michael J., Michelle Baillie & Shawndel Fraser
2009 Rating RateMyProfessors.com: A comparison of online and official student evaluations of teaching.
College Teaching 57(2). 89–92.


Charles, Maggie
2006 The construction of stance in reporting clauses: A cross-disciplinary study of theses.
Applied Linguistics 271. 492–518.


Charles, Maggie
2007 Argument or evidence? Disciplinary variation in the use of the noun-that pattern in stance construction.
English for Specific Purposes 261. 203–218.


Chen, Yining & Leon B. Hoshower
2003 Student evaluation of teaching effectiveness: An assessment of student perception and motivation.
Assessment and Evaluation in Higher Education 281. 71–88.


Chung, Cindy K. & James W. Pennebaker
2008 Revealing dimensions of thinking in open-ended self-descriptions: An automated meaning extraction method for natural language.
Journal of Research in Personality 42(1). 96–132.


Coffin, Caroline
2002 The voices of history: Theorising the interpersonal semantics of historical discourses.
Text 22(4). 503–528.


Coffin, Caroline & Kieran O’Halloran
Conrad, Susan & Douglas Biber
2000 Adverbial marking of stance in speech and writing. In
Susan Hunston &
Geoff Thompson (eds.), 56–73.

Crawley, Michael J
2007 The R book. New Jersey, NJ: Wiley.


Davison, Elizabeth & Jammie Price
2009 How do we rate? An evaluation of online student evaluations.
Assessment & Evaluation in Higher Education 34(1). 51–65.


Drasovean, Anda & Caroline Tagg
Submitted.
Evaluation and its community-building role.
Dunbar, George
2012 Adaptive Resonance Theory as a model of polysemy and vagueness in the cognitive lexicon.
Cognitive Linguistics 23(3). 507–537.


Eggins, Suzanne & Diane Slade
1997 Analysing casual conversation. London: Cassell.

Felton, James, Peter T. Koper, John Mitchell & Michael Stinson
2008 Attractiveness, easiness and other issues: Student evaluations of professors on Ratemyprofessors.
com. Assessment & Evaluation in Higher Education 33(1). 45–61.


Felton, James, John Mitchell & Michael Stinson
2004 Web-based student evaluations of professors : The relations between perceived quality, easiness and sexiness.
Assessment & Evaluation in Higher Education 29(1). 91–108.


Francis, Gill
1995 Corpus-driven grammar and its relevance to the learning of English in a cross-cultural situation. Unpublished manuscript.
Fuoli, Matteo
2012 Assessing social responsibility: A quantitative analysis of Appraisal in BP’s and IKEA’s social reports.
Discourse and Communication 6(1). 55–81.


Garside, Roger & Nicholas Smith
1997 A hybrid-grammatical tagger: CLAWS 4. In
Roger Garside,
Geoffrey Leech &
Anthony McEnery (eds.),
Corpus annotation: Linguistic information from computer text corpora, 102–121. Harlow: Longman.

Gee, James Paul
2005 Semiotic social spaces and affinity spaces: From The Age of Mythology to today’s schools. In
David Barton &
Karin Tusting (eds.),
Beyond communities of practice: Language power and social context, 214–232. Cambridge: CUP.


Halliday, M.A.K
1978 Language as social semiotic: The social interpretation of language and meaning. London: Arnold.

Halliday, M.A.K
1994 An introduction to functional grammar, 2nd edition. London: Arnold.

Hardy, Jack A. & Ute Römer
2013 Revealing disciplinary variation in student writing: A multi-dimensional analysis of the Michigan Corpus of Upper-level Student papers (MICUSP).
Corpora 81. 183–207.


Hiltunen, Turo
2010 Grammar and disciplinary culture: A corpus-based study. Helsinki: University of Helsinki.

Hunston, Susan
1993 Evaluation and ideology in scientific writing. In
M. Ghadessy (ed.),
Register analysis: Theory and practice, 57–74. London: Pinter.

Hunston, Susan
2011 Corpus approaches to evaluation: Phraseology and evaluative meaning. London: Routledge.

Hunston, Susan & Geoff Thompson
(eds.) 2000 Evaluation in text: Authorial stance and the construction of discourse. Oxford: OUP.

Hyland, Ken
2005 Metadiscourse. London: Continuum.

Hyland, Ken
2009 Academic discourse. London: Continuum.

Hyland, Ken
2012 Disciplinary identities: Individuality and community in academic discourse. Cambridge: CUP.

Iedema, Rick, Susan Feez & Peter R.R. White
1994 Media literacy. Sydney: Disadvantaged Schools Program, NSW Department of School Education.

Knight, Naomi
2008 ‘Still cool…and american too!’: An SFL analysis of deferred bonds in internet messaging humour. In
Nina Nørgaard (ed.),
Systemic Functional Linguistics in use, Odense Working Papers in Language and Communication, 481–502. Retrieved 3rd January 2014 from
[URL].

Kramer, Adam D.I. & Cindy K. Chung
2011 Dimensions of self-expression in Facebook status updates. Proceedings of the 2008 International Conference on Weblogs and Social Media (ICWSM2011), Palo Alto, CA, July 2011. 169–176.
Kristiansen, Gitte & Dirk Geeraerts
2013 Contexts and usage in Cognitive Sociolinguistics.
Journal of Pragmatics 521. 1–4.


Liu Bing
2010 Sentiment analysis and subjectivity. In
Nitin Indurkhya &
Fred J. Damerau (eds.),
Handbook of Natural Language Processing, 2nd edn, 627–665. Boca Raton, FL: Chapman & Hall / CRC.

Liu Dilin
2013 Salience and construal in the use of synonymy: A study of two sets of near-synonymous nouns.
Cognitive Linguistics 24(1). 67–113.


Marsh, Herbert W
2007 Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In
Raymond P. Perry &
John C. Smart (eds.),
The scholarship of teaching and learning in higher education: An evidence-based perspective, 319–383. Berlin: Springer.


Martin, James R
2000 Beyond exchange: Appraisal systems in English. In
Susan Hunston &
Geoff Thompson (eds.), 142–175.

Martin, James R. & Peter R.R. White
2005 The language of evaluation: Appraisal in English. London: Palgrave.


Norman, Geoffrey R. & David L. Streiner
1994 Biostatistics: The bare essentials. St Louis: Mosby.

Polanyi, Livia
1985 Telling the American story: From the structure of linguistic texts to the grammar of a culture. Norwood, NJ: Ablex.

Pütz, Martin, Justyna A. Robinson & Monika Reif
Ramírez-Esparza, Nairan, Cindy K. Chung, Ewa Kacewicz & James W. Pennebaker
2008 The psychology of word use in depression forums in English and in Spanish. Testing two text analytic approaches.
Proceedings of the 2008 International Conference on Weblogs and Social Media (ICWSM2008)
, Menlo Park, CA, April 2008. 102–108.
RateMyProfessors.com
2013 About us. Retrieved August 6, 2013, from
[URL]
Revelle, William
2013 Psych. Procedures for personality and psychological research, Northwestern University, Evanston, Illinois, USA,
[URL]. Version = 1.3.2

Sanders, Shane, Bhavneet Walia, Joel Potter & Kenneth W. Linna
2011 Do more online instructional ratings lead to better prediction of instructor quality? Practical Assessment, Research & Evaluation 16(2). 1–6.

Schiffrin, Deborah
1984 How a story says what it means and does.
Text 41. 313–346.


Schönefeld, Doris
2013 ‘It is…quite common for theoretical predictions to go untested (BNC_CMH)’. A register-specific analysis of the English go un-V-ed construction.
Journal of Pragmatics 521: 17–33.


Shevlin, Mark, Philip Banyard, Mark Davies & Mark Griffiths
2000 The validity of student evaluation of teaching in higher education: Love me, love my lectures? Assessment and Evaluation in Higher Education 251. 397–405.


Stenglin, Maree K
2004 Packaging curiosities: Towards a grammar of three-dimensional space. Sydney: University of Sydney PhD thesis.

Su, Hang
In preparation.
Adjectives of judgement in biographical discourse: A corpus study. Birmingham: University of Birmingham PhD thesis.
Teubert, Wolfgang
2010 Meaning, discourse and society. Cambridge: CUP.


Thompson, Geoff & Susan Hunston
2000 Evaluation: An introduction. In
Susan Hunston &
Geoff Thompson (eds.), 1–27.

Tuggy, David
1993 Ambiguity, polysemy and vagueness.
Cognitive Linguistics 41. 273–290.


Turney, Peter D
2002 Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews.
Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics
, Philadelphia, July 2002. 417–424.
Wolf, Markus, Cindy K. Chung & Hans Kordy
2010 Inpatient treatment to online aftercare: E-mailing themes as a function of therapeutic outcomes.
Psychotherapy research: Journal of the Society for Psychotherapy Research 20(1). 71–85.


Zappavigna, Michelle
2011 Ambient affiliation: A linguistic perspective on Twitter.
New Media & Society 13(5). 788–806.


Cited by
Cited by 14 other publications
Blackburn, Kate G., Gamze Yilmaz & Ryan L. Boyd
2018.
Food for thought: Exploring how people think and talk about food online.
Appetite 123
► pp. 390 ff.

Etaywe, Awni & Michele Zappavigna
Foxman, Maxwell, David M Markowitz & Donna Z Davis
2021.
Defining empathy: Interconnected discourses of virtual reality’s prosocial impact.
New Media & Society 23:8
► pp. 2167 ff.

Goddard, Cliff, Maite Taboada & Radoslava Trnavac
Ho, Mei-ching
Huntley, Selene J., Michaela Mahlberg, Viola Wiegand, Yves van Gennip, Hui Yang, Rachel S. Dean & Marnie L. Brennan
2018.
Analysing the opinions of UK veterinarians on practice-based research using corpus linguistic and mathematical methods.
Preventive Veterinary Medicine 150
► pp. 60 ff.

Ikizer, Elif G., Nairán Ramírez-Esparza & Ryan L. Boyd
2019.
#sendeanlat (#tellyourstory): Text Analyses of Tweets About Sexual Assault Experiences.
Sexuality Research and Social Policy 16:4
► pp. 463 ff.

Li, Jing, Lei Lei & Le Cheng
2020.
Mapping Evaluation, Appraisal and Stance in Discourse (2000–2015): A Bibliometric Analysis.
Glottotheory 10:1-2
► pp. 31 ff.

Markowitz, David M.
2021.
The Meaning Extraction Method: An Approach to Evaluate Content Patterns From Large-Scale Language Data.
Frontiers in Communication 6

Markowitz, David M. & Paul Slovic
2020.
Social, psychological, and demographic characteristics of dehumanization toward immigrants.
Proceedings of the National Academy of Sciences 117:17
► pp. 9260 ff.

Su, Hang
2016.
How products are evaluated? Evaluation in customer review texts.
Language Resources and Evaluation 50:3
► pp. 475 ff.

Su, Hang & Susan Hunston
2019.
Adjective complementation patterns and judgement: Aligning lexical-grammatical and discourse-semantic approaches in appraisal research.
Text & Talk 39:3
► pp. 415 ff.

[no author supplied]
This list is based on CrossRef data as of 19 march 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.