Article published In:
ConcentricVol. 48:1 (2022) ► pp.114–146
A corpus-pragmatics approach to evaluation in professor reviews
This study takes a corpus pragmatics approach to investigate the use of evaluative language in professor reviews, focusing on how review writers express evaluation through recurrent four-word sequences and the pragmatic functions of these sequences in positive and negative reviews on the website, RateMyProfessors.com. Based on an analysis of a 2.9-million-word corpus of free text comments, the findings indicate that positive reviews used more 4-grams, and more varied types, than negative ones. The 4-word sequences were found to carry out four pragmatic functions: attitudinal evaluation, reader engagement, referential expression, and discourse organization. While a similar distribution of the main functional categories was observed among the top 100 4-grams in both review types, with evaluative clusters being most predominant, distinctive intra-genre variations were found in the ways review writers employed different functional sub-categories. For example, positive reviews relied heavily on hedged suggestion 4-grams to engage readers, whereas negative reviews used directive 4-grams for the same purpose. These findings suggest the important role of multi-word sequences in the understanding of evaluative resources in professor reviews of different valence types.
Article outline
- 1.Introduction
- 2.Evaluation across genres
- 3.Theoretical framework
- 4.Methodology
- 4.1Corpora
- 4.2Analysis
- 5.Results
- 5.1High-frequency four-grams in positive and negative reviews
- 5.2Pragmatic functions of high-frequency 4-grams
- 5.2.1Attitudinal evaluation 4-grams
- 5.2.2Reader engagement 4-grams
- 5.2.3Referential 4-grams
- 5.2.4Discourse organizing 4-grams
- 6.Discussion and conclusion
- Acknowledgements
- Notes
-
References
References
Al-Ali, Mohammed Nahar
Bednarek, Monika
2008 ‘An
increasingly familiar tragedy’: Evaluative collocation and conflation. Functions of
Language 15.1:7–34. 
Biber, Douglas, Stig Johansson, Geoffrey Leech, Susan Conrad, and Edward Finegan
(eds.) 1999 Longman
of Grammar of Spoken and Written English. Essex, UK: Pearson Education.
Biber, Douglas, Susan Conrad, and Viviana Cortes
2004
If
you look at…: Lexical bundles in university teaching and textbooks. Applied
Linguistics 25.3:371–405.
Biber, Douglas, and Federica Barbieri
2007 Lexical
bundles in university spoken and written registers. English for Specific
Purposes 26.3:263–286.
Blackwell, Sarah E.
2010 Evaluation as a pragmatic act in
Spanish film narratives. Journal of
Pragmatics 42.11:2945–2963.
Bychkovska, Tetyana, and Joseph J. Lee
2017 At
the same time: Lexical bundles in L1 and L2 university student argumentative writing. Journal
of English for Academic
Purposes 301:38–52.
Cenni, Irene, and Patrick Goethals
2017 Negative
hotel reviews on TripAdvisor: A cross-linguistic analysis. Discourse, Context &
Media 161:22–30.
Clancy, Brian, and Anne O’Keeffe
2015 Pragmatics. The
Cambridge handbook of English corpus linguistics, ed. by Douglas Biber and Randi Reppen, 235–251. Cambridge, UK: Cambridge University Press.
Cortes, Viviana
2004 Lexical
bundles in published and student disciplinary writing: Examples from history and
biology. English for Specific
Purposes 23.4:397–423.
Cortes, Viviana, and Jack A. Hardy
2013 Analyzing
the semantic prosody and semantic preference of lexical bundles. Critical and Corpus-based
Approaches to Intercultural Rhetoric, ed. by Diane Belcher and Gayle Nelson, 180–201. University of Michigan Press.
Cunningham, Kelly J., and Stephanie Link
2021 Video
and text feedback on ESL writing: Understanding attitude and negotiating relationships. Journal
of Second Language Writing 521:100797.
Dam-Jensen, Helle, and Karen Korning Zethsen
2007 Pragmatic
patterns and the lexical system – A reassessment of evaluation in language. Journal of
Pragmatics 39.9:1608–1623.
Fortanet, Inmaculada
2008 Evaluative
language in peer review referee reports. Journal of English for Academic
Purposes 7.1:27–37.
Grabowski, Łukasz
2015 Keywords
and lexical bundles within English pharmaceutical discourse: A corpus-driven
description. English for Specific
Purposes 381:23–33.
Gray, Bethany, and Douglas Biber
2013 Lexical
frames in academic prose and conversation. International Journal of Corpus
Linguistics 18.1:109–136.
Haugh, Michael
2018 Corpus-based
metapragmatics. Methods in Pragmatics, ed. by Andreas H. Jucker, Klaus P. Schneider and Wolfram Bublitz, 619–644. Boston: De Gruyter Mouton.
Hewings, Martin
2004 An
‘important contribution’ or ‘tiresome reading’? A study of evaluation in peer reviews of journal article
submissions. Journal of Applied
Linguistics 1.3:247–274.
Hrežo, Vladislav
2020 Exploring
Phraseology in EU Legal Discourse. Language, Culture, Politics. International
Journal 1.5:29–52.
Hunston, Susan
2011 Corpus
Approaches to Evaluation: Phraseology and Evaluative Language. New York: Routledge.
Hyland, Ken
2008 As
can be seen: Lexical bundles and disciplinary variation. English for Specific
Purposes 27.1:4–21.
Hyland, Ken, and Polly Tse
2009 ‘The
leading journal in its field’: Evaluation in journal descriptions. Discourse
Studies 11.6:703–720.
Hyland, Ken
2012 Bundles
in academic discourse. Annual Review of Applied
Linguistics 321: 150–169.
Hyon, Sunny
2011 Evaluation
in tenure and promotion letters: Constructing faculty as communicators, stars, and
workers. Applied
Linguistics 32.4:389–407.
Itakura, Hiroko
2013 Hedging
praise in English and Japanese book reviews. Journal of
Pragmatics 45.1:131–148.
Itakura, Hiroko, and Amy B. Tsui
2011 Evaluation
in academic discourse: Managing criticism in Japanese and English book reviews. Journal of
Pragmatics 43.5:1366–1379.
Liu, Chen-Yu, and Hao-Jan Howard Chen
2020 Analyzing
the functions of lexical bundles in undergraduate academic lectures for pedagogical
use. English for Specific
Purposes 581:122–137.
Luzón, María José
2012 “Your argument is wrong”: A
contribution to the study of evaluation in academic weblogs. Text &
Talk 32.2:145–165.
Martin, James R., and Peter White
(eds.) 2005 The
Language of Evaluation: Appraisal in English. New York: Palgrave Macmillan.
Millar, Neil, and Susan Hunston
2015 Adjectives,
communities, and taxonomies of evaluative meaning. Functions of
Language 22.3:297–331.
Moreno, Ana I., and Lorena Suárez
2008 A
study of critical attitude across English and Spanish academic book reviews. Journal of English
for Academic
Purposes 7.1:15–26.
Pan, Fan, Randi Reppen, and Douglas Biber
2016 Comparing
patterns of L1 versus L2 English academic professionals: Lexical bundles in Telecommunications research
journals. Journal of English for Academic
Purposes 211:60–71.
Pomerantz, Anita
1986 Extreme
case formulations: A way of legitimizing claims. Human
Studies 9.2:219–229.
Reber, Jeffrey S., Robert D. Ridge, and Samuel D. Downs
2017 Perceptual
and behavioral effects of expectations formed by exposure to positive or negative Ratemyprofessors.com
evaluations. Cogent
Psychology 4.1:1–16.
Römer, Ute
2005 “This
seems somewhat counterintuitive, though…”: Negative evaluation in linguistic book reviews by male and female
authors. Strategies in Academic Discourse, ed.
by Elena Tognini-Bonelli and Gabriella Del Lungo Camiciotti, 97–115. Philadelphia, PA: John Benjamins.
2008 Identification
impossible?: A corpus approach to realizations of evaluative meaning in academic
writing. Functions of
Language 15.1:115–130.
Rühlemann, Christoph, and Karin Aijmer
2015 Introduction.
Corpus pragmatics: Laying the foundations. Corpus Pragmatics: A
handbook, ed. by Karin Aijmer and Christoph Rühlemann, 1–26. Cambridge: Cambridge University Press.
Scherr, Sebastian, Philipp Müller, and Victoria Fast
2013 Single
comments or average ratings: Which elements of RateMyProfessors. com™ shape university students’ judgments and course choice
intentions? Educational Assessment, Evaluation and
Accountability 25.2:131–141.
Scott, Mike
2017 WordSmith
Tools Version 7. Stroud: Lexical Analysis Software.
Simpson-Vlach, Rita, and Nick C. Ellis
2010 An
academic formulas list: New methods in phraseology research. Applied
Linguistics 31.4:487–512.
Siyanova-Chanturia, Anna, and Taha Omidian
2019 Key
issues in researching multiword items. The Routledge Handbook of Vocabulary
Studies, ed. by Stuart Webb, 529–544. New York: Routledge.
Skalicky, Stephen
2013 Was
this analysis helpful? A genre analysis of the Amazon.com discourse community and its “most helpful” product
reviews. Discourse, Context &
Media 2.2:84–93.
Song, Chengfang
2015 Lexical
realization of attitudinal meaning and its coding in discourse analysis. Text &
Talk 35.3:383–405.
Stubbs, Michael
2001 Words
and Phrases: Corpus Studies of Lexical
Semantics. Oxford: Blackwell publishers.
Su, Hang
2016 How
products are evaluated? Evaluation in customer review texts. Language Resources and
Evaluation 50.3:475–495.
Su, Hang, and Susan Hunston
2019 Language
patterns and attitude revisited: Adjective patterns, attitude and appraisal. Functions of
Language 26.3:343–371.
Subtirelu, Nicholas
2018 Exploring
the intersection of gender and race in evaluations of mathematics instructors on
ratemyprofessors.com. Studies in Corpus-based Sociolinguistics, ed.
by Eric Friginal, 219–235. New York: Routledge.
Taboada, Maite
2011 Stages
in an online review genre. Text &
Talk 31.2:247–269.
Taboada, Maite, Marta Carretero, and Jennifer Hinnell
2014 Loving
and hating the movies in English, German and Spanish. Languages in
Contrast 14.1:127–161.
Taboada, Maite
2016 Sentiment
analysis: An overview from linguistics. Annual Review of
Linguistics 21:325–347.
Thominet, Luke
2016 10/10
would review again: Variation in the player game review genre. Technical Communication
Quarterly 25.3:167–177.
Thompson, Geoffrey, and Susan Hunston
2000 Evaluation:
An introduction. Evaluation in text: Authorial Stance and the Construction of
Discourse, ed. by Susan Hunston and Geoffrey Thompson, 1–27. New York: Oxford University Press.
Vásquez, Camilla
2011 Complaints
online: The case of TripAdvisor. Journal of
Pragmatics 43.6:1707–1717.
2014 The
Discourse of Online Consumer Reviews. New York: Bloomsbury Publishing.
White, Peter
2000 Dialogue
and inter-subjectivity: Reinterpreting the semantics of modality and hedging. Dialogue Analysis
VII: Working with Dialogue, ed. by Malcolm Coulthard, Janet Cotterill and Frances Rock, 67–80. Tubingen: Neimeyer.
Wright, Heidi R.
2019 Lexical bundles in stand-alone
literature reviews: Sections, frequencies, and functions. English for Specific
Purposes 541:1–14.
Yin, Xiaoyi, and Shuangling Li
2021 Lexical
bundles as an intradisciplinary and interdisciplinary mark: A corpus-based study of research articles from business, biology,
and applied linguistics. Applied Corpus
Linguistics 1.1:100006.