Cover not available
Article published In:
Pragmatics and Society
Vol. 13:2 (2022) ► pp.193–223

Clean room, uncomfortable bed

A corpus analysis of evaluation devices in hotel reviews

 | HSE University | University of Vigo
Online customer reviews, being an essential factor that determines success or failure in business, in particular in the tourism industry, demand close attention since the investigation of this type of discourse might bear some implications both for language specialists and for hotel managers. This paper is a quantitative and qualitative study of evaluation devices occurring in the corpus of hotel reviews from Booking​.com. The analysis is based on methods associated with corpus linguistics and Systemic Functional Linguistics (Halliday 1994; Eggins 2004), specifically, the Appraisal framework (Martin & White 2005), which is applied here to a new type of discourse, that of hotel ratings. The study focuses on the categories of Attitude, Graduation and Engagement suggested by Martin and White and investigates inscribed and invoked evaluation. In order to adapt the Attitude dimensions to the corpus under study, new subcategories were created as a contribution to evaluation theory. The practical result of the study is a list of evaluation devices employed in hotel reviews; the list may be useful for further corpus analyses and for designing systems for automated analysis of customer evaluation. Since online customer reviews are commonly used by hotels as a benchmark of their guests’ satisfaction and a valuable information source of features and services that need to be improved, the present study is also likely to make an important contribution to the industry.
Keywords: hotel reviews, evaluation theory, Appraisal framework, evaluation devices, corpus analysis
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1Evaluation
    • 2.2Corpus studies of online reviews
  • 3.Data and methods
  • 4.Application of the appraisal framework to hotel reviews
    • 4.1Attitude
      • 4.1.1Appreciation
      • 4.1.2Judgement
      • 4.1.3Affect
    • 4.2Graduation and engagement
  • 5.Results
    • 5.1Attitude
    • 5.2Graduation
    • 5.3Engagement
    • 5.4Targets of evaluation
    • 5.5Invoked evaluation
  • 6.Discussion and conclusion
  • Acknowledgements
  • Notes
  • References
Published online: 23 June 2022
DOI logo
https://doi.org/10.1075/ps.19046.smi
References (69)
References
Amin, Muslim, Zatilaziya Yahya, Wan F. A. Ismayatim, Siti Z. Nasharuddin, and Emma Kassim. 2013. “Service quality dimension and customer satisfaction: An empirical study in the Malaysian hotel industry.” Services Marketing Quarterly 34 (2): 115–125. DOI logoGoogle Scholar
Antony, Laurence. 2014. AntConc (Version 3.4.4) [Computer Software]. Tokyo: Waseda University. Available from [URL]
Banerjee, Snehasish, & Alton Y. K. Chua. 2016. “In search of patterns among travellers’ hotel ratings in TripAdvisor”. Tourism Management 531: 125–131. DOI logoGoogle Scholar
Banerjee, Snehasish, and Alton Y. K. Chua. 2017. “Theorizing the textual differences between authentic and fictitious reviews: Validation across positive, negative and moderate polarities.” Internet Research 27 (2): 321–337. DOI logoGoogle Scholar
Ben-Aaron, Diana. 2005. “Given and news: Evaluation in newspaper stories about national anniversaries.” Text – Interdisciplinary Journal for the Study of Discourse 25 (5): 691–718. DOI logoGoogle Scholar
Biber, Douglas, and Edward Finegan. 1988. “Adverbial stance types in English.” Discourse Processes 11 (1): 1–34. DOI logoGoogle Scholar
. 1989. “Styles of stance in English: lexical and grammatical marking of evidentiality and affect.” Text 9 (1): 93–124.Google Scholar
Biber, Douglas. 2006. “Stance in spoken and written university registers.” Journal of English for Academic Purposes 5 (2): 97–116. DOI logoGoogle Scholar
Booking.com URL: [URL] (Accessed April 8, 2018). DOI logo
Boucher, Jerry, and Charles E. Osgood. 1969. “The Pollyanna hypothesis.” Journal of Verbal Learning and Verbal Behavior 8 (1): 1–8. DOI logoGoogle Scholar
Brookes, Gavin, and Paul Baker. 2017. “What does patient feedback reveal about the NHS? A mixed methods study of comments posted to the NHS Choices online service.” BMJ Open 7 (4). DOI logoGoogle Scholar
Buhalis, Dimitrios, and Rob Law. 2008. “Progress in information technology and tourism management: 20 years on and 10 years after the Internet – The state of eTourism research.” Tourism Management 29 (4): 609–623. DOI logoGoogle Scholar
Camiciottoli, Belinda C. 2013. Rhetoric in Financial Discourse: A Linguistic Analysis of ICT-mediated Disclosure Genres. Amsterdam & New York: Rodopi. DOI logoGoogle Scholar
Cedefop’s European skills and jobs survey. 2014. URL: [URL] (Accessed February 11, 2019).
Cenni, Irene, and Patrick Goethals. 2017. “Negative hotel reviews on TripAdvisor: A cross-linguistic analysis.” Discourse, Context & Media 161: 22–30. DOI logoGoogle Scholar
Chipkin, Harvey. 2012, July 31. “Consumer trends 2012: review websites.” Travel Weekly. URL: [URL] Accessed: April 4, 2019.
Coffin, Caroline, and Kieran O’Halloran. 2006. “The role of appraisal and corpora in detecting covert evaluation.” Functions of Language 13 (1): 77–110. DOI logoGoogle Scholar
Conrad, Susan, and Douglas Biber. 2000. “Adverbial marking of stance in speech and writing.” In Evaluation in Text: Authorial Stance and the Construction of Discourse, ed. by Susan Hunston and Geoff Thompson, 56–73. Oxford: Oxford University Press.Google Scholar
Cook, Lory S., David E. Bowen, Richard B. Chase, Sriram Dasu, Doug M. Stewart, and David A. Tansik. 2002. “Human resource issues in service design.” Journal of Operations Management 20 (2): 159–174. DOI logoGoogle Scholar
Eggins, Suzanne. 2004. An Introduction to Systemic Functional Linguistics. London & New York: Continuum.Google Scholar
Halliday, M. A. K. 1994. An Introduction to Functional Grammar. London: Edward Arnold.Google Scholar
Hao, Jing, and Sally Humphrey. 2012. “The role of ‘coupling’ in biological experimental reports.” Linguistics and the Human Sciences 5 (2): 169–194. DOI logoGoogle Scholar
Ho-Dac, Nga N., Stephen J. Carson, and William L. Moore. 2013. “The effects of positive and negative online customer reviews: do brand strength and category maturity matter?” Journal of Marketing 77 (6): 37–53. DOI logoGoogle Scholar
Hoffman, Lily M., and Jiri Musil. 2009. “Prague, tourism and the post-industrial city.” A Great Cities Institute Working Papers. Chicago: University of Illinois.Google Scholar
Hood, Susan. 2012. “Voice and stance as APPRAISAL: Persuading and positioning in research writing across intellectual fields.” In Stance and Voice in Written Academic Genres, ed. by Ken Hyland and Carmen S. Guinda. London: Palgrave Macmillan. DOI logoGoogle Scholar
Hunston, Susan. 2011. Corpus Approaches to Evaluation: Phraseology and Evaluative Language. New York & London: Routledge.Google Scholar
Hyland, Ken. 1998. “Exploring corporate rhetoric: Metadiscourse in the CEO’s letter.” Journal of Business Communication 35 (2): 224–244. DOI logoGoogle Scholar
. 2005. Metadiscourse: Exploring Interaction in Writing. London and New York: Continuum.Google Scholar
Jackendoff, Ray. 1997. “Twistin’ the night away.” Language 73 (3): 534–559. DOI logoGoogle Scholar
Jeong, Miyoung, and Myunghee M. Jeon. 2008. “Customer reviews of hotel experiences through consumer generated media (CGM).” Journal of Hospitality & Leisure Marketing 17 (1–2): 121–138. DOI logoGoogle Scholar
Kachniewska, Magdalena. 2002. Zarządzanie Jakością Usług Turystycznych [Quality Management in the Tourist Industry]. Warszawa: Difin.Google Scholar
Kaltenbacher, Martin. 2006. “Culture related linguistic differences in tourist websites: the emotive and the factual – a corpus analysis within the framework of APPRAISAL.” In System and Corpus: Exploring Connections, ed. by Geoff Thompson and Susan Hunston, 269–92. London: Equinox.Google Scholar
Keuc, Natasha. 2014. “Factors influencing the decision on the measurement of customer satisfaction.” In Economic and Social Development: Book of Proceedings, ed. by Ivica Filipovic, Marina Klacmer Calopa, and Fran Galetic, 552–558. Varazdin: Varazdin Development and Entrepreneurship Agency.Google Scholar
Li, Huiying, Qiang Ye, and Rob Law. 2013. “Determinants of Customer Satisfaction in the Hotel Industry: An Application of Online Review Analysis.” Asia Pacific Journal of Tourism Research 18 (7): 784–802. DOI logoGoogle Scholar
Lin, Ingrid Y. 2004. “Evaluating a servicescape: the effect of cognition and emotion.” International Journal of Hospitality Management 23 (2): 163–178. DOI logoGoogle Scholar
Martin, James R. 2000. “Beyond exchange: Appraisal systems in English.” In Evaluation in Text. Authorial Stance and the Construction of Discourse, ed. by Susan Hunston and Geoff Thompson, 142–175. Oxford: Oxford University Press.Google Scholar
Martin, James R., and David Rose. 2003. Working with Discourse. Meaning beyond the Clause. London: Continuum.Google Scholar
Martin, James R., and Peter R. R. White. 2005. The Language of Evaluation: Appraisal in English. Basingstoke: Palgrave Macmillan. DOI logoGoogle Scholar
Martin-Fuentes, Eva, and Juan P. Mellinas. 2018. “Hotels that most rely on Booking. com–online travel agencies (OTAs) and hotel distribution channels.” Tourism Review. DOI logoGoogle Scholar
Mel’cuk, Igor. 1996. “Lexical functions: A tool for the description of lexical relations in a lexicon.” In Lexical Junctions in Lexicography and Natural Language Processing, ed. by Leo L. Wanner. 87-102. Amsterdam & Philadelphia: John Benjamins.Google Scholar
Mellinas, Juan P. 2019. “Dependency of Spanish urban hotels on Booking. com.” Tourism Analysis 24 (1): 3–12. DOI logoGoogle Scholar
Mohsin, Asad, and Tim Lockyer. 2010. “Customer perceptions of service quality in luxury hotels in New Delhi, India: an exploratory study.” International Journal of Contemporary Hospitality Management 22 (2): 160–173. DOI logoGoogle Scholar
Oteíza, Teresa. 2017. “The appraisal framework and discourse analysis”. In The Routledge Handbook of Systemic Functional Linguistics, ed. by Tom Bartlett and Gerard O’Grady, 481–496. New York: Routledge.Google Scholar
Oxford English Dictionary (OED) URL: [URL] Accessed July 18, 2021.
Ozimek, Irena, Julita Szlachciuk, Olena Kulykovets and Natalia Przeździecka-Czyżewska. 2017. “Factors Influencing the Choice and Quality Assessment of Hotel Facilities in Poland”. Folia Oeconomica Stetinensia 17 (1): 170–181. DOI logoGoogle Scholar
Pang, Bo, and Lillian Lee. 2008. “Opinion Mining and Sentiment Analysis.” Foundations and Trends in Information Retrieval 2 (1–2): 1–135. DOI logoGoogle Scholar
Pawley, Andrew, and Frances H. Syder. 1983. “Two puzzles for linguistic theory: nativelike selection and nativelike fluency.” In Language and Communication 7(1), ed. by Jack C. Richards and Richard W. Schmidt, 191–226. London: Longman.Google Scholar
Pierini, Patrizia. 2009. “Adjectives in tourism English on the web: a corpus-based study.” CÍRCULO de Lingüística Aplicada a la Comunicación 401: 93–116.Google Scholar
Reuland, Ruud, Janet Choudry, and Ans Fagel. 1985. “Research in the field of hospitality.” International Journal of Hospitality Management 4 (4): 141–146. DOI logoGoogle Scholar
Ryu, Kisang, and Soocheong S. Jang. 2007. “The effect of environmental perceptions on behavioral intentions through emotions: The case of upscale restaurants.” Journal of Hospitality & Tourism Research 31 (1): 56–72. DOI logoGoogle Scholar
Sammons, Gail, Pat Moreo, Lori F. Benson, and Fred Demicco. 1999. “Analysis of female business travelers’ selection of lodging accommodations.” Journal of Travel & Tourism Marketing 8 (1): 65–83. DOI logoGoogle Scholar
Silver, Marc. 2003. “The stance of stance: a critical look at ways stance is expressed and modeled in academic discourse.” Journal of English for Academic Purposes 2 (4): 359–374. DOI logoGoogle Scholar
Sinclair, John. 1991. Corpus, Concordance and Collocation. Oxford: Oxford University Press.Google Scholar
Smith, Trevor, and Andrew J. Spencer. 2011. “Predictors of value for money in Jamaican all-inclusive hotels.” International Journal of Humanities and Social Science 1 (4): 93–102.Google Scholar
Sparks, Beverley. 2001. “Managing service failure through recovery.” In Service quality management in hospitality, tourism and leisure, ed. by Jay Kandampully, Connie Mok, and Beverley Sparks, 193–219. New York & London: Routledge.Google Scholar
Su, Hang. 2016. “How products are evaluated? Evaluation in customer review texts.” Language Resources and Evaluation 50 (3): 475–495. DOI logoGoogle Scholar
Thompson, Geoff. 1996. Introducing Functional Grammar. London: Edward Arnold.Google Scholar
. 2014. “AFFECT and emotion, target-value mismatches, and Russian dolls: Refining the APPRAISAL model”. In Evaluation in context, ed. by Geoff Thompson and Laura Alba-Juez, 47–66. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Thompson, Geoff, and Laura Alba-Juez. 2014. Evaluation in Context. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Tian, Youfei. 2013. “Engagement in online hotel reviews: A comparative study.” Discourse, Context & Media 2 (4): 184–191. DOI logoGoogle Scholar
Torres, Edwin N., and Sheryl Kline. 2013. “From customer satisfaction to customer delight.” International Journal of Contemporary Hospitality Management 25 (5): 642–659. DOI logoGoogle Scholar
Tuan, Nham P., and Nguyen T. H. Linh. 2014. “Impact of service quality performance on customer satisfaction: A case study of Vietnam’s five star hotel.” ABAC Journal 34 (3): 53–70.Google Scholar
Vásquez, Camilla. 2011. “Complaints online: The case of TripAdvisor.” Journal of Pragmatics 43 (6): 1707–1717. DOI logoGoogle Scholar
Wang, Hongway, Pei Yin, Lijuan Zheng, and James N. K. Liu. 2014. “Sentiment classification of online reviews: using sentence-based language model.” Journal of Experimental & Theoretical Artificial Intelligence 26 (1): 13–31. DOI logoGoogle Scholar
White, Peter R. R. 2002. “Appraisal – the language of evaluation and stance.” In The Handbook of Pragmatics, ed. by Jef Verschueren, Jan-Ola Östman, Jan Blommaert, and Chris Bulcaen, 1–27. Amsterdam: John Benjamins.Google Scholar
2003. “Beyond modality and hedging: a dialogic view of the language of intersubjective stance.” Appraisal: Special Edition of Text 23 (2): 259–84. DOI logoGoogle Scholar
2011. “Appraisal.” In Discursive Pragmatics, ed. by Jan Zienkowski, Jan-Ola Östman, and Jef Verschueren, 14–36. Amsterdam: John Benjamins. DOI logoGoogle Scholar
Yang, Yang, Zhenxing Mao, and Jingyin Tang. 2018. “Understanding guest satisfaction with urban hotel location.” Journal of Travel Research 57 (2): 243–259. DOI logoGoogle Scholar
Cited by (3)

Cited by three other publications

Cheng, Shi
2024. A review of interpersonal metafunction studies in systemic functional linguistics (2012–2022). Journal of World Languages 10:3 ► pp. 623 ff. DOI logo
Contu, Giulia, Cinzia Dessí, Carla Massidda & Marco Ortu
2024. Online reviews explain differences in coastal and inland tourists’ satisfaction. Scientific Reports 14:1 DOI logo
Yasin, Dhiyaul Hafidh Fatah, Raudlotul Firdaus Fatah Yasin, Muhammad Husni Hasbullah & Ku Mohd Syarbaini Ku Yaacob
2024. An Analytical Inquiry into Ethical Dilemmas in Event Planning: Insights from Muslim Organizers. In Islamic Finance [Contributions to Management Science, ], ► pp. 477 ff. DOI logo

This list is based on CrossRef data as of 26 december 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.