Cover not available
Article published In:
Scientific Study of Literature
Vol. 10:1 (2020) ► pp.35–63

Captivating, splendid or instructive?

Assessing the impact of reading in online book reviews

ORCID logo | Huygens Institute for the History of the Netherlands
ORCID logo | Humanities Cluster, Royal Netherlands Academy of Arts and Sciences

What is the impact of reading fiction? We analyze online Dutch book reviews to detect overall affective impact, narrative feelings, response to style and reflection. We create a set of rules that analyze the reviews and detect the impact aspects. We evaluate the detection by asking raters about the presence of these aspects in reviews and comparing these ratings to our detection. Interrater agreements are weak to moderate; however, there is a significant correlation between the model’s predictions for all impact aspects except reflection. The detected impact correlates with book genres in the way one would expect: Narrative feelings are highest for thrillers, and stylistic response is highest for literary books. We can thus estimate some aspects of the response books evoke in readers. Initial results suggest that the appreciation of style is linked to reflection in the reader. However, the concepts underlying the impact categories need further exploration.

Keywords: online book review, reading impact, aesthetic feeling, narrative feeling, reflection, survey
Article outline
  • Reading impact
  • Online book discussion
    • Related research
    • Limitations
    • The impact predictor: Development
  • Data
    • Impact terms
    • Book aspect terms
    • Term – impact rule set
    • Issues
  • The impact predictor: Evaluation
    • Survey
    • Survey results
    • Interrater agreement
    • Rater-model agreement
    • Rule coverage
    • First results
      • Impact of genres
      • Aesthetic feeling and reflection
    • Discussion and prospects
  • Acknowledgements
  • Notes
  • References
Published online: 9 December 2020
DOI logo
https://doi.org/10.1075/ssol.20003.boo
References
Albrechtslund, A. -M. B.
(2019) Amazon, Kindle, and Goodreads: implications for literary consumption in the digital age. Consumption Markets & Culture, 1–16. DOI logoGoogle Scholar
Appel, M., Koch, E., Schreier, M., & Groeben, N.
(2002) Aspekte des Leseerlebens: Skalenentwicklung. Zeitschrift für Medienpsychologie, 141, 149–154. DOI logoGoogle Scholar
Bachmann-Stein, A.
(2015) Zur Praxis des Bewertens in Laienrezensionen. In H. Kaulen & C. Gansel (Eds.), Literaturkritik heute. Tendenzen–Traditionen–Vermittlung (pp. 77–91). Göttingen: V&R Unipress. DOI logoGoogle Scholar
Berthoud, E., & Elderkin, S.
(2013) The Novel Cure: An A to Z of Literary Remedies. Edinburgh, London: Canongate Books.Google Scholar
Boot, P.
(2011) Towards a Genre Analysis of Online Book Discussion: socializing, participation and publication in the Dutch booksphere. Selected Papers of Internet Research, IR 12.0. [URL]
(2012) Contextual factors in literary quality judgments: A quantitative analysis of an online writing community. Paper presented at Digital Humanities 2012. [URL]
(2017) A Database of Online Book Response and the Nature of the Literary Thriller. Paper presented at Digital Humanities 2017. [URL]
boyd, d., & Ellison, N. B.
(2008) Social network sites: Definition, history, and scholarship. Journal of computer-mediated Communication, 13(1), 210–230. DOI logoGoogle Scholar
Busselle, R., & Bilandzic, H.
(2009) Measuring narrative engagement. Media Psychology, 12(4), 321–347. DOI logoGoogle Scholar
Chevalier, J. A., & Mayzlin, D.
(2006) The effect of word of mouth on sales: Online book reviews. Journal of marketing research, 43(3), 345–354. DOI logoGoogle Scholar
Dal Cin, S., Zanna, M. P., & Fong, G. T.
(2004) Narrative persuasion and overcoming resistance. In E. S. Knowles & J. A. Linn (Eds.), Resistance and persuasion (pp. 175–191). Mahwah (NJ) & London: Lawrence Erlbaum Associates.Google Scholar
Domsch, S.
(2009) Critical genres. Generic changes of literary criticism in computer-mediated communication. In J. Giltrow & D. Stein (Eds.), Genres on the Internet: issues in the theory of genre (pp. 221–238). Amsterdam: John Benjamins Publishing Company. DOI logoGoogle Scholar
Driscoll, B., & Rehberg Sedo, D.
(2019) Faraway, So Close: Seeing the Intimacy in Goodreads Reviews. Qualitative Inquiry, 1077800418801375. DOI logoGoogle Scholar
Finn, E. F.
(2011) The Social Lives of Books: Literary Networks in Contemporary American Fiction. (PhD), Stanford University. Retrieved from [URL]
Gerrig, R. J., & Rapp, D. N.
(2004) Psychological processes underlying literary impact. Poetics Today, 25(2), 265–281. DOI logoGoogle Scholar
Goffman, E.
(1956) The presentation of self in everyday life. Edinburgh: University of Edinburgh. Social Sciences Research Centre.Google Scholar
Gruzd, A., & Rehberg Sedo, D. N.
(2012) #1b1t: Investigating Reading Practices at the Turn of the Twenty-first Century. Mémoires du livre, 3(2). [URL]
Gutjahr, P. C.
(2002) No Longer Left Behind: Amazon.com, Reader-Response, and the Changing Fortunes of the Christian Novel in America. Book History, 51, 209–236. DOI logoGoogle Scholar
Hajibayova, L.
(2019) Investigation of Goodreads’ reviews: Kakutanied, deceived or simply honest? Journal of Documentation, 75(3), 612–626. DOI logoGoogle Scholar
Hosoya, G., Schindler, I., Beermann, U., Wagner, V., Menninghaus, W., Eid, M., et al.
(2017) Mapping the conceptual domain of aesthetic emotion terms: A pile-sort study. Psychology of Aesthetics, Creativity, and the Arts, 11(4), 457. DOI logoGoogle Scholar
Jaakkola, M.
(2019) From re-viewers to me-viewers: The #Bookstagram review sphere on Instagram and the uses of the perceived platform and genre affordances. Interactions: Studies in Communication & Culture, 10(1–2), 91–110.Google Scholar
Jamieson, S.
(2004) Likert scales: How to (ab) use them. Medical education, 38(12), 1217–1218. DOI logoGoogle Scholar
Keen, S.
(2007) Empathy and the Novel. Oxford: Oxford University Press. DOI logoGoogle Scholar
Kidd, D. C., & Castano, E.
(2013) Reading literary fiction improves theory of mind. Science, 342(6156), 377–380. DOI logoGoogle Scholar
Knobloch-Westerwick, S., & Keplinger, C.
(2006) Mystery appeal: Effects of uncertainty and resolution on the enjoyment of mystery. Media Psychology, 8(3), 193–212. DOI logoGoogle Scholar
Knoop, C. A., Wagner, V., Jacobsen, T., & Menninghaus, W.
(2016) Mapping the aesthetic space of literature “from below”. Poetics, 561, 35–49. DOI logoGoogle Scholar
Koopman, E.
(2016) Effects of “Literariness” on Emotions and on Empathy and Reflection After Reading. Psychology of Aesthetics, Creativity, and the Arts, 10(1), 82–98. DOI logoGoogle Scholar
Koopman, E. M. E., & Hakemulder, F.
(2015) Effects of literature on empathy and self-reflection: A theoretical-empirical framework. Journal of Literary Theory, 9(1), 79–111. DOI logoGoogle Scholar
Kuijpers, M. M.
(2014) Absorbing stories. The effects of textual devices on absorption and evaluative responses. (Ph. D), Utrecht, Utrecht.Google Scholar
Kuijpers, M. M., Hakemulder, F., Tan, E. S., & Doicaru, M. M.
(2014) Exploring absorbing reading experiences. Scientific Study of Literature, 4(1). DOI logoGoogle Scholar
LeBreton, J. M., & Senter, J. L.
(2008) Answers to 20 questions about interrater reliability and interrater agreement. Organizational research methods, 11(4), 815–852. DOI logoGoogle Scholar
Lindell, M. K., and Brandt, C. J.
(1997) Measuring interrater agreement for ratings of a single target. Appl. Psychol. Meas. 211, 271–278. DOI logoGoogle Scholar
Martens, M.
(2016) Publishers, Readers, and Digital Engagement. London: Palgrave McMillan. DOI logoGoogle Scholar
McDonald, R.
(2007) The death of the critic. London, New York: Continuum International Publishing Group.Google Scholar
Mehling, G., Kellermann, A., Kellermann, H., & Rehfeldt, M.
(2018) Leserrezensionen auf amazon.de: Eine teilautomatisierte inhaltsanalytische Studie. Bamberg: University of Bamberg Press.Google Scholar
Miall, D. S., & Kuiken, D.
(2002) A feeling for fiction: Becoming what we behold. Poetics, 30(4), 221–241. DOI logoGoogle Scholar
Murray, S.
(2015) Charting the Digital Literary Sphere. Contemporary Literature, 56(2), 311–339. DOI logoGoogle Scholar
(2018) Reading Online: Updating the State of the Discipline. Book History, 21(1), 370–396. DOI logoGoogle Scholar
Naik, Y.
(2012) Finding good reads on Goodreads. Reference & User Services Quarterly, 51(4), 319–323. DOI logoGoogle Scholar
Naper, C.
(2016) Experiencing the Social Melodrama in the Twenty-first Century. In P. M. Rothbauer, K. I. Skjerdingstad, L. E. McKechnie, & K. Oterholm (Eds.), Plotting the Reading Experience: Theory/Practice/Politics. Waterloo (Ontario): Wilfrid Laurier University Press.Google Scholar
Norman, G.
(2010) Likert scales, levels of measurement and the “laws” of statistics. Advances in health sciences education, 15(5), 625–632. DOI logoGoogle Scholar
Oatley, K.
(1994) A taxonomy of the emotions of literary response and a theory of identification in fictional narrative. Poetics, 23(1–2), 53–74.Google Scholar
O’Neill, T. A.
(2017) An overview of interrater agreement on Likert scales for researchers and practitioners. Frontiers in psychology, 81, 777. DOI logoGoogle Scholar
Ott, M., Cardie, C., & Hancock, J.
(2012) Estimating the prevalence of deception in online review communities. Paper presented at the Proceedings of the 21st international conference on World Wide Web, Lyon, France. [URL]. DOI logo
Papacharissi, Z.
(2011) Conclusion: A networked self. In Z. Papacharissi (Ed.), A networked self: Identity, community, and culture on social network sites (pp. 304–318). New York: Routledge.Google Scholar
Pianzola, F., Rebora, S., & Lauer, G.
(2020) Wattpad as a resource for literary studies. Quantitative and qualitative examples of the importance of digital social reading and readers’ comments in the margins. PloS one, 15(1), e0226708. DOI logoGoogle Scholar
Rebora, S., Lendvai, P., & Kuijpers, M.
(2018) Reader experience labeling automatized: Text similarity classification of user-generated book reviews. Paper presented at the EADH 2019, Galway. [URL]
Rehfeldt, M.
(2017a) „Ganz große, poetische Literatur–Lesebefehl!“ Unterschiede und Gemeinsamkeiten von Amazon-Rezensionen zu U-und E-Literatur Lesen X. 0. Rezeptionsprozesse in der digitalen Gegenwart. Göttingen: V&R unipress. DOI logoGoogle Scholar
(2017b) Leserrezensionen als Rezeptionsdokumente. Zum Nutzen nicht-professioneller Literaturkritiken für die Literaturwissenschaft. In A. Bartl & M. Behmer (Eds.), Die Rezension. Aktuelle Tendenzen der Literaturkritik (pp. 275–289). Würzburg: Köningshausen und Neumann.Google Scholar
Ridenour, L., & Jeong, W.
(2016) Leveraging the power of social reading and big data: an analysis of co-read clusters of books on Goodreads. IConference 2016 Proceedings. [URL]. DOI logo
Ross, C. S.
(1999) Finding without seeking: the information encounter in the context of reading for pleasure. Information Processing & Management, 35(6), 783–799. DOI logoGoogle Scholar
Sabine, G., & Sabine, P.
(1983) Books That Made the Difference: What People Told Us. The Shoe String Press, Inc.Google Scholar
Sairio, A.
(2014) ‘No other reviews, no purchase, no wish list’: Book reviews and community norms on Amazon.com. Studies in Variation, Contacts and Change in English, 151. Retrieved from [URL]
Saricks, J. G.
(2005) Articulating a Book’s Appeal. Readers’ advisory service in the public library. Third Edition (pp. 40–73). Chicago: American Library Association.Google Scholar
Schindler, I., Hosoya, G., Menninghaus, W., Beermann, U., Wagner, V., Eid, M., et al.
(2017) Measuring aesthetic emotions: A review of the literature and a new assessment tool. PloS one, 12(6), e0178899. DOI logoGoogle Scholar
Smith, D.
(2004) Amazon reviewers brought to book. Guardian. Retrieved from [URL]
Spiteri, L. F., & Pecoskie, J.
(2016) Affective taxonomies of the reading experience: Using user-generated reviews for readers’ advisory. Proceedings of the Association for Information Science and Technology, 53(1), 1–9. DOI logoGoogle Scholar
Stein, S.
(2015) Laienliteraturkritik–Charakteristika und Funktionen von Laienrezensionen im Literaturbetrieb. In H. Kaulen & C. Gansel (Eds.), Literaturkritik Heute (pp. 59–76). Göttingen: V&R Unipress. DOI logoGoogle Scholar
Steiner, A.
(2010) Personal Readings and Public Texts: Book Blogs and Online Writing about Literature. Culture unbound, 2(28), 471–494. DOI logoGoogle Scholar
Streitfeld, D.
(2012 2012–08–25). The Best Book Reviews Money Can Buy. New York Times. Retrieved from [URL]
Taboada, M.
(2011) Stages in an online review genre. Text & Talk-An Interdisciplinary Journal of Language, Discourse & Communication Studies, 31(2), 247–269. DOI logoGoogle Scholar
Thelwall, M., & Kousha, K.
(2016) Goodreads: A social network site for book readers. Journal of the Association for Information Science and Technology, 68(4), 972–983. DOI logoGoogle Scholar
Thelwall, M.
(2019) Reader and author gender and genre in Goodreads. Journal of Librarianship and Information Science, 51(2), 403–430. DOI logoGoogle Scholar
Tóth, M., & Audunson, R.
(2012) Websites for booklovers as meeting places. Library Hi Tech, 30(4), 655–672. DOI logoGoogle Scholar
Van Noord, G.
(2006) At last parsing is now operational. Paper presented at the TALN06. Verbum Ex Machina. Actes de la 13e conference sur le traitement automatique des langues naturelles. [URL]
Van Putten-Brons, S., & Boot, P.
(2017) June is Dutch Literature Month!: Online Book Reviewers and Their Role in the Transmission of Dutch Literature to the English-Speaking World. In E. Brems, O. Réthelyi & T. van Kalmthout (Eds.), Doing Double Dutch (pp. 313–327). Leuven: Leuven University Press. DOI logoGoogle Scholar
Vásquez, C.
(2014) ‘Usually not one to complain but…’: Constructing identities in user-generated online reviews. The language of social media (pp. 65–90). London: Palgrave Macmillan. DOI logoGoogle Scholar
Wallace, L. K.
(2016) “My History, Finally Invented”: Nightwood and Its Publics. QED: A Journal in GLBTQ Worldmaking, 3(3), 71–94. DOI logoGoogle Scholar
Worrall, A.
(2019) “Connections above and beyond”: Information, translation, and community boundaries in Library Thing and Goodreads. Journal of the Association for Information Science and Technology, 70(7), 742–753. DOI logoGoogle Scholar