Article published in:
Scientific Study of Literature
Vol. 10:1 (2020) ► pp. 35–63

Captivating, splendid or instructive?

Assessing the impact of reading in online book reviews

| Huygens Institute for the History of the Netherlands
| Humanities Cluster, Royal Netherlands Academy of Arts and Sciences
What is the impact of reading fiction? We analyze online Dutch book reviews to detect overall affective impact, narrative feelings, response to style and reflection. We create a set of rules that analyze the reviews and detect the impact aspects. We evaluate the detection by asking raters about the presence of these aspects in reviews and comparing these ratings to our detection. Interrater agreements are weak to moderate; however, there is a significant correlation between the model’s predictions for all impact aspects except reflection. The detected impact correlates with book genres in the way one would expect: Narrative feelings are highest for thrillers, and stylistic response is highest for literary books. We can thus estimate some aspects of the response books evoke in readers. Initial results suggest that the appreciation of style is linked to reflection in the reader. However, the concepts underlying the impact categories need further exploration.
Keywords: online book review, reading impact, aesthetic feeling, narrative feeling, reflection, survey
Article outline
  • Reading impact
  • Online book discussion
    • Related research
    • Limitations
    • The impact predictor: Development
  • Data
    • Impact terms
    • Book aspect terms
    • Term – impact rule set
    • Issues
  • The impact predictor: Evaluation
    • Survey
    • Survey results
    • Interrater agreement
    • Rater-model agreement
    • Rule coverage
    • First results
      • Impact of genres
      • Aesthetic feeling and reflection
    • Discussion and prospects
  • Acknowledgements
  • Notes
  • References
Published online: 09 December 2020
https://doi.org/10.1075/ssol.20003.boo
References
Albrechtslund, A. -M. B.
(2019) Amazon, Kindle, and Goodreads: implications for literary consumption in the digital age. Consumption Markets & Culture, 1–16. CrossrefGoogle Scholar
Appel, M., Koch, E., Schreier, M., & Groeben, N.
(2002) Aspekte des Leseerlebens: Skalenentwicklung. Zeitschrift für Medienpsychologie, 141, 149–154. CrossrefGoogle Scholar
Bachmann-Stein, A.
(2015) Zur Praxis des Bewertens in Laienrezensionen. In H. Kaulen & C. Gansel (Eds.), Literaturkritik heute. Tendenzen–Traditionen–Vermittlung (pp. 77–91). Göttingen: V&R Unipress. CrossrefGoogle Scholar
Berthoud, E., & Elderkin, S.
(2013) The Novel Cure: An A to Z of Literary Remedies. Edinburgh, London: Canongate Books.Google Scholar
Boot, P.
(2011) Towards a Genre Analysis of Online Book Discussion: socializing, participation and publication in the Dutch booksphere. Selected Papers of Internet Research, IR 12.0. https://​journals​.uic​.edu​/ojs​/index​.php​/spir​/article​/viewFile​/9076​/7167
(2012) Contextual factors in literary quality judgments: A quantitative analysis of an online writing community. Paper presented at Digital Humanities 2012. https://​pure​.knaw​.nl​/ws​/files​/474742​/2012​_boot​_02​_contextualfactors​.pdf
(2017) A Database of Online Book Response and the Nature of the Literary Thriller. Paper presented at Digital Humanities 2017. https://​dh2017​.adho​.org​/abstracts​/208​/208​.pdf
boyd, d., & Ellison, N. B.
(2008) Social network sites: Definition, history, and scholarship. Journal of computer-mediated Communication, 13(1), 210–230. CrossrefGoogle Scholar
Busselle, R., & Bilandzic, H.
(2009) Measuring narrative engagement. Media Psychology, 12(4), 321–347. CrossrefGoogle Scholar
Chevalier, J. A., & Mayzlin, D.
(2006) The effect of word of mouth on sales: Online book reviews. Journal of marketing research, 43(3), 345–354. CrossrefGoogle Scholar
Dal Cin, S., Zanna, M. P., & Fong, G. T.
(2004) Narrative persuasion and overcoming resistance. In E. S. Knowles & J. A. Linn (Eds.), Resistance and persuasion (pp. 175–191). Mahwah (NJ) & London: Lawrence Erlbaum Associates.Google Scholar
Domsch, S.
(2009) Critical genres. Generic changes of literary criticism in computer-mediated communication. In J. Giltrow & D. Stein (Eds.), Genres on the Internet: issues in the theory of genre (pp. 221–238). Amsterdam: John Benjamins Publishing Company. CrossrefGoogle Scholar
Driscoll, B., & Rehberg Sedo, D.
(2019) Faraway, So Close: Seeing the Intimacy in Goodreads Reviews. Qualitative Inquiry, 1077800418801375. CrossrefGoogle Scholar
Finn, E. F.
(2011) The Social Lives of Books: Literary Networks in Contemporary American Fiction. (PhD), Stanford University. Retrieved from https://​stacks​.stanford​.edu​/file​/druid:mk148kb9574​/The%20Social%20Lives%20of%20Books​-augmented​.pdf
Gerrig, R. J., & Rapp, D. N.
(2004) Psychological processes underlying literary impact. Poetics Today, 25(2), 265–281. CrossrefGoogle Scholar
Goffman, E.
(1956) The presentation of self in everyday life. Edinburgh: University of Edinburgh. Social Sciences Research Centre.Google Scholar
Gruzd, A., & Rehberg Sedo, D. N.
(2012) #1b1t: Investigating Reading Practices at the Turn of the Twenty-first Century. Mémoires du livre, 3(2). https://​www​.erudit​.org​/en​/journals​/memoires​/2012​-v3​-n2​-memoires0117​/1009347ar​.pdf
Gutjahr, P. C.
(2002) No Longer Left Behind: Amazon.com, Reader-Response, and the Changing Fortunes of the Christian Novel in America. Book History, 51, 209–236. CrossrefGoogle Scholar
Hajibayova, L.
(2019) Investigation of Goodreads’ reviews: Kakutanied, deceived or simply honest? Journal of Documentation, 75(3), 612–626. CrossrefGoogle Scholar
Hosoya, G., Schindler, I., Beermann, U., Wagner, V., Menninghaus, W., Eid, M., et al.
(2017) Mapping the conceptual domain of aesthetic emotion terms: A pile-sort study. Psychology of Aesthetics, Creativity, and the Arts, 11(4), 457. CrossrefGoogle Scholar
Jaakkola, M.
(2019) From re-viewers to me-viewers: The #Bookstagram review sphere on Instagram and the uses of the perceived platform and genre affordances. Interactions: Studies in Communication & Culture, 10(1–2), 91–110.Google Scholar
Jamieson, S.
(2004) Likert scales: How to (ab) use them. Medical education, 38(12), 1217–1218. CrossrefGoogle Scholar
Keen, S.
(2007) Empathy and the Novel. Oxford: Oxford University Press. CrossrefGoogle Scholar
Kidd, D. C., & Castano, E.
(2013) Reading literary fiction improves theory of mind. Science, 342(6156), 377–380. CrossrefGoogle Scholar
Knobloch-Westerwick, S., & Keplinger, C.
(2006) Mystery appeal: Effects of uncertainty and resolution on the enjoyment of mystery. Media Psychology, 8(3), 193–212. CrossrefGoogle Scholar
Knoop, C. A., Wagner, V., Jacobsen, T., & Menninghaus, W.
(2016) Mapping the aesthetic space of literature “from below”. Poetics, 561, 35–49. CrossrefGoogle Scholar
Koopman, E.
(2016) Effects of “Literariness” on Emotions and on Empathy and Reflection After Reading. Psychology of Aesthetics, Creativity, and the Arts, 10(1), 82–98. CrossrefGoogle Scholar
Koopman, E. M. E., & Hakemulder, F.
(2015) Effects of literature on empathy and self-reflection: A theoretical-empirical framework. Journal of Literary Theory, 9(1), 79–111. CrossrefGoogle Scholar
Kuijpers, M. M.
(2014) Absorbing stories. The effects of textual devices on absorption and evaluative responses. (Ph. D), Utrecht, Utrecht.Google Scholar
Kuijpers, M. M., Hakemulder, F., Tan, E. S., & Doicaru, M. M.
(2014) Exploring absorbing reading experiences. Scientific Study of Literature, 4(1). CrossrefGoogle Scholar
LeBreton, J. M., & Senter, J. L.
(2008) Answers to 20 questions about interrater reliability and interrater agreement. Organizational research methods, 11(4), 815–852. CrossrefGoogle Scholar
Lindell, M. K., and Brandt, C. J.
(1997) Measuring interrater agreement for ratings of a single target. Appl. Psychol. Meas. 211, 271–278. CrossrefGoogle Scholar
Martens, M.
(2016) Publishers, Readers, and Digital Engagement. London: Palgrave McMillan. CrossrefGoogle Scholar
McDonald, R.
(2007) The death of the critic. London, New York: Continuum International Publishing Group.Google Scholar
Mehling, G., Kellermann, A., Kellermann, H., & Rehfeldt, M.
(2018) Leserrezensionen auf amazon.de: Eine teilautomatisierte inhaltsanalytische Studie. Bamberg: University of Bamberg Press.Google Scholar
Miall, D. S., & Kuiken, D.
(2002) A feeling for fiction: Becoming what we behold. Poetics, 30(4), 221–241. CrossrefGoogle Scholar
Murray, S.
(2015) Charting the Digital Literary Sphere. Contemporary Literature, 56(2), 311–339. CrossrefGoogle Scholar
(2018) Reading Online: Updating the State of the Discipline. Book History, 21(1), 370–396. CrossrefGoogle Scholar
Naik, Y.
(2012) Finding good reads on Goodreads. Reference & User Services Quarterly, 51(4), 319–323. CrossrefGoogle Scholar
Naper, C.
(2016) Experiencing the Social Melodrama in the Twenty-first Century. In P. M. Rothbauer, K. I. Skjerdingstad, L. E. McKechnie, & K. Oterholm (Eds.), Plotting the Reading Experience: Theory/Practice/Politics. Waterloo (Ontario): Wilfrid Laurier University Press.Google Scholar
Norman, G.
(2010) Likert scales, levels of measurement and the “laws” of statistics. Advances in health sciences education, 15(5), 625–632. CrossrefGoogle Scholar
Oatley, K.
(1994) A taxonomy of the emotions of literary response and a theory of identification in fictional narrative. Poetics, 23(1–2), 53–74.Google Scholar
O’Neill, T. A.
(2017) An overview of interrater agreement on Likert scales for researchers and practitioners. Frontiers in psychology, 81, 777. CrossrefGoogle Scholar
Ott, M., Cardie, C., & Hancock, J.
(2012) Estimating the prevalence of deception in online review communities. Paper presented at the Proceedings of the 21st international conference on World Wide Web, Lyon, France. https://​dl​.acm​.org​/citation​.cfm​?id​=2187864. Crossref
Papacharissi, Z.
(2011) Conclusion: A networked self. In Z. Papacharissi (Ed.), A networked self: Identity, community, and culture on social network sites (pp. 304–318). New York: Routledge.Google Scholar
Pianzola, F., Rebora, S., & Lauer, G.
(2020) Wattpad as a resource for literary studies. Quantitative and qualitative examples of the importance of digital social reading and readers’ comments in the margins. PloS one, 15(1), e0226708. CrossrefGoogle Scholar
Rebora, S., Lendvai, P., & Kuijpers, M.
(2018) Reader experience labeling automatized: Text similarity classification of user-generated book reviews. Paper presented at the EADH 2019, Galway. https://​eadh2018​.exordo​.com​/files​/papers​/76​/final​_draft​/Goodreads​_EADH2018​_Final​.pdf
Rehfeldt, M.
(2017a) „Ganz große, poetische Literatur–Lesebefehl!“ Unterschiede und Gemeinsamkeiten von Amazon-Rezensionen zu U-und E-Literatur Lesen X. 0. Rezeptionsprozesse in der digitalen Gegenwart. Göttingen: V&R unipress. CrossrefGoogle Scholar
(2017b) Leserrezensionen als Rezeptionsdokumente. Zum Nutzen nicht-professioneller Literaturkritiken für die Literaturwissenschaft. In A. Bartl & M. Behmer (Eds.), Die Rezension. Aktuelle Tendenzen der Literaturkritik (pp. 275–289). Würzburg: Köningshausen und Neumann.Google Scholar
Ridenour, L., & Jeong, W.
(2016) Leveraging the power of social reading and big data: an analysis of co-read clusters of books on Goodreads. IConference 2016 Proceedings. https://​www​.ideals​.illinois​.edu​/bitstream​/handle​/2142​/89314​/Ridenour292​.pdf​?sequence​=1. Crossref
Ross, C. S.
(1999) Finding without seeking: the information encounter in the context of reading for pleasure. Information Processing & Management, 35(6), 783–799. CrossrefGoogle Scholar
Sabine, G., & Sabine, P.
(1983) Books That Made the Difference: What People Told Us. The Shoe String Press, Inc.Google Scholar
Sairio, A.
(2014) ‘No other reviews, no purchase, no wish list’: Book reviews and community norms on Amazon.com. Studies in Variation, Contacts and Change in English, 151. Retrieved from http://​www​.helsinki​.fi​/varieng​/series​/volumes​/15​/sairio/
Saricks, J. G.
(2005) Articulating a Book’s Appeal. Readers’ advisory service in the public library. Third Edition (pp. 40–73). Chicago: American Library Association.Google Scholar
Schindler, I., Hosoya, G., Menninghaus, W., Beermann, U., Wagner, V., Eid, M., et al.
(2017) Measuring aesthetic emotions: A review of the literature and a new assessment tool. PloS one, 12(6), e0178899. CrossrefGoogle Scholar
Smith, D.
(2004) Amazon reviewers brought to book. Guardian. Retrieved from https://​www​.theguardian​.com​/technology​/2004​/feb​/15​/books​.booksnews
Spiteri, L. F., & Pecoskie, J.
(2016) Affective taxonomies of the reading experience: Using user-generated reviews for readers’ advisory. Proceedings of the Association for Information Science and Technology, 53(1), 1–9. CrossrefGoogle Scholar
Stein, S.
(2015) Laienliteraturkritik–Charakteristika und Funktionen von Laienrezensionen im Literaturbetrieb. In H. Kaulen & C. Gansel (Eds.), Literaturkritik Heute (pp. 59–76). Göttingen: V&R Unipress. CrossrefGoogle Scholar
Steiner, A.
(2010) Personal Readings and Public Texts: Book Blogs and Online Writing about Literature. Culture unbound, 2(28), 471–494. CrossrefGoogle Scholar
Streitfeld, D.
(2012 2012–08–25). The Best Book Reviews Money Can Buy. New York Times. Retrieved from https://​www​.nytimes​.com​/2012​/08​/26​/business​/book​-reviewers​-for​-hire​-meet​-a​-demand​-for​-online​-raves​.html
Taboada, M.
(2011) Stages in an online review genre. Text & Talk-An Interdisciplinary Journal of Language, Discourse & Communication Studies, 31(2), 247–269. CrossrefGoogle Scholar
Thelwall, M., & Kousha, K.
(2016) Goodreads: A social network site for book readers. Journal of the Association for Information Science and Technology, 68(4), 972–983. CrossrefGoogle Scholar
Thelwall, M.
(2019) Reader and author gender and genre in Goodreads. Journal of Librarianship and Information Science, 51(2), 403–430. CrossrefGoogle Scholar
Tóth, M., & Audunson, R.
(2012) Websites for booklovers as meeting places. Library Hi Tech, 30(4), 655–672. CrossrefGoogle Scholar
Van Noord, G.
(2006) At last parsing is now operational. Paper presented at the TALN06. Verbum Ex Machina. Actes de la 13e conference sur le traitement automatique des langues naturelles. https://​www​.let​.rug​.nl​/~vannoord​/papers​/taln​.pdf
Van Putten-Brons, S., & Boot, P.
(2017) June is Dutch Literature Month!: Online Book Reviewers and Their Role in the Transmission of Dutch Literature to the English-Speaking World. In E. Brems, O. Réthelyi & T. van Kalmthout (Eds.), Doing Double Dutch (pp. 313–327). Leuven: Leuven University Press. CrossrefGoogle Scholar
Vásquez, C.
(2014) ‘Usually not one to complain but…’: Constructing identities in user-generated online reviews. The language of social media (pp. 65–90). London: Palgrave Macmillan. CrossrefGoogle Scholar
Wallace, L. K.
(2016) “My History, Finally Invented”: Nightwood and Its Publics. QED: A Journal in GLBTQ Worldmaking, 3(3), 71–94. CrossrefGoogle Scholar
Worrall, A.
(2019) “Connections above and beyond”: Information, translation, and community boundaries in Library Thing and Goodreads. Journal of the Association for Information Science and Technology, 70(7), 742–753. CrossrefGoogle Scholar