Chapter published in:
Metaphor and Metonymy in the Digital Age: Theory and methods for building repositories of figurative language
Edited by Marianna Bolognesi, Mario Brdar and Kristina Š. Despot
[Metaphor in Language, Cognition, and Communication 8] 2019
► pp. 99–121

VisMet and the crowd

What social tagging reveals about visual metaphors

| University of Oxford
| VU University Amsterdam
| VU University Amsterdam
This chapter describes the data collection and analysis related to a new digital resource soon to be added to the VisMet 1.0 corpus of visual metaphor (http://​www​.vismet​.org​/VisMet/, Bolognesi, van den Heerik, van den Berg, 2018), consisting of crowdsourced tags. Tags are keywords used by online coders, non-expert of metaphors, to annotate and describe the images to which they were exposed, for different amounts of seconds.The semantic information retrieved through this Social tagging experiment allows us to explore the type of information that users extract from visual metaphors, in a setting that better resembles the natural environment in which these images are usually experienced (i.e., for very limited amounts of seconds). We hereby provide methodological guidelines on this innovative procedure and report the results of our data collection and content analysis in which we manually classified the type of semantic information encoded in the tags.
Keywords: visual metaphor, social tagging, crowdsourcing task, metaphor processing
References

References

Ames, M., & Naaman, M.
(2007) Why we tag: motivations for annotation in mobile and online media. In Proceedings of the SIGCHI conference 2007 (pp. 971–980). New York: ACM. CrossrefGoogle Scholar
Aroyo, L., & Welty, C.
(2014) The three sides Crowd Truth. Human Computation, 1(1), 31–44. CrossrefGoogle Scholar
Artstein, R., & Poesio, M.
(2008) Inter-coder agreement for computational linguistics. Computational Linguistics, 34, 555–596. CrossrefGoogle Scholar
Bolognesi, M.
(2016) Flickr distributional tagspace: Evaluating the semantic spaces emerging from flickr tags distributions. In M. Jones (Ed.), Big data in cognitive science (pp. 144–173). New York: Routledge.Google Scholar
(2017a) Conceptual metaphors and metaphorical expressions in images. In A. Baicchi, & E. Pinelli (Eds.), Cognitive modelling in language and discourse across cultures (pp. 367–383). Cambridge: Cambridge Scholars.Google Scholar
(2017b) Using semantic feature norms to investigate how the visual and verbal modes afford metaphor construction and expression. Language and Cognition 9(3), 525–552. CrossrefGoogle Scholar
Bolognesi, M., Pilgram R., & Van den Heerik R.
(2017) Reliability in semantic categorization: The case of semantic feature norms classification. Behavior Research Methods 49(6), 1984–2001. CrossrefGoogle Scholar
Bolognesi, M., van den Heerik, R., & van den Berg, E.
(2018) VisMet: an online corpus of visual metaphors. In G. J. Steen (Ed.) , Visual metaphor: Structure and process (pp. 89–117). Amsterdam: John Benjamins Publishing Company.Google Scholar
Cattuto, C., Barrat, A., Baldassarri, A., Schehr, G., & Loreto, V.
(2009) Collective dynamics of social annotation. PNAS, 106(26), 10511–10515. CrossrefGoogle Scholar
El Refaie, E.
(2003) Understanding visual metaphors: The example of newspaper cartoons. Visual Communication, 2, 75–95. CrossrefGoogle Scholar
Forceville, C.
(1996) Pictorial metaphors in advertising. London: Routledge. CrossrefGoogle Scholar
Forceville, C., & Urios-Aparisi, E.
(Eds.) (2009) Multimodal metaphor. Berlin: Mouton de Gruyter. CrossrefGoogle Scholar
Heckner, M., Heilemann, M., & Wolff, C.
(2009) Personal information management vs. resource sharing: Towards a model of information behaviour in social tagging systems. Proceedings of the Third international AAAI conference on weblogs and social media (ICWSM) (pp. 42–49).
Jones, M. N.
(Ed.) (2016) Big data in cognitive science. New York: Taylor and Francis Psychology Press. CrossrefGoogle Scholar
Kremer, G., & Baroni, M.
(2011) A set of semantic norms for German and Italian. Behavior Research Methods, 43, 97–109. CrossrefGoogle Scholar
Lakoff, G., & Johnson, M.
(1980) Metaphors we live by. Chicago: University of Chicago Press.Google Scholar
Le Pair, R., & Van Mulken, M.
(2010) Effects of verbal anchoring in visual metaphors on perceived complexity and appreciation. In A. V. Prokhorov (Ed.), Topical issues of advertising: Theory and practice (Vol. II, pp. 26–36). Tambov: TSU.Google Scholar
Leder, H., Belke, B., Oeberst, A., & Augustin, D.
(2004) A model of aesthetic appreciation and aesthetic judgments. British Journal of Psychology, 95, 489–508. CrossrefGoogle Scholar
Marlow, C., Naaman, M., Boyd, D., & Davis, M.
(2006) HT06, Tagging Paper, Taxonomy, Flickr, Academic Article, ToRead. Proceedings of the seventeenth conference on hypertext and hypermedia (pp. 31–40). Crossref
McRae, K., Cree, G., Seidenberg, M., & McNorgan, C.
(2005) Semantic feature production norms for a large set of living and nonliving things. Behavioral Research Methods, 37, 547–559. CrossrefGoogle Scholar
Nov, O., Naaman, M., & Ye, C.
(2009). Motivational, structural and tenure factors that impact online community photo sharing. Proceedings of AAAI international conference on weblogs and social media (ICWSM 2009) .
Phillips, B., & McQuarrie, E.
(2004) Beyond visual metaphor: A new typology of visual rhetoric in advertising. Marketing Theory, 4, 113–136. CrossrefGoogle Scholar
Poesio, M.
(2004) Discourse annotation and semantic annotation in the GNOME corpus. Proceedings of the 2004 ACL workshop on discourse annotation (pp. 72–79). Crossref
Recchia, G., & Jones, M. N.
(2012) The semantic richness of abstract concepts. Frontiers in Human Neuroscience, 6, 315. CrossrefGoogle Scholar
Schilperoord, J.
(2018) Ways with pictures, visual incongruities and metaphor. In G. J. Steen (Ed.), Visual metaphor: Structure and process. Amsterdam: John Benjamins Publishing Company.Google Scholar
Steen, G. J.
(2008) The paradox of metaphor: Why we need a three-dimensional model of metaphor. Metaphor and Symbol, 23(4), 213–241. CrossrefGoogle Scholar
(2011) The contemporary theory of metaphor – now new and improved! Review of Cognitive Linguistics, 9(1), 26–64. CrossrefGoogle Scholar
Strohmaier M., Körner, C., & Kern, R.
(2012) Understanding why users tag: A survey of tagging motivation literature and results from an empirical study. Web Semantics, 17, 1–11. CrossrefGoogle Scholar
Šorm, E., & Steen, G. J.
(2018) VisMip: Towards a method for visual metaphor identification. In G. J. Steen (Ed.), Visual Metaphor: Structure and Process (pp. 47–88). Amsterdam: John Benjamins Publishing Company. CrossrefGoogle Scholar
(2013) Processing visual metaphor: A study in thinking out loud. Metaphor and the Social World, 3(1), 1–34. CrossrefGoogle Scholar
Van Hooft, A., van Mulken, M., & Nedergist, U.
(2013) Cultural differences? Visual metaphor in advertising: Comprehension and tolerance in ambiguity in four European countries. In S. Rosengren, M. Dahlén, & S. Okazaki (Eds.), Advances in advertising research (Vol. IV, pp. 351–364). Wiesbaden: Springer Gabler. CrossrefGoogle Scholar
Van Mulken, M., & Le Pair, R.
(2012) Appreciation and interpretation of visual metaphors in advertising across three European countries. In F. MacArthur, J. L. Oncins-Martínez, M. Sánchez-García, & A. M. Piquer-Píriz (Eds.), Metaphor in use. Context, culture, and communication (pp. 177–193). Amsterdam: John Benjamins Publishing Company. CrossrefGoogle Scholar
Van Mulken, M., Le Pair, R., & Forceville, C.
(2010) The impact of perceived complexity, deviation and comprehension on the appreciation of visual metaphor in advertising across three European countries. Journal of Pragmatics, 42, 3418–3430. CrossrefGoogle Scholar
Van Weelden, L., Maes, A., Schilperoord, J., & Swerts, M.
(2012) How object shape affects visual metaphor processing. Experimental Psychology, 59(6), 364–371. CrossrefGoogle Scholar