Chapter published in:
Operationalizing Iconicity
Edited by Pamela Perniss, Olga Fischer and Christina Ljungberg
[Iconicity in Language and Literature 17] 2020
► pp. 213–230

Show me how you feel

Iconicity and systematicity in visual morphology

| University of British Columbia
| University of Arizona
| University of British Columbia
| University of British Columbia
| Tilburg University

Visual representations are an ideal place to look at the balance of iconicity and systematicity, given that visual images often look like what they represent but also can be characterized along a scale from highly photorealistic to highly schematic or cartoony. Here we examine the contrast between different styles by presenting participants with “visual morphology” of upfixes – representations like hearts or lightbulbs that float above faces – where the face and/or upfix are either cartoony or photorealistic. Overall, we find that cartoony images, relative to photorealistic images, are easier to process, and therefore demand less attention and facilitate responding. We argue that these results support the view that drawings, and visual morphology, draw on schematic knowledge stored in long-term memory.

Article outline
  • 1.Introduction
  • 2.Methods
    • 2.1Stimuli
    • 2.2Participants
    • 2.3Procedure
    • 2.4Data analysis
  • 3.Results
    • 3.1Accuracy
    • 3.2Response times
    • 3.3Eye-tracking: Dwell times
    • 3.4Eye-tracking: Number of fixations
    • 3.5Eye-tracking: Entry time
  • 4.Discussion
  • 5.Conclusion
  • References
Published online: 13 May 2020
https://doi.org/10.1075/ill.17.13ken
References
Bateman, John A., and Janina Wildfeuer
2014 “Defining units of analysis for the systematic analysis of comics: A discourse-based approach.” Studies in Comics 5 (2): 373–403. CrossrefGoogle Scholar
Churches, Owen, Mike Nicholls, Myra Thiessen, Mark Kohler, and Hannah Keage
2014 “Emoticons in mind: An event-related potential study.” Social Neuroscience 9 (2): 196–202. CrossrefGoogle Scholar
Cohn, Neil.
2012 “Explaining “I can’t draw”: Parallels between the structure and development of language and drawing.” Human Development 55 (4): 167–192. CrossrefGoogle Scholar
2013 The visual language of comics: Introduction to the structure and cognition of sequential images. London, UK: Bloomsbury.Google Scholar
2018 “Combinatorial morphology in visual languages.” In The Construction of Words: Advances in Construction Morphology, edited by Geert Booij, 175–199. London: Springer. CrossrefGoogle Scholar
Cohn, Neil, and Sean Ehly
2016 “The vocabulary of manga: Visual morphology in dialects of Japanese Visual Language.” Journal of Pragmatics 92: 17–29. CrossrefGoogle Scholar
Cohn, Neil, and Stephen Maher
2015 “The notion of the motion: The neurocognition of motion lines in visual narratives.” Brain Research 1601: 73–84. CrossrefGoogle Scholar
Cohn, Neil, Beena Murthy, and Tom Foulsham
2016 “Meaning above the head: Combinatorial constraints on the visual vocabulary of comics.” Journal of Cognitive Psychology 28 (5): 559–574. CrossrefGoogle Scholar
Faja, Susan, Sara Jane Webb, Kristen Merkle, Elizabeth Aylward, and Geraldine Dawson
2009 “Brief Report: Face Configuration Accuracy and Processing Speed Among Adults with High-Functioning Autism Spectrum Disorders.” Journal of Autism and Developmental Disorders 39 (3): 532–538. CrossrefGoogle Scholar
Folk, Charles L, Roger W Remington, and James C Johnston
1992 “Involuntary covert orienting is contingent on attentional control settings.” Journal of Experimental Psychology: Human perception and performance 18 (4): 1030.Google Scholar
Forceville, Charles.
2011 “Pictorial runes in Tintin and the Picaros .” Journal of Pragmatics 43 (3): 875–890. CrossrefGoogle Scholar
Goffaux, Valérie, and Bruno Rossion
2006 “Faces are” spatial – “holistic face perception is supported by low spatial frequencies.” Journal of Experimental Psychology: Human Perception and Performance 32 (4): 1023–1039.Google Scholar
Gray, Katie LH, Wendy J Adams, Nicholas Hedger, Kristiana E Newton, and Matthew Garner
2013 “Faces and awareness: Low-level, not emotional factors determine perceptual dominance.” Emotion 13 (3): 537–544. CrossrefGoogle Scholar
Hoang Duc, Albert, Paul Bays, and Masud Husain
2008 “Chapter 5.5–Eye movements as a probe of attention.” In Progress in Brain Research, edited by Christopher Kennard and R. John Leigh, 403–411. Elsevier.Google Scholar
Jackendoff, Ray, and Jenny Audring
2016 “Morphological schemas.” The Mental Lexicon 11 (3): 467–493. CrossrefGoogle Scholar
Jakobson, Roman
1941/1962 “Kindersprache, Aphasie, und allgemeine Lautgesetze.” In Selected writings I, 328–401. The Hague: Mouton.Google Scholar
Joseph, Robert M., and James Tanaka
2003 “Holistic and part-based face recognition in children with autism.” Journal of Child Psychology and Psychiatry 44 (4): 529–542. CrossrefGoogle Scholar
Kendall, Lia N., Quentin Raffaelli, Alan Kingstone, and Rebecca M. Todd
2016 “Iconic faces are not real faces: enhanced emotion detection and altered neural processing as faces become more iconic.” Cognitive Research: Principles and Implications 1 (1): 1–14. Crossref.Google Scholar
Kowler, Eileen.
2011 “Eye movements: The past 25 years.” Vision Research 51 (13): 1457–1483. CrossrefGoogle Scholar
McCloud, Scott.
1993 Understanding Comics: The Invisible Art. New York, NY: Harper Collins.Google Scholar
Newton, Douglas P.
1985 “Children’s Perception of Pictorial Metaphor.” Educational Psychology 5 (2): 179–185. CrossrefGoogle Scholar
Peirce, Charles Sanders
1931 “Division of Signs.” In Collected Papers of Charles Sanders Peirce: Vol. 2: Elements of Logic., edited by Charles Hartshorne and Paul Weiss, 134–173. Cambridge, MA: Harvard University Press.Google Scholar
Prazak, Emily R., and E. Darcy Burgund
2014 “Keeping it real: Recognizing expressions in real compared to schematic faces.” Visual Cognition 22 (5): 737–750. CrossrefGoogle Scholar
Rosset, Delphine B., Cécilie Rondan, David Da Fonseca, Andreia Santos, Brigitte Assouline, and Christine Deruelle
2008 “Typical Emotion Processing for Cartoon but not for Real Faces in Children with Autistic Spectrum Disorders.” Journal of Autism and Developmental Disorders 38 (5): 919–925. CrossrefGoogle Scholar
Rosset, Delphine B., Andreia Santos, David Da Fonseca, François Poinso, Kate O’Connor, and Christine Deruelle
2010 “Do children perceive features of real and cartoon faces in the same way? Evidence from typical development and autism.” Journal of Clinical and Experimental Neuropsychology 32 (2): 212–218. CrossrefGoogle Scholar
Tasić, Miloš, and Dušan Stamenković
2018 “Exploring pictorial runes in Luca Enoch’s comic book series Gea .” Facta Universitatis, Series: Linguistics and Literature 15 (2): 123–141.Google Scholar
Willats, John.
1997 Art and representation: New principles in the analysis of pictures. Princeton: Princeton University Press.Google Scholar
Wilson, Brent.
1988 “The Artistic Tower of Babel: Inextricable Links Between Culture and Graphic Development.” In Discerning Art: Concepts and Issues, edited by George W. Hardiman and Theodore Zernich, 488–506. Champaign, IL: Stipes Publishing Company.Google Scholar
Wilson, Brent, and Marjorie Wilson
1977 “An Iconoclastic View of the Imagery Sources in the Drawings of Young People.” Art Education 30 (1): 4–12. CrossrefGoogle Scholar
Yue, Xiaomin, Brittany S. Cassidy, Kathryn J. Devaney, Daphne J. Holt, and Roger B. H. Tootell
2011 “Lower-Level Stimulus Features Strongly Influence Responses in the Fusiform Face Area.” Cerebral Cortex 21 (1): 35–47. CrossrefGoogle Scholar
Cited by

Cited by 1 other publications

Cohn, Neil & Tom Foulsham
2022. Meaning above (and in) the head: Combinatorial visual morphology from comics and emoji. Memory & Cognition 50:7 ► pp. 1381 ff. Crossref logo

This list is based on CrossRef data as of 06 january 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.