Chapter 7
Relevance, style and multimodality
Typographical features as stylistic devices
Brightly coloured textual inserts, which often occupy a sizable part of the TV screen, have become a key
feature in Japanese TV. This paper unpacks the contribution of such multimodal stimuli to inference, and the
consequences this has for the interpretation process. Using data derived from a mixed-methods approach (i.e.
eye-tracking and a multimodal content analysis), we evaluate the relationship between cognitive processing and
communicative stimuli. We demonstrate how typographical features (colours and fonts) are used as highlighting
stylistic devices by TV producers to manipulate the viewer comprehension process by guiding the audience to an
intended interpretation. The results suggest how editorial choices regarding typographical features to trigger certain
effects might be subsumed under the current view of style in relevance theory.
Article outline
- 1.Introduction
- 1.1Background: Telops on Japanese TV
- 2.Typographical features, stylistics and multimodal analysis
- 2.1Suitability of typographical features
- 2.2Social semiotic approach to typography: Multimodal analysis of fonts
- 2.3Issues with the multimodal approach
- 3.Relevance theory and pragmatic effects
- 4.Case study: Theoretical explanation and reception study of viewer behaviour
- 4.1Research question and research design
- 4.2Telops and typographical features
- 4.3Typographical features and interpretation – telops and relevance theory
- 4.3.1“Affective” interpretation and typographical features
- 4.3.2Propositional attitudes and typographical features
- 5.An empirical approach to telops
- 5.1Methodology: Eye-tracking
- 5.2Results: Eye-tracking data
- 6.Conclusion
-
Acknowledgements
-
Notes
-
References
-
Sources of visual material
References
Blakemore, D.
(
2008)
Apposition
and affective communication.
Language and
Literature, 17 (1), 37–57.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bornstein, R. F., & Becker-Matero, N.
(
2012)
Metaphor
and psychodynamic research. In
K. S. Montana (Ed.),
Metaphor
and fields: Common ground, common language, and the future of
psychoanalysis (pp. 92–103). London: Routledge
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Butterick, M.
(
2008)
Font
recommendations. Available at:
[URL]
Clark, B.
(
2013)
Relevance
theory. Cambridge: Cambridge University Press.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Color
Matters
(
2012)
The Meanings of
yellow.
[URL]
Díaz-Cintas J., & Remael A.
(
2007)
Audiovisual
translation: Subtitling. Manchester: St. Jerome.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Doyle, J. R., & Bottomley, P. A.
(
2009)
The
massage in the medium: Transfer of connotative meaning from typeface to names and
products.
Applied Cognitive
Psychology 23(3), 396–409.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Green, G.
(
1980)
Some
wherefores of English
inversions.
Language, 56, 582–601.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Halliday, M. A. K., & Hasan, R.
(
1976)
Cohesion
in
English. London: Longman.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Halliday, M. A. K .
(
2007)
Language and
education. London: Continuum.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Henderson, P. W., Giese, J. L., & Cote, J. A.
(
2004)
Impression
management using typeface design.
Journal of
Marketing, 68(4), 60–72.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Just, M. A., & Carpenter, P. A.
(
1993)
The
intensity dimension of thought: Pupillometric indices of sentence
processing.
Canadian Journal of Experimental
Psychology, 47 (2), 310–339.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kimura, T., Hosoi, A., Honda, N., Kato, Y., Kawamura, F., Koizumi, A., Oosawa, Y., Suzuki, E. and Watabe, K.
(
2000) “
テレビ画面に踊る文字たちの生態学” [Physiology of Letters Dancing on TV
screen],
Galac, 366.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lakens, D., Semin Gün R., & Foroni, F.
(
2012)
But
for the bad, there would not be good: Grounding valence in brightness through shared relational
structures.
Journal of Experimental Psychology:
General, 141(3), 584–594.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Meyer, Ch. F.
1992 Apposition in contemporary English. Cambridge: Cambridge University Press.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Meier, B. P., Robinson, M. D., & Caven, A. J.
(
2008)
Why
a Big Mac is a good Mac: Associations between affect and size.
Basic and
Applied Social
Psychology, 30, 46–5.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Meier, B. P., Robinson, M. D., Crawford, L. E., & Ahlvers, W. J.
(
2007)
When
“light” and “dark” thoughts become light and dark responses: Affect biases brightness
judgments.
Emotion 7, 366–376.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Nørgaard, N.
(
2009)
The
semiotics of typography in literary texts. A multimodal approach.
Orbis
Litterarum, 64(2), 141–160.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
O’Brien, S.
(
2009)
Eye
tracking in translation process research: Methodological challenges and
solutions. In
I. Mees,
F. Alves, &
S. Gopferich (Eds.),
Copenhagen
studies in language: Methodology, technology and innovation in translation process
research (pp. 251–266). Copenhagen: Samfundsitteratur.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
O’Hagan, M.
(
2010)
Japanese
TV entertainment: Framing humour with open caption
telop. In
D. Chiaro (Ed.),
Translation,
humour and the media: Translation and
humour, Volume 2 (pp. 70–88). London: Continuum.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Reinhard, T.
(
1981)
Pragmatics
and linguistics: An analysis of sentence
topics.
Philosophica, 27, 53–94.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Sasamoto, R.
(
2014)
Impact
caption as a highlighting device: Attempts at viewer manipulation on
TV.
Discourse, Context and
Media, 6, 1–10.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Sasamoto, R., & Doherty, S.
(
2013)
Manipulating
the mind of others – the use of subtitles as viewer manipulation.
Cognitive
Futures of the
Humanities, Bangor, UK
04–
APR–13–
06–
APR–13.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Schultheis, H., & Jameson, A.
(
2004)
Assessing
cognitive load in adaptive hypermedia systems: Physiological and behavioural
methods. In
W. Nejdl, &
P. de Bra (Eds.),
Adaptive
hypermedia and adaptive web-based
systems (pp. 18–24). Eindhoven: Springer Verlag.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Shiota, E.
(
2003) “
関連性理論とテロップの理解” [Relevance Theory and Understanding of Telop].
Journal of Ryukoku
University, 63–91.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Sperber, D & Wilson, D.
(
1995)
Relevance:
Communication and cognition. 2nd
ed. Oxford: Blackwell.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Stökl, H.
(
2005)
Typography:
Body and dress of a text – a signing mode between language and image.
Visual
Communication, 4 (2), 204–214.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Takahashi T., & Katayama N.
(
2012) ‘
伝わるデザイン:研究発表のユニバーサルデザイン[Design to Communicate: Universal Design for Research
Presentation]’ [Online]. Available
at
[URL]
van Leeuwen, T.
(
2005)
Typographic
meaning.
Visual
Communication, 4( 2), 137–142.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wharton, T.
(
2009)
Pragmatics
and non-verbal
communication. Cambridge: Cambridge University Press.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilson, D.
(
2000)
Metarepresentation
in linguistic communication. In
D. Sperber (Ed.),
Metarepresentations:
A multidisplinary
perspective (pp. 411–48). Oxford: Oxford University Press.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilson, D., & Sperber, D.
(
1993)
Linguistic
form and
relevance.
Lingua, 90(1), 1–25.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilson, D., & Sperber, D.
(
2012)
Meaning
and relevance. Cambridge: Cambridge University Press.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wilson, D. & Wharton, T.
(
2006)
Relevance
and prosody.
Journal of
Pragmatics, 38, 1559–1579.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Yus, F.
(
2009)
Visual
metaphor versus verbal metaphor: A unified
account. In
Ch. Forceville, &
E. Uriós-Aparisi (Eds.),
Multimodal
Metaphor (pp. 145–172). Berlin: Mouton de Gruyter.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Sources of visual material
Himitsu no
Kenmin Show 2007–2015 Nihon
Television Network. Produced by
Shinichi Nose,
Minoru Sato, and
Kie Shimizu. Tokyo.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Honmadekka!?
2009–2015 Fuji Television
Network. Produced by
Michiko Kametaka and
Masayuki Harashima. Tokyo.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by
Cited by 3 other publications
Sasamoto, Ryoko, Stephen Doherty & Minako O’Hagan
Scott, Kate
2023.
Nutritional labeling, communication design, and relevance.
Frontiers in Communication 8
![DOI logo](//benjamins.com/logos/doi-logo.svg)
Scott, Kate & Rebecca Jackson
This list is based on CrossRef data as of 28 june 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.