Multimodality, conventionality and inheritance in dialogic
constructions
The chapter investigates the multimodal correlates of a family of dialogic challenging
constructions in Modern Greek, anchored to the form ela (2nd person singular imperative of the verb
erxome ‘come’). First, prosodic features of these constructions are analyzed in terms of
intonation measurements in a production experiment, then gestural features are tested in terms of conventionality in a
recognition task. Gesture salience (defined independently of linguistic input) and the
functional space of gestures are discussed as factors that may explain successful and
unsuccessful results in matching gestures and constructions. Preliminary results suggest that while prosodic features
can be integrated in constructional (grammatical) descriptions and into inheritance relations, the gestural correlates
cannot as straightforwardly. Recognizing gesture constructions, optionally coindexed with grammatical
ones, may be one way of resolving the issue.
Article outline
- 1.Introduction
- 2.The constructions
- 2.1The simple ela challenging construction
- 2.2The ela tora construction
- 2.3The ela more construction
- 2.4The ela (mu) de construction
- 3.Intonation features and inheritance networks
- 3.1Methodology
- 3.2Results
- 3.3Integrating intonation features in constructional networks
- 4.Non-verbal correlates of the ela challenging constructions
- 4.1Data, methodology and results
- 4.2Successful matching and the role of salience
- 4.3Unsuccessful matching and the functional space of gestures
- 4.4Inheritance relations and gesture constructions
- 5.Summary and conclusions
-
Notes
-
References
References (56)
References
Ariel, M. (2008). Pragmatics
and grammar. Cambridge: Cambridge University Press. 

Babiniotis, G. (2002). Λεξικό της Νέας Ελληνικής γλώσσας [Dictionary of the
Modern Greek
language]. Athens: Lexicology center.
Baltazani, M., Gryllia, S., & Arvaniti, A. (2020). The
intonation and pragmatics of Greek wh-questions. Language and
Speech, 63(1), 56–94. 

Boersma, P., & Weenink, D. (2003). Praat:
Doing phonetics by computer, version 5.3.42. [URL]
Bressem, J., & Müller, C. (2017). The
Negative-Assessment-Construction. A multimodal pattern based on a recurrent
gesture? Linguistics
Vanguard, 3(s1). 

Bybee, J. (2010). Language,
usage and
cognition. Cambridge: Cambridge University Press. 

Chu, M., Meyer, A., Foulkes, L., & Kita, S. (2014). lndividual
differences in frequency and saliency of speech-accompanying gestures: The role of cognitive abilities and
empathy. Journal of Experimental
Psychology, 143(2), 694–709. 

Cienki, A. (1998). Metaphoric
gestures and some of their relations to verbal metaphoric
expressions. In J.-P. Koenig (Ed.), Discourse
and
cognition (pp. 189–204). Stanford CA: CSLI Publications.
Cienki, A. (2016). Cognitive
Linguistics, gesture studies, and multimodal communication. Cognitive
Linguistics, 27(4), 603–618. 

Clark, H. (1973). Space,
time, semantics, and the child. In T. E. Moore (Ed.), Cognitive
development and the acquisition of
language (pp. 27–63). New York: Academic Press. 

Cohen, Ph., Coulston, R., & Krout, K. (2002). Multimodal
interaction during multiparty dialogues: Initial
results. In IEEE Conference on Multimodal
Interfaces. 

Croft, W. (2001). Radical
Construction Grammar. Syntactic theory in typological
perspective. Oxford: Oxford University Press. 

Croft, W., & Poole, K. T. (2008). Inferring
universals from grammatical variation: multidimensional scaling for typological
analysis. Theoretical
Linguistics, 34(1), 1–37. 

Debras, C., & Cienki, A. (2012). Some
uses of head tilts and shoulder shrugs during human interaction, and their relation to
stancetaking. Paper presented at
the International Conference on Social
Computing. 
Diessel, H. (2013). Construction
Grammar and first language acquisition. In T. Hoffmann, & G. Trousdale (Eds.), The
Oxford Handbook of Construction
Grammar. Oxford: Oxford University Press, (online publication). 

Eibl-Eibesfeldt, I. (2007[1988]). Human
ethology. London: Routledge.
Eisenstein, J., Barzilay, R., & Davis, R. (2007). Turning
lectures into comic books using linguistically salient
gestures. In Proceedings of the Twenty-Second AAAI
Conference on Artificial Intelligence, Vancouver, British Columbia, Canada.
Eisenstein, J., & Christoudias, M. C. (2004). A
Salience-based approach to gesture-speech
alignment. In Proceedings of the Human Language
Technology Conference of the North American Chapter of the Association for Computational Linguistics:
HLT-NAACL
2004 (pp. 25–32). Boston, Massachusetts, USA. Association for Computational Linguistics.
Ellis, N. (2006a). Language
acquisition as rational contingency learning. Applied
Linguistics, 27(1), 1–24. 

Ellis, N. (2006b). Selective
attention and transfer phenomena in SLA: Contingency, cue competition, salience, interference, overshadowing,
blocking and perceptual learning. Applied
Linguistics, 27(2), 1–31. 

Ellis, N. (2013). Construction
Grammar and second language acquisition. In T. Hoffmann, & G. Trousdale (Eds.), The
Oxford Handbook of Construction
Grammar. Oxford: Oxford University Press, (online publication). 

Ellis, N., & Sagarra, N. (2011). Learned
attention in adult language acquisition: A replication and generalization study and
meta-analysis. Studies in Second Language
Acquisition, 33(4), 589–624. 

Fillmore, C. (1981). Pragmatics
and the description of discourse. In P. Cole (Ed.), Radical
pragmatics (pp. 143–166). New York: Academic Press.
Goldberg, A. E. (1995). Constructions:
A Construction Grammar approach to argument
structure. Chicago: University of Chicago Press.
Hinnell, J., & Rice, S. (2019). The
embodied marking of stance in North American English discourse: Stacked and
idiomatic. Presented at the 15th International
Cognitive Linguistics Conference, Kwansei Gakuin University, Nishinomiya,
Japan.
Hoffmann, T. (2017). Multimodal
constructs — multimodal constructions? The role of constructions in the working
memory. Linguistics
Vanguard, 3(s1).
Jehoul, A., Brône, G., & Feyaerts, K. (2017). The
shrug as marker of obviousness. Corpus evidence from Dutch face-to-face
conversations. Linguistics
Vanguard, 3(s1). 

Kendon, A. (2004). Gesture:
Visible action as
utterance. Cambridge: Cambridge University Press. 

Kress, G., & van Leeuwen, T. (2006[1996]). Reading
images: The grammar of visual
design. London: Routledge. 

Ladewig, S. (2014). Recurrent
gestures. In C. Müller, A. Cienki, E. Fricke, S. Ladewig, D. McNeill, & J. Bressem (Eds.), Body-language-communication:
An international handbook on multimodality in human interaction. Handbücher zur
Sprach — und Kommunikationswissenschaft / Handbooks of Linguistics and Communication Science (HSK)Edition:
38.2, Chapter: 118. Berlin & Boston: De Gruyter Mouton. 

Mackridge, P. (1985). The
Modern Greek language: A descriptive analysis of standard Modern
Greek. Oxford: Oxford University Press.
Marcos-Ramiro, A., Pizarro-Perez, D., Marron-Romera, M., Nguyen, L. S., & Gatica-Perez, D. (2013). Body
communicative cue extraction for conversational analysis. IEEE
International Conference on Face and Gesture Recognition. 
McNeill, D. (1992). Hand
and mind. Chicago: University of Chicago Press. 

McNeill, D. (2005). Gesture
and thought. Chicago: University of Chicago Press. 

Michaelis, L., & Lambrecht, K. (1996). Toward
a construction-based theory of language function: The case of nominal
extraposition. Language, 72, 215–247. 

Müller, C. (2004). Forms
and uses of the Palm Up Open Hand: A case of a gesture
family? In C. Müller, & R. Posner (Eds.), The
semantics and pragmatics of everyday
gesture (pp. 233–256). Berlin: Weidler Verlag.
Müller, C., Bressem, J., & Ladewig, S. H. (2013). Towards
a grammar of gesture: A form-based view. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill, & S. Teßendorf (Eds.), Body — language — communication.
An international handbook on multimodality in human interaction (Handbooks of
linguistics and communication science
38.1.) (pp. 707–733). Berlin & Boston: De Gruyter Mouton. 

Nikiforidou, K., Marmaridou, S., Mikros, G. K. (2014). What’s
in a dialogic construction? A constructional approach to polysemy and the grammar of
challenge. Cognitive
Linguistics, 25(4), 655–699. 

Oviatt, S. L., DeAngeli, A., & Kuhn, K. (1997). Integration
and synchronization of input modes during multimodal human-computer
interaction. In Human factors in computing systems
(CHI’97) (pp. 415–422). ACM Press.
Schoonjans, S. (2017). Multimodal
Construction Grammar issues are Construction Grammar issues. Linguistics
Vanguard, 3(s1). 

Sommerer, L. & Van de Velde, F. (in
press). Constructional
networks. In M. Fried, & K. Nikiforidou (Eds.), The
Cambridge Handbook of Construction
Grammar. Cambridge: Cambridge University Press.
Steen, F., & Turner, M. (2013). Multimodal
construction grammar. In M. Borkent, B. Dancygier, & J. Hinnell (Eds.), Language
and the creative
mind (pp. 255–274). Stanford, CA: CSLI Publications. 

Sweetser, E. (2009). What
does it mean to compare language and gesture? Modalities and
contrasts. In J. Guo, E. Lieven, & N. Budwig (Eds.), Crosslinguistic
approaches to the psychology of language: Studies in the tradition of Dan Isaac
Slobin (pp. 357–366). New York: Psychology Press. 

Tabacaru, S., & Lemmens, M. (2014). Raised
eyebrows as gestural triggers in humour: The case of sarcasm and
hyper-understanding. European Journal of Humour
Research, 2(22),11–31. 

Tzartzanos, A. (1991[1963]). Νεοελληνική Σύνταξις (της Κοινής Δημοτικής) [Syntax
of Modern
Greek], Volume Β’. Athens: Οργανισμός Εκδόσεως Διδακτικών Βιβλίων.
Ward, N. G. (2019). Prosodic
patterns in English
conversation. Cambridge: Cambridge University Press. 

Waring, H. (1976). The
intonation of Modern Greek. Ph.D.
Dissertation. University of London.
Wells, J. C. (2006). English
intonation. An
introduction. Cambridge: Cambridge University Press.
Yin, Y., & Davis, R. (2013). Gesture
spotting and recognition using salience detection and concatenated hidden markov
models. In Proceedings of the 15th ACM on multimodal
interaction (ICMI
’13) (pp. 489–494). ACM: New York, USA. 

Ziem, A. (2017). Do
we really need a Multimodal Construction Grammar? Linguistics
Vanguard, 3(s1). 

Zima, E. (2014). English
multimodal motion constructions. A Construction Grammar perspective. Studies Van De Bkl — Travaux Du
Cbl — Papers Of The Lsb2014B 8. [URL]
Zima, E. (2017). On
the multimodality of [all the way from X PREP Y]. Linguistics
Vanguard, 3(s1). 

Zima, E., & Bergs, A. (2017). Multimodality
and construction grammar. Linguistics
Vanguard, 3(s1). 
