Article published In:
The Mental Lexicon: Online-First ArticlesEye-voice and finger-voice spans in adults’ oral reading of connected texts [*] *
Implications for reading research and assessment
Alessandro Lento | Institute for Computational Linguistics “A. Zampolli” | Biomedical Campus University
The present paper investigates the interaction between eye movements, voice articulation and the movements of the index finger dynamically pointing to a text line in oral finger-point reading of Italian. During finger-point reading, the finger appears to be ahead of the voice most of the times, by a margin that is significantly modulated by the distribution of phrasal and prosodic units in the reading text. Eye movements replicate the same effects on a different time scale. The eye is ahead of both voice and finger by a wide margin (confirming evidence observed for English and German sentence reading), while showing a tendency to re-synchronise with voice articulation at the right edge of strong prosodic units (sentence boundaries). Our evidence suggests a multicomponent view of the time span between the eye/finger and the voice. The span is shown to be the dynamic outcome of an optimally adaptive reading strategy, resulting from the interaction between individual decoding skills, the reader’s phonological buffer capacity, and the structural complexity of a reading text. Proficient readers modulate their span to compensate for the different timing between word fixation and word articulation, read faster, and dynamically adjust their processing window to the meaningful, prosodic units of a text.
Keywords: finger-point reading, eye-tracking, finger-tracking, eye-voice span, finger-voice span, eye-finger coordination, parallel processing, working memory, phonological buffer, adaptive reading
Article outline
- 1.Introduction
- 1.1The eye-voice span
- 2.Materials and methods
- Participants
- Procedure
- Text materials
- Apparatus
- Eye-tracking
- Finger-tracking
- Speech recording
- Data pre-processing and cleaning
- Eye-tracking
- Finger-tracking
- Speech processing
- Eye-voice span (EVS)
- Finger-voice span (FVS)
- Modelling issues
- 3.Results
- 3.1The dynamic of EVS and FVS
- 3.2Onset voice span and eye/finger movements
- 3.3Modelling the offset voice span
- 4.Discussion
- The phonological buffer hypothesis
- The adaptive reading hypothesis
- Integrating the two hypotheses
- 5.Concluding remarks
- Acknowledgements
- Notes
-
References
References
Abney, S. P.
(1991) Parsing
by chunks. In R. C. Berwick (Ed.), Principle-based
parsing (pp. 257–278). Springer. 
Attardi, G.
(2006) Experiments
with a multilanguage non-projective dependency parser. In The 10th
International Conference on Computational Natural Language Learning
(CoNLL-X2006) (pp. 166–170). New York.
Baddeley, A.
(2007) Working
memory, thought, and action (Vol. 451). OuP Oxford.
Bates, D., Maechler, M., Bolker, B., Walker, S., Christensen, R. H. B., Singmann, H.
, … others (2009) Package
‘lme4’. r-pro ject.org.
Brainard, D. H.
(1997) The
psychophysics toolbox. Spatial
vision,
10
(4), 433–436.
Breen, M.
(2014) Empirical
investigations of the role of implicit prosody in sentence processing. Language and Linguistics
Compass,
8
(2), 37–50.
Breen, M., Kaswer, L., Van Dyke, J. A., Krivokapić, J., & Landi, N.
(2016) Imitated
prosodic fluency predicts reading comprehension ability in good and poor high school
readers. Frontiers in
psychology,
7
1, 1026.
Brysbaert, M., & Stevens, M.
(2018) Power
analysis and effect size in mixed effects models: A tutorial. Journal of
cognition,
1
(1).
Buswell, G. T.
(1920) An
experimental study of the eye-voice span in reading. Journal of Educational
Psychology,
4
(12), 217–227.
(1921) The
relationship between eye-perception and voice-response in reading. Journal of Educational
Psychology,
12
(4), 217–227.
Carr, J. W., Pescuma, V. N., Furlan, M., Ktori, M., & Crepaldi, D.
(2021) Algorithms
for the automated correction of vertical drift in eye-tracking data. Behavior Research
Methods, 1–24.
Cohen, J. D., Servan-Schreiber, D., & McClelland, J. L.
(1992) A
parallel distributed processing approach to automaticity. The American journal of
psychology, 239–269.
Crepaldi, D., Ferro, M., Marzi, C., Nadalini, A., Pirrelli, V., & Taxitari, L.
(2022) Finger
movements and eye movements during adults’ silent and oral
reading. In R. Levie, A. Bar-On, O. Ashkenazi, E. Dattner, & G. Brandes (Eds.), Developing
Language and Literacy: Studies in Honor of Dorit Diskin
Ravid (pp. 443–471). Springer International Publishing.
Dell’Orletta, F., Federico, M., Lenci, A., Montemagni, S., & Pirrelli, V.
(2007) Maximum
Entropy for Italian PoS Tagging. Intelligenza
Artificiale,
4
(2).
Dell’Orletta, F., Montemagni, S., & Venturi, G.
(2011) READ-IT:
Assessing readability of Italian texts with a view to text
simplification. In Proceedings of the second workshop on Speech and
Language Processing for Assistive
Technologies (pp. 73–83).
De Luca, M., Pontillo, M., Primativo, S., Spinelli, D., & Zoccolotti, P.
(2013) The
eye-voice lead during oral reading in developmental dyslexia. Frontiers in human
neuroscience,
7
1, 696.
Drieghe, D., Rayner, K., & Pollatsek, A.
(2008) Mislocated
fixations can account for parafoveal-on-foveal effects in eye movements during
reading. Quarterly Journal of Experimental
Psychology,
61
(8), 1239–1249.
Federici, S., Montemagni, S., & Pirrelli, V.
(1996) Shallow
Parsing and Text Chunking: a View on Underspecification in Syntax. Proceedings of ESSLLI’96
Workshop on Robust Parsing, 35–44.
Ferro, M., Cappa, C., Giulivi, S., Marzi, C., Nahli, O., Cardillo, F. A., & Pirrelli, V.
(2018) Readlet:
Reading for understanding. In Proceedings of 5th IEEE Congress on
Information Science & Technology (IEEE CiST’18). Marrakech, Morocco.
Fuchs, L. S., Fuchs, D., Hosp, M. K., & Jenkins, J. R.
(2001) Oral
reading fluency as an indicator of reading competence: A theoretical, empirical, and historical
analysis. Scientific studies of
reading,
5
(3), 239–256.
Gathercole, S. E.
(2006) Nonword
repetition and word learning: The nature of the relationship. Applied
psycholinguistics, 27 (4), 513–543.
Grabe, W., & Stoller, F. L.
(2019) Teaching
and researching reading (3rd
ed.). Routledge.
Hughes, B., McClelland, A., & Henare, D.
(2014) On
the nonsmooth, nonconstant velocity of braille reading and reversals. Scientific Studies of
Reading, 18 (2), 94–113.
Huovinen, E., Ylitalo, A.-K., & Puurtinen, M.
(2018) Early
attraction in temporally controlled sight reading of music. Journal of Eye Movement
Research,
11
(2), 1–30.
Hyönä, J., & Bertram, R.
(2004) Do
frequency characteristics of nonfixated words influence the processing of fixated words during
reading? European Journal of Cognitive
Psychology, 16 (1–2), 104–127.
Inhoff, A. W., Solomon, M., Radach, R., & Seymour, B. A.
(2011) Temporal
dynamics of the eyevoice span and eye movement control during oral reading. Journal of
Cognitive
Psychology,
23
(5), 543–558.
Inhoff, A. W., Starr, M., & Shindler, K. L.
(2000) Is
the processing of words during eye fixations in reading strictly serial? Perception &
Psychophysics,
62
(7), 1474–1484.
Kentner, G., & Vasishth, S.
(2016) Prosodic
focus marking in silent reading: effects of discourse context and rhythm. Frontiers in
Psychology,
7
1, 319.
Kliegl, R., Nuthmann, A., & Engbert, R.
(2006) Tracking
the mind during reading: The influence of past, present, and future words on fixation
durations. Journal of experimental psychology:
General,
135
(1), 12.
Kumle, L., Võ, M. L.-H., & Draschkow, D.
(2021) Estimating
power in (generalized) linear mixed models: An open introduction and tutorial in r. Behavior
research
methods,
53
(6), 2528–2543.
Laubrock, J., & Kliegl, R.
(2015) The
eye-voice span during reading aloud. Frontiers in
psychology,
6
1, 1432.
Lawson, E. A.
(1961) A
note on the influence of different orders of approximation to the english language upon eye-voice
span. Quarterly Journal of Experimental
Psychology,
13
(1), 53–55.
Lenci, A., Montemagni, S., & Pirrelli, V.
(2003) “Chunk-it”.
An Italian Shallow Parser for Robust Syntactic Annotation. Linguistica
Computazionale,
XVIII–XIX
1, 353–386.
Levin, H., & Cohn, J. A.
(1968) Effects
of instruction on the eye-voice span. In H. Levin, E. J. Gibson, & J. J. Gibson (Eds.), The
analysis of reading skills: A program of basic and applied research. final
report (pp. 254–283). Ithaca, New York: Cornell University.
Levin, H., & Turner, E. A.
(1968) Sentence
structure and the eye-voice span. In H. Levin, E. J. Gibson, & J. J. Gibson (Eds.), The
analysis of reading skills: A program of basic and applied research. final
report (pp. 196–220). Ithaca, New York: Cornell University.
Lim, Y., Park, J. M., Rhyu, S.-Y., Chung, C. K., Kim, Y., & Yi, S. W.
(2019) Eye-hand
span is not an indicator of but a strategy for proficient sight-reading in piano
performance. Scientific
Reports,
9
(1), 1–11.
Lio, G., Fadda, R., Doneddu, G., Duhamel, J.-R., & Sirigu, A.
(2019) Digit-tracking
as a new tactile interface for visual perception analysis. Nature
Communications, 10 (5392), 1–13.
Lyding, V., Stemle, E., Borghetti, C., Brunello, M., Castagnoli, S., Dell’ Orletta, F., … Pirrelli, V.
(2014) The
Paisa corpus of Italian web texts. In F. Bildhauer & R. Schäfer (Eds.), (p. 36–43). Gothenburg, Sweden: Association for Computational Linguistics.
Maffei, L.
(2018) Elogio
della parola. Bologna: il Mulino.
Marzi, C., Rodella, A., Nadalini, A., Taxitari, L., & Pirrelli, V.
(2020) Does
finger-tracking point to child reading strategies? In J. Monti, F. Dell’Orletta, & F. Tamburini (Eds.), Proceedings
of 7th Italian Conference on Computational
Linguistics (Vol. 27691). Bologna.
Mesmer, H. A. E., & Lake, K.
(2010) The
role of syllable awareness and syllable-controlled text in the development of finger-point
reading. Reading
Psychology,
31
(2), 176–201.
Mesmer, H. A. E., & Williams, T. O.
(2015) Examining
the role of syllable awareness in a model of concept of word: Findings from
preschoolers. Reading Research
Quarterly,
50
(4), 483–497.
Moors, A.
(2016) Automaticity:
Componential, causal, and mechanistic explanations. Annual review of
psychology,
67
1, 263–287.
Morton, J.
(1964a) The
effects of context upon speed of reading, eye movements and eye-voice span. Quarterly Journal
of Experimental
Psychology,
16
(4), 340–354.
(1964b) A
model for continuous language behaviour. Language and
Speech,
7
(1), 40–70.
Nonaka, T., Ito, K., & Stoffregen, T. A.
(2021) Structure
of variability in scanning movement predicts braille reading performance in
children. Scientific
reports,
11
(1), 1–12.
Paap, K. R., & Noel, R. W.
(1991) Dual-route
models of print to sound: Still a good horse race. Psychological
research,
53
(1), 13–24.
Pate, J. K., & Goldwater, S.
(2011) Unsupervised
syntactic chunking with acoustic cues: computational models for prosodic
bootstrapping. In Proceedings of the 2nd Workshop on Cognitive
Modeling and Computational
Linguistics (pp. 20–29).
Pollatsek, A., & Treiman, R.
(2015) The
oxford handbook of reading. Oxford University Press.
Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., … Vesely, K.
(2011) The
Kaldi Speech Recognition Toolkit. In IEEE 2011 Workshop on Automatic
Speech Recognition and Understanding. IEEE Signal Processing Society. (IEEE Catalog No.: CFP11SRW-USB)
Protopapas, A., Altani, A., & Georgiou, G. K.
(2013) Development
of serial processing in reading and rapid naming. Journal of Experimental Child
Psychology,
116
(4), 914–929.
R Core Team
(2023) R: A language and
environment for statistical computing. [URL]. Vienna, Austria: R Foundation for Statistical Computing.
Reichle, E. D.
(2021) Computational
models of reading: A handbook. Oxford University Press.
Sakoe, H., & Chiba, S.
(1978) Dynamic
programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics,
Speech, and Signal
Processing,
26
(1), 43–49.
Schilling, S. G., Carlisle, J. F., Scott, S. E., & Zeng, J.
(2007) Are
fluency measures accurate predictors of reading achievement? The Elementary School
Journal,
107
(5), 429–448.
Shmyrev, N. V., & Vosk Core Team
(2020) Vosk
Speech Recognition Toolkit: Offline speech recognition API for Android, iOS, Raspberry Pi and servers with Python, Java, C#
and Node. [URL]. GitHub repository.
Silva, S., Reis, A., Casaca, L., Petersson, K. M., & Faísca, L.
(2016) When
the Eyes no longer lead: Familiarity and Length Effects on Eye-Voice Span. Frontiers in
Psychology,
7
1.
Taxitari, L., Cappa, C., Ferro, M., Marzi, C., Nadalini, A., & Pirrelli, V.
(2021) Using
mobile technology for reading assessment. In Proceedings of 6th IEEE
Congress on Information Science & Technology (IEEE CiST’20). Agadir, Morocco.
Uhry, J. K.
(1999) Invented
spelling in kindergarten: The relationship with finger-point reading. Reading and
Writing,
11
1, 441–464.
(2002) Finger-point
reading in kindergarten: The role of phonemic awareness, one-to-one correspondence, and rapid serial
naming. Scientific Studies of
Reading,
6
(4), 319–342.
Vihman, M. M.
(2022) The
developmental origins of phonological memory. Psychological
review,
129
(6), 1495–1508.