Implicit learning of non-adjacent dependencies
A graded, associative account
Language and other higher-cognitive functions require structured sequential behavior including non-adjacent relations. A fundamental question in cognitive science is what computational machinery can support both the learning and representation of such non-adjacencies, and what properties of the input facilitate such processes. Learning experiments using miniature languages with adult and infants have demonstrated the impact of high variability (Gómez, 2003) as well as nil variability (Onnis, Christiansen, Chater, & Gómez (2003; submitted) of intermediate elements on the learning of nonadjacent dependencies. Intriguingly, current associative measures cannot explain this U shape curve. In this chapter, extensive computer simulations using five different connectionist architectures reveal that Simple Recurrent Networks (SRN) best capture the behavioral data, by superimposing local and distant information over their internal ‘mental’ states. These results provide the first mechanistic account of implicit associative learning of non-adjacent dependencies modulated by distributional properties of the input. We conclude that implicit statistical learning might be more powerful than previously anticipated.
References
Allen, J., & Seidenberg, M.S
(
1999)
The emergence of grammaticality in connectionist networks. In
B. MacWhinney (Ed.),
The emergence of language (pp. 115–151). Mahwah, NJ: Lawrence Erlbaum Associates.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Botvinick, M., & Plaut, D.C
(
2006)
Short-term memory for serial order: A recurrent neural network model.
Psychological Review, 113, 201–233.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Botvinick, M., & Plaut, D.C
(
2004)
Doing without schema hierarchies: A recurrent connectionist approach to normal and impaired routine sequential action.
Psychological Review, 111, 395–429.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Brakel, P., & Frank, S.L
(
2009)
Strong systematicity in sentence processing by simple recurrent networks. In
N.A. Taatgen,
H. van Rijn,
J. Nerbonne &
L. Schomaker (Eds.),
Proceedings of the 31st Annual Conference of the Cognitive Science Society (pp. 1599–1604). Austin, TX: Cognitive Science Society.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Chater, N., & Conkey, P
(
1992)
Finding linguistic structure with recurrent neural networks. In
Proceedings of the 14th Annual Conference of the Cognitive Science Society
(pp. 402–407). Hillsdale, New Jersey: Psychology Press.
Chomsky, N
(
1959)
A review of BF Skinner's Verbal Behavior.
Language, 35(1), 26–58.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Christiansen, M.H., Allen, J., & Seidenberg, M.S
(
1998)
Learning to segment speech using multiple cues: A connectionist model.
Language and Cognitive Processes, 13, 221–268.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Christiansen, M.H., & Chater, N
(
1999)
Toward a connectionist model of recursion in human linguistic performance.
Cognitive Science, 23, 157–205.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Christiansen, M.H., Conway, C.M., & Curtin, S
(
2000)
A connectionist single-mechanism account of rule-like behavior in infancy. In
L.R. Gleitman &
A.K. Joshi (Eds.),
The Proceedings of the 22nd Annual Conference of the Cognitive Science Society (pp. 83–88). Philadelphia, PA: University of Pennsylvania.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Christiansen, M.H., & MacDonald, M.C
(
2009)
A usage-based approach to recursion in sentence processing.
Language Learning, 59(Suppl. 1), 126–161.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cleeremans, A., Servan-Schreiber, D., & McClelland, J.L
(
1989)
Finite state automata and simple recurrent networks.
Neural Computation, 1, 372–381.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Destrebecqz, A., & Cleeremans, A
Cottrell, G.W., & Plunkett, K
(
1995)
Acquiring the mapping from meanings to sounds.
Connection Science, 6, 379–412.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dell, G.S., Juliano, C., & Govindjee, A
(
1993)
Structure and content in language production: A theory of frame constraints in phonological speech errors.
Cognitive Science, 17, 149–195.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dienes, Z
(
1992)
Connectionist and memory-array models of artificial grammar learning.
Cognitive Science, 23, 53–82.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dulany, D.E., Carlson, R.A., & Dewey, G.I
(
1984)
A case of syntactical learning and judgement: How conscious and how abstract? Journal of Experimental Psychology: General, 113, 541–555.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Elman, J.L
(
1990)
Finding structure in time.
Cognitive science, 14(2), 179–211.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Elman, J.L
(
1991)
Distributed representations, simple recurrent networks, and grammatical structure.
Machine Learning, 7, 195–224.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Estes, K., Evans, J., Alibali, M., & Saffran, J
(
2007)
Can infants map meaning to newly segmented words? Psychological Science, 18(3), 254.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Farkaš, I., & Crocker, M.W
(
2008)
Syntactic systematicity in sentence processing with a recurrent self-organizing network.
Neurocomputing, 71(7), 1172–1179.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Frank, M.C., Goldwater, S., Griffiths, T.L., & Tenenbaum, J.B
(
2010)
Modeling human performance in statistical word segmentation.
Cognition, 117(2), 107–125.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Frank, S.L
in press).
Getting real about systematicity. In
P. Calvo &
J. Symons (Eds.)
Systematicity and cognitive architecture: Conceptual and empirical issues 25 years after Fodor & Pylyshyn's challenge to connectionism Cambridge, MA The MIT Press
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
Frinken, V., Fischer, A., Manmatha, R., & Bunke, H
(
2012)
A novel word spotting method based on recurrent neural networks.
IEEE Transactions on, Pattern Analysis and Machine Intelligence, 34(2), 211–224.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gaskell, M.G., Hare, M., & Marslen-Wilson, W.D
(
1995)
A connectionist model of phonological representation in speech perception.
Cognitive Science, 19, 407–439.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gibson, F.P., Fichman, M., & Plaut, D.C
(
1997)
Learning in dynamic decision tasks: Computational model and empirical evidence.
Organizational Behavior and Human Decision Processes, 71, 1–35.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Gómez, R
(
2002)
Variability and detection of invariant structure.
Psychological Science, 13, 431–436.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Harm, M.W., & Seidenberg, M.S
(
1999)
Phonology, reading acquisition, and dyslexia: Insights from connectionist models.
Psychological Review, 106, 491–528.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hinoshita, W., Arie, H., Tani, J., Okuno, H.G., & Ogata, T
(
2011)
Emergence of hierarchical structure mirroring linguistic composition in a recurrent neural network.
Neural Networks, 24(4), 311–320.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Johnstone, T. & Shanks, D.R
(
2001)
Abstractionist and processing accounts of implicit learning.
Cognitive Psychology, 42, 61–112.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Jordan, M.I
(
1986)
Attractor dynamics and parallelism in a connectionist sequential machine. In
Proceedings of the Eighth Annual Conference of the Cognitive Science Society
. Hillsdale, NJ: Lawrence Erlbaum Associates.
Kinder, A. & Shanks, D.R
(
2001)
Amnesia and the declarative/procedural distinction: A recurrent network model of classification, recognition, and repetition priming.
Journal of Cognitive Neuroscience, 13, 648–669.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kirov, C., & Frank, R
(
2012)
Processing of nested and cross-serial dependencies: An automaton perspective on SRN behaviour.
Connection Science, 24(1), 1–24.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lashley, K.S
(
1951)
The problem of serial order in behavior. In
L.A. Jeffress (Ed.),
Cerebral mechanisms in behavior (pp. 112–146). New York, NY: Wiley.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Luce, R.D
(
1963)
Detection and recognition. In
R.D. Luce,
R.R. bush, &
E. Galanter (Eds.),
Handbook of mathematical psychology. New York, NY: Wiley.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Maraqa, M., Al-Zboun, F., Dhyabat, M., & Zitar, R.A
(
2012)
Recognition of Arabic Sign Language (ArSL) using recurrent neural networks.
Journal of Intelligent Learning Systems and Applications, 4(1), 41–52.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Maskara, A., & Noetzel, A
(
1992)
Forced simple recurrent neural network and grammatical inference. In
Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society
(pp. 420–425). Hillsdale, NJ: Lawrence Erlbaum Associates.
Miikkulainen, R., & Mayberry III, M.R
(
1999)
Disambiguation and grammar as emergent soft constraints. In
B. MacWhinney (Ed.),
Emergence of language, 153–176. Mahwah, NJ: Lawrence Erlbaum Associates.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Misyak, J.B., & Christiansen, M.H
(
2012)
Statistical learning and language: An individual differences study.
Language Learning, 62, 302–331.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Misyak, J.B., Christiansen, M.H. & Tomblin, J.B
(
2010a)
On-line individual differences in statistical learning predict language processing.
Frontiers in Psychology, Sept.14.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Misyak, J.B., Christiansen, M.H. & Tomblin, J.B
(
2010b)
Sequential expectations: The role of prediction- based learning in language.
Topics in Cognitive Science, 2, 138–153.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Moss, H.E., Hare, M.L., Day, P., & Tyler, L.K
(
1994)
A distributed memory model of the associative boost in semantic priming.
Connection Science, 6, 413–427.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Munakata, Y., McClelland, J.L., & Siegler, R.S
(
1997)
Rethinking infant knowledge: Toward an adaptive process account of successes and failures in object permanence tasks.
Psychological Review, 104, 686–713.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Onnis, L., Christiansen, M.H., Chater, N., & Gómez, R
submitted).
Statistical learning of non-adjacent relations. Submitted manuscript.
Onnis, L., Christiansen, M.H., Chater, N., & Gómez, R
(
2003)
Reduction of uncertainty in human sequential learning: Preliminary evidence from Artificial Grammar Learning. In
R. Alterman &
D. Kirsh (Eds.),
Proceedings of the 25th Annual Conference of the Cognitive Science Society. Boston, MA: Cognitive Science Society.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Onnis, L., Monaghan, P., Christiansen, M.H., & Chater, N
(
2004)
Variability is the spice of learning, and a crucial ingredient for detecting and generalizing in nonadjacent dependencies. In
Proceedings of the 26th annual conference of the Cognitive Science Society
(pp. 1047–1052). Mahwah, NJ: Lawrence Erlbaum.
Pacton, S., Perruchet, P., Fayol, M., & Cleeremans, A
(
2001)
Implicit learning out of the lab: The case of orthographic regularities.
Journal of Experimental Psychology: General, 130, 401–426.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Perruchet, P., & Pacteau, C
(
1990)
Synthetic grammar learning: Implicit rule abstraction or explicit fragmentary knowledge? Journal of Experimental Psychology: General, 119, 264–275.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Perruchet, P., & Pacton, S
(
2006)
Implicit learning and statistical learning: One phenomenon, two approaches.
Trends In Cognitive Sciences, 10(5), 233–238.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Plaut, D.C., & Kello, C.T
(
1999)
The emergence of phonology from the interplay of speech comprehension and production: A distributed connectionist approach. In
B. MacWhinney (Ed.),
The emergence of language (pp. 381–415). Mahwah, NJ: Lawrence Erlbaum Associates.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Redington, M., & Chater, N
(
2002)
Knowledge representation and transfer in artificial grammar learning (AGL). In
R.M. French &
A. Cleeremans (Eds.),
Implicit learning and consciousness: An empirical, philosophical, and computational consensus in the making. Hove: Psychology Press.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Rohde, D.L.T., & Plaut, D.C
(
1999)
Language acquisition in the absence of explicit negative evidence: How important is starting small? Cognition, 72, 67–109.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Saffran, J.R., Aslin, R.N., & Newport, E.L
(
1996)
Statistical learning by 8-month-old infants.
Science, 274, 1926–1928.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Saffran, J
(
2001)
Words in a sea of sounds: The output of infant statistical learning.
Cognition, 81, 149–169.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Servan-Schreiber, D., Cleeremans, A. & McClelland, J.L
(
1991)
Graded state machines: The representation of temporal dependencies in simple recurrent networks.
Machine Learning, 7, 161–193.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Si, Y., Xu, J., Zhang, Z., Pan, J., & Yan, Y
(
2012)
An improved Mandarin voice input system using recurrent neural network language model. In
Computational Intelligence and Security (CIS),
Eighth International Conference on
IEEE (pp. 242–246).
Socher, R., Manning, C.D., & Ng, A.Y
(
2010)
Learning continuous phrase representations and syntactic parsing with recursive neural networks. In
Proceedings of the NIPS-2010 Deep Learning and Unsupervised Feature Learning Workshop
. Hilton: Cheakmus.
Sutskever, I., Martens, J., & Hinton, G
(
2011)
Generating text with recurrent neural networks. In
Proceedings of the 2011 International Conference on Machine Learning (ICML-2011).
Tabor, W
(
2011)
Recursion and recursion-like structure in ensembles of neural elements. In
H. Sayama,
A. Minai,
D. Braha, &
Y. Bar-Yam (Eds.),
Unifying themes in complex systems. Proceedings of the VIII International Conference on Complex Systems
(pp. 1494–1508). Berlin: Springer.
Takac, M., Benuskova, L, & Knott, A
(
2012)
Mapping sensorimotor sequences to word sequences: A connectionist model of language acquisition and sentence generation.
Cognition, 125, 288–308.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Vokey, J.R., & Brooks, L.R
(
1992)
Salience of item knowledge in learning artificial grammar.
Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 328–344.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by
Cited by 3 other publications
Andringa, Sible & Patrick Rebuschat
2015.
NEW DIRECTIONS IN THE STUDY OF IMPLICIT AND EXPLICIT LEARNING.
Studies in Second Language Acquisition 37:2
► pp. 185 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
Deocampo, Joanne A., Tricia Z. King & Christopher M. Conway
2019.
Concurrent Learning of Adjacent and Nonadjacent Dependencies in Visuo-Spatial and Visuo-Verbal Sequences.
Frontiers in Psychology 10
![DOI logo](//benjamins.com/logos/doi-logo.svg)
Katan, Pesia, Shani Kahta, Ayelet Sasson & Rachel Schiff
2017.
Performance of children with developmental dyslexia on high and low topological entropy artificial grammar learning task.
Annals of Dyslexia 67:2
► pp. 163 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
This list is based on CrossRef data as of 23 june 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.