A graded, associative account: Implicit learning of non-adjacent dependencies

Onnis, Luca; Destrebecqz, Arnaud; Christiansen, Morten H.; Chater, Nick; Cleeremans, Axel

doi:10.1075/sibil.48.10onn

Part of

Implicit and Explicit Learning of Languages
Edited by Patrick Rebuschat
[Studies in Bilingualism 48] 2015
► pp. 213–246

Implicit learning of non-adjacent dependencies

A graded, associative account

Luca Onnis | Nanyang Technological University

Arnaud Destrebecqz | Université Libre de Bruxelles

Morten H. Christiansen | Cornell University

Nick Chater | University of Warwick

Axel Cleeremans | Université Libre de Bruxelles

Language and other higher-cognitive functions require structured sequential behavior including non-adjacent relations. A fundamental question in cognitive science is what computational machinery can support both the learning and representation of such non-adjacencies, and what properties of the input facilitate such processes. Learning experiments using miniature languages with adult and infants have demonstrated the impact of high variability (Gómez, 2003) as well as nil variability (Onnis, Christiansen, Chater, & Gómez (2003; submitted) of intermediate elements on the learning of nonadjacent dependencies. Intriguingly, current associative measures cannot explain this U shape curve. In this chapter, extensive computer simulations using five different connectionist architectures reveal that Simple Recurrent Networks (SRN) best capture the behavioral data, by superimposing local and distant information over their internal ‘mental’ states. These results provide the first mechanistic account of implicit associative learning of non-adjacent dependencies modulated by distributional properties of the input. We conclude that implicit statistical learning might be more powerful than previously anticipated.

Keywords: artificial grammar learning, artificial language learning, computational models, connectionism, grammar, implicit learning, non-adjacent dependencies, sequence learning, Simple Recurrent Networks, statistical learning

Published online: 24 September 2015

https://doi.org/10.1075/sibil.48.10onn

References (60)

References

Allen, J., & Seidenberg, M.S. (1999). The emergence of grammaticality in connectionist networks. In B. MacWhinney (Ed.), The emergence of language (pp. 115–151). Mahwah, NJ: Lawrence Erlbaum Associates.

Botvinick, M., & Plaut, D.C. (2006). Short-term memory for serial order: A recurrent neural network model. Psychological Review, 113, 201–233.

. (2004). Doing without schema hierarchies: A recurrent connectionist approach to normal and impaired routine sequential action. Psychological Review, 111, 395–429.

Brakel, P., & Frank, S.L. (2009). Strong systematicity in sentence processing by simple recurrent networks. In N.A. Taatgen, H. van Rijn, J. Nerbonne & L. Schomaker (Eds.), Proceedings of the 31st Annual Conference of the Cognitive Science Society (pp. 1599–1604). Austin, TX: Cognitive Science Society.

Chater, N., & Conkey, P. (1992). Finding linguistic structure with recurrent neural networks. In Proceedings of the 14th Annual Conference of the Cognitive Science Society (pp. 402–407). Hillsdale, New Jersey: Psychology Press.

Chomsky, N. (1959). A review of BF Skinner's Verbal Behavior. Language, 35(1), 26–58.

Christiansen, M.H., Allen, J., & Seidenberg, M.S. (1998). Learning to segment speech using multiple cues: A connectionist model. Language and Cognitive Processes, 13, 221–268.

Christiansen, M.H., & Chater, N. (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23, 157–205.

Christiansen, M.H., Conway, C.M., & Curtin, S. (2000). A connectionist single-mechanism account of rule-like behavior in infancy. In L.R. Gleitman & A.K. Joshi (Eds.), The Proceedings of the 22nd Annual Conference of the Cognitive Science Society (pp. 83–88). Philadelphia, PA: University of Pennsylvania.

Christiansen, M.H., & MacDonald, M.C. (2009). A usage-based approach to recursion in sentence processing. Language Learning, 59(Suppl. 1), 126–161.

Cleeremans, A., Servan-Schreiber, D., & McClelland, J.L. (1989). Finite state automata and simple recurrent networks. Neural Computation, 1, 372–381.

Destrebecqz, A., & Cleeremans, A. (2003). Temporal factors in sequence learning. In Luis Jiménez (Ed.), Attention and implicit learning. Amsterdam: John Benjamins.

Cottrell, G.W., & Plunkett, K. (1995). Acquiring the mapping from meanings to sounds. Connection Science, 6, 379–412.

Dell, G.S., Juliano, C., & Govindjee, A. (1993). Structure and content in language production: A theory of frame constraints in phonological speech errors. Cognitive Science, 17, 149–195.

Dienes, Z. (1992). Connectionist and memory-array models of artificial grammar learning. Cognitive Science, 23, 53–82.

Dulany, D.E., Carlson, R.A., & Dewey, G.I. (1984). A case of syntactical learning and judgement: How conscious and how abstract? Journal of Experimental Psychology: General, 113, 541–555.

Elman, J.L. (1990). Finding structure in time. Cognitive science, 14(2), 179–211.

. (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7, 195–224.

Estes, K., Evans, J., Alibali, M., & Saffran, J. (2007). Can infants map meaning to newly segmented words? Psychological Science, 18(3), 254.

Farkaš, I., & Crocker, M.W. (2008). Syntactic systematicity in sentence processing with a recurrent self-organizing network. Neurocomputing, 71(7), 1172–1179.

Frank, M.C., Goldwater, S., Griffiths, T.L., & Tenenbaum, J.B. (2010). Modeling human performance in statistical word segmentation. Cognition, 117(2), 107–125.

Frank, S.L. (in press). Getting real about systematicity. In P. Calvo & J. Symons (Eds.), Systematicity and cognitive architecture: Conceptual and empirical issues 25 years after Fodor & Pylyshyn's challenge to connectionism. Cambridge, MA: The MIT Press.

Frinken, V., Fischer, A., Manmatha, R., & Bunke, H. (2012). A novel word spotting method based on recurrent neural networks. IEEE Transactions on, Pattern Analysis and Machine Intelligence, 34(2), 211–224.

Gaskell, M.G., Hare, M., & Marslen-Wilson, W.D. (1995). A connectionist model of phonological representation in speech perception. Cognitive Science, 19, 407–439.

Gibson, F.P., Fichman, M., & Plaut, D.C. (1997). Learning in dynamic decision tasks: Computational model and empirical evidence. Organizational Behavior and Human Decision Processes, 71, 1–35.

Gómez, R. (2002). Variability and detection of invariant structure. Psychological Science, 13, 431–436.

Harm, M.W., & Seidenberg, M.S. (1999). Phonology, reading acquisition, and dyslexia: Insights from connectionist models. Psychological Review, 106, 491–528.

Hinoshita, W., Arie, H., Tani, J., Okuno, H.G., & Ogata, T. (2011). Emergence of hierarchical structure mirroring linguistic composition in a recurrent neural network. Neural Networks, 24(4), 311–320.

Johnstone, T. & Shanks, D.R. (2001). Abstractionist and processing accounts of implicit learning. Cognitive Psychology, 42, 61–112.

Jordan, M.I. (1986). Attractor dynamics and parallelism in a connectionist sequential machine. In Proceedings of the Eighth Annual Conference of the Cognitive Science Society . Hillsdale, NJ: Lawrence Erlbaum Associates.

Kinder, A. & Shanks, D.R. (2001). Amnesia and the declarative/procedural distinction: A recurrent network model of classification, recognition, and repetition priming. Journal of Cognitive Neuroscience, 13, 648–669.

Kirov, C., & Frank, R. (2012). Processing of nested and cross-serial dependencies: An automaton perspective on SRN behaviour. Connection Science, 24(1), 1–24.

Lashley, K.S. (1951). The problem of serial order in behavior. In L.A. Jeffress (Ed.), Cerebral mechanisms in behavior (pp. 112–146). New York, NY: Wiley.

Luce, R.D. (1963). Detection and recognition. In R.D. Luce, R.R. bush, & E. Galanter (Eds.), Handbook of mathematical psychology. New York, NY: Wiley.

Maraqa, M., Al-Zboun, F., Dhyabat, M., & Zitar, R.A. (2012). Recognition of Arabic Sign Language (ArSL) using recurrent neural networks. Journal of Intelligent Learning Systems and Applications, 4(1), 41–52.

Maskara, A., & Noetzel, A. (1992). Forced simple recurrent neural network and grammatical inference. In Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society (pp. 420–425). Hillsdale, NJ: Lawrence Erlbaum Associates.

Miikkulainen, R., & Mayberry III, M.R. (1999). Disambiguation and grammar as emergent soft constraints. In B. MacWhinney (Ed.), Emergence of language, 153–176. Mahwah, NJ: Lawrence Erlbaum Associates.

Misyak, J.B., & Christiansen, M.H. (2012). Statistical learning and language: An individual differences study. Language Learning, 62, 302–331.

Misyak, J.B., Christiansen, M.H. & Tomblin, J.B. (2010a). On-line individual differences in statistical learning predict language processing. Frontiers in Psychology, Sept.14.

. (2010b). Sequential expectations: The role of prediction- based learning in language. Topics in Cognitive Science, 2, 138–153.

Moss, H.E., Hare, M.L., Day, P., & Tyler, L.K. (1994). A distributed memory model of the associative boost in semantic priming. Connection Science, 6, 413–427.

Munakata, Y., McClelland, J.L., & Siegler, R.S. (1997). Rethinking infant knowledge: Toward an adaptive process account of successes and failures in object permanence tasks. Psychological Review, 104, 686–713.

Onnis, L., Christiansen, M.H., Chater, N., & Gómez, R. (submitted). Statistical learning of non-adjacent relations. Submitted manuscript.

. (2003). Reduction of uncertainty in human sequential learning: Preliminary evidence from Artificial Grammar Learning. In R. Alterman & D. Kirsh (Eds.), Proceedings of the 25^th Annual Conference of the Cognitive Science Society. Boston, MA: Cognitive Science Society.

Onnis, L., Monaghan, P., Christiansen, M.H., & Chater, N. (2004). Variability is the spice of learning, and a crucial ingredient for detecting and generalizing in nonadjacent dependencies. In Proceedings of the 26th annual conference of the Cognitive Science Society (pp. 1047–1052). Mahwah, NJ: Lawrence Erlbaum.

Pacton, S., Perruchet, P., Fayol, M., & Cleeremans, A. (2001). Implicit learning out of the lab: The case of orthographic regularities. Journal of Experimental Psychology: General, 130, 401–426.

Perruchet, P., & Pacteau, C. (1990). Synthetic grammar learning: Implicit rule abstraction or explicit fragmentary knowledge? Journal of Experimental Psychology: General, 119, 264–275.

Perruchet, P., & Pacton, S. (2006). Implicit learning and statistical learning: One phenomenon, two approaches. Trends In Cognitive Sciences, 10(5), 233–238.

Plaut, D.C., & Kello, C.T. (1999). The emergence of phonology from the interplay of speech comprehension and production: A distributed connectionist approach. In B. MacWhinney (Ed.), The emergence of language (pp. 381–415). Mahwah, NJ: Lawrence Erlbaum Associates.

Redington, M., & Chater, N. (2002). Knowledge representation and transfer in artificial grammar learning (AGL). In R.M. French & A. Cleeremans (Eds.), Implicit learning and consciousness: An empirical, philosophical, and computational consensus in the making. Hove: Psychology Press.

Rohde, D.L.T., & Plaut, D.C. (1999). Language acquisition in the absence of explicit negative evidence: How important is starting small? Cognition, 72, 67–109.

Saffran, J.R., Aslin, R.N., & Newport, E.L. (1996). Statistical learning by 8-month-old infants. Science, 274, 1926–1928.

Saffran, J. (2001). Words in a sea of sounds: The output of infant statistical learning. Cognition, 81, 149–169.

Servan-Schreiber, D., Cleeremans, A. & McClelland, J.L. (1991). Graded state machines: The representation of temporal dependencies in simple recurrent networks. Machine Learning, 7, 161–193.

Si, Y., Xu, J., Zhang, Z., Pan, J., & Yan, Y. (2012). An improved Mandarin voice input system using recurrent neural network language model. In Computational Intelligence and Security (CIS), Eighth International Conference on IEEE (pp. 242–246).

Socher, R., Manning, C.D., & Ng, A.Y. (2010). Learning continuous phrase representations and syntactic parsing with recursive neural networks. In Proceedings of the NIPS-2010 Deep Learning and Unsupervised Feature Learning Workshop . Hilton: Cheakmus.

Sutskever, I., Martens, J., & Hinton, G. (2011). Generating text with recurrent neural networks. In Proceedings of the 2011 International Conference on Machine Learning (ICML-2011).

Tabor, W. (2011). Recursion and recursion-like structure in ensembles of neural elements. In H. Sayama, A. Minai, D. Braha, & Y. Bar-Yam (Eds.), Unifying themes in complex systems. Proceedings of the VIII International Conference on Complex Systems (pp. 1494–1508). Berlin: Springer.

Takac, M., Benuskova, L, & Knott, A. (2012). Mapping sensorimotor sequences to word sequences: A connectionist model of language acquisition and sentence generation. Cognition, 125, 288–308.

Vokey, J.R., & Brooks, L.R. (1992). Salience of item knowledge in learning artificial grammar. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 328–344.

Cited by (3)

Cited by three other publications

Deocampo, Joanne A., Tricia Z. King & Christopher M. Conway

2019. Concurrent Learning of Adjacent and Nonadjacent Dependencies in Visuo-Spatial and Visuo-Verbal Sequences. Frontiers in Psychology 10

Katan, Pesia, Shani Kahta, Ayelet Sasson & Rachel Schiff

2017. Performance of children with developmental dyslexia on high and low topological entropy artificial grammar learning task. Annals of Dyslexia 67:2 ► pp. 163 ff.

Andringa, Sible & Patrick Rebuschat

2015. NEW DIRECTIONS IN THE STUDY OF IMPLICIT AND EXPLICIT LEARNING. Studies in Second Language Acquisition 37:2 ► pp. 185 ff.

This list is based on CrossRef data as of 24 july 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.