Flege’s Speech Learning Model (SLM) focuses on how the perception and production of sounds in a second language (L2) are influenced by the first or native language (L1). A central claim of the model is that as the phonetic space becomes increasingly devoted or committed to the L1, L2 speech learning becomes more difficult. In particular, L2 learners have trouble establishing new perceptual categories, especially for sounds that are similar to those in the L1, and with respect to production, foreign accents are more likely to result. A corollary claim of SLM is that L1 sound perception is still quite flexible or “open” in childhood, in contrast to the impression given by much of the infant speech perception literature. This Lexical Restructuring Model (LRM) is also concerned with L1 sound perception in childhood, but focuses to a greater extent on the relations between phonological and lexical levels of representation and processing. According to LRM, a) lexical representations are initially holistic (i.e., based on units larger than the phonemic/phonetic segment, such as the syllable), and b) as the child’s mental lexicon grows, representations become more fine-grained and/or segmentally-based. This paper elaborates on these claims of the two models and their implications for spoken word recognition by native and non-native listeners, both children and adults. The extent to which these claims are supported by recent findings are considered, as well as suggestions for future research.