This article discusses the importance of social interaction for the development of the representations for symbolic communication. It suggests that there is no need to distinguish between different representational systems emerging at different stages of development. Instead, it proposes that representations are rich right from the beginning of a child’s life, and that they are driven mainly by acting and interacting in the physical and social world. The more variety in a child’s interactional experience (i.e., synchrony, sequentiality, and prediction), the more enriched and abstracted the representations become. The article reviews literature providing evidence for the ways in which infants’ development toward symbolic communication benefits from repeated social (inter)action and considers some implications for computational approaches.