This article examines the influence of perception on creole phonological restructuring, drawing comparisons to loanword adaptation and second language learning and outlining a formal framework within which change can be described and explained. The three scenarios of contact-induced modification are compared and contrasted, focusing on the nature of contact, the role of different source and target languages, and the means by which participants access source tokens. Data from Haitian, showing diachronic modification to lexifier rhotics, is used to illustrate the position that perception may be the primary causal factor in phonological modification in some instances. It is argued that source ambiguity and substrate (L1) perceptual knowledge underlie restructuring. Perceptual competence is formalized in a broadly Optimality Theoretic grammar using level-specific constraints referring to the parsing and categorization of experiential input. Putative substrate data are used to establish a perceptual grammar which, when applied to experiential input, predict attested outcomes in Haitian.