This article presents a theoretical framework for the understanding of semantic processing of people with Asperger’s Syndrome (AS). Based on the hypothesis that individuals with AS have a specific linguistic deficit affecting less systemized aspects of semantic processing, individuals with AS using metaphoric expressions presented out of a larger context were studied. The metaphoric expressions included novel metaphoric combinations (e.g., WILTING HOPE), the comprehension of which could not be helped by previous exposure and learning. Instead, comprehension relied on participants’ ability to connect two concepts, seemingly unrelated, in a non-systemized way. The findings show that whereas literal expressions did not pose any particular interpretation problems to participants with AS, metaphoric expressions, and especially novel ones, were mainly interpreted incorrectly, suggesting that there are two different types of semantic processing: one which is rule-based (literal language) and one involving a degree of rule-violation (metaphorical language).