Detailed empirical data are needed as a basis for hypothesis testing and model evaluation in L2 speech perception. Predictions from current models depend upon notions of similarity and dissimilarity between L1 and L2 sounds. New data from 20 English listeners labeling 19 Korean syllable-initial consonants in three vowel contexts as the closest English consonant, as well as judging similarity to the English consonant are presented. Results are contrasted with those from an earlier parallel study of Korean listener labeling of English syllable initial consonants. L2 speakers perceive sounds within the comprehensive framework of their L1 phonological system using language specific cues. A better knowledge of cross-language relationships will result in better modeling and experimental design.