This inquiry explored linguistic influences on comprehensibility (understanding ease) and accentedness (linguistic nativelikeness) in second language (L2) learners’ extemporaneous speech. Target materials covered picture narratives from 40 native French speakers of English from different competence levels. The narratives were then rated by 20 native speakers with or without linguistic and pedagogical experience for comprehensibility, accentedness, and 11 linguistic variables spanning the domains of phonology, lexis, grammar, and discourse structure. Outcomes displayed that comprehensibility was associated with various linguistic variables (vowel/consonant errors, word stress, fluency, lexis, grammar), whereas accentedness was mainly related to pronunciation (vowel/consonant errors, word stress). Native-speaking listeners thus seem to pay specific attention to pronunciation, rather than lexis and grammar, to assess nativelikeness but tend to consider various sources of linguistic information in L2 speech in evaluating comprehensibility. The use of listener ratings (perceptual measures) in judging linguistic aspects of learner speech and their implications for language evaluation and pedagogy are debated.