This article focuses on individual differences in emotion recognition ability among 356 first language (L1) and 564 foreign language (LX) users of English. Recognizing emotions can be particularly challenging in LX contexts. Depending on their linguistic profile, individuals may interpret input very differently, and LX learners and users have been found to perform significantly worse than native control groups (Rintell 1984) in tests of emotion recognition ability. The present article investigates the effect of three independent variables, namely, L1 versus LX status, proficiency in English, and cultural background, on emotion recognition ability. It used an online survey in which participants had to identify the emotion portrayed by a native English-speaking actress in six audiovisual clips. Despite LX users having lower proficiency scores, English L1 users and LX users’ emotion recognition ability scores were broadly similar. A significant positive relationship was found between LX proficiency and emotion recognition ability. A similar but only marginally significant relationship emerged among L1 users. A significant effect of L1 culture was found on emotion recognition ability scores, with Asian LX users scoring significantly lower than European LX users. It thus seems that audiovisual input allows advanced LX users to recognize emotions in LX as well as L1 users. That said, LX proficiency and L1 culture do have an effect on emotion recognition ability.