This research examined the extent to which pragmatic comprehension, namely accurate and speedy comprehension of conversational implicatures, is related to cognitive processing skills and general listening abilities. Thirty-five Japanese students learning English as a second language completed five tasks: (1) a pragmatic listening test (PLT) that measured the ability to comprehend implied speakers’ intentions, (2) a phonemic discrimination test, (3) a listening section of the institutional TOEFL, (4) a working memory test, and (5) a lexical access test that measured the ability to make speedy semantic judgment. The students’ pragmatic comprehension was analyzed for accuracy (scores) and comprehension speed (time taken to answer items correctly). Results revealed a significant relationship between accuracy scores of the PLT and the TOEFL listening scores, but not with phonemic discrimination ability. Response time of pragmatic listening significantly correlated with the semantic access speed, but not with working memory.