Assessing L2 learners’ pragmatic ability in problem-solving situations at English-medium university
This paper discusses the oral pragmatic ability of current and prospective university students (international students) in an
English-speaking country by examining how they are differentiated in the assessed levels of this target ability. A total of 67 students, all
of whom were second language (L2) speakers of English, completed a set of oral discourse production tasks simulating interpersonal settings
encountered at university. Their task performances were analyzed qualitatively and quantitatively. The results show a substantial gap
between the highest performing university students and the prospective students in terms of their measured pragmatic ability. However, some
university students scored quite poorly and were actually outperformed by a certain number of prospective students who were preparing to
satisfy the English language requirements for university admission. These findings suggest that being a university student is not
necessarily an indicator of having sufficiently developed pragmatic ability to participate in interpersonal settings in an academic
environment. The findings shed light on the importance of providing students learning and assessment opportunities for pragmatics so they
can gain greater awareness of their own developing pragmatic ability in an English-medium university.
Article outline
- 1.Introduction
- 2.Literature review
- 3.The present study
- 3.1Assessment tasks
- 3.2Participants
- 3.2.1Test-takers
- 3.2.2Interlocutors for the dialogue tasks
- 3.2.3Raters
- 3.3Procedures
- 3.3.1Test administration procedure
- 3.3.2Rating
- 3.4Data analysis
- 4.Results
- 4.1RQ1: Pragmatic features discriminating among test-takers
- 4.1.1Actions accomplished in pursuit of the communicative goal
- 4.1.2Use of linguistically oriented features
- 4.1.3Use of interactionally oriented features
- 4.2Differentiation of university-level and preentry test-takers
- 5.Discussion
- 6.Limitations and future directions
- Acknowledgements
- Note
-
References
References
Al-Gahtani, S., & Roever, C.
(
2012)
Proficiency and sequential organization of L2 requests.
Applied Linguistics, 33(1), 42–65.


Al-Gahtani, S., & Roever, C.
(
2013)
‘Hi doctor, give me handouts’: Low-proficiency learners and requests.
ETL Journal, 67(4), 413–424.

Atkinson, J., & Heritage, J.
(Eds (
1984)
Structures of social action. Cambridge University Press.

Brown, P., & Levinson, S. C.
(
1987)
Politeness: Some universals in language usage. Cambridge University Press.


Cohen, A. D.
(
2018)
Learning pragmatics from native and non-native language teachers. Multilingual Matters.


Crystal, D.
(
1997)
The Cambridge encyclopedia of language (2nd ed.). Cambridge University Press.

Galaczi, E.
(
2014)
Interactional competence across proficiency levels: How do learners manage interaction in paired speaking tests? Applied Linguistics, 35(5), 553–574.


Grabowski, K.
(
2013)
Investigating the construct validity of a role-play test designed to measure grammatical and pragmatic knowledge at multiple proficiency levels. In
S. Ross &
G. Kasper (Eds.),
Assessing second language pragmatics (pp. 149–171). Palgrave Macmillan.


Heritage, J.
(
1984)
Garfinkel and ethnomethodology. Policy Press.

Hudson, T., Detmer, E., & Brown, J. D.
(
1992)
A framework for testing cross-cultural pragmatics (Technical report #2). University of Hawai‘i, Second Language Teaching and Curriculum Center.

Ikeda, N.
(
2017)
Measuring L2 oral pragmatic abilities for use in social contexts: Development and validation of an assessment instrument for L2 pragmatic performance in university settings [Unpublished doctoral dissertation]. University of Melbourne.
Ishida, M.
(
2009)
Development of interactional competence: Changes in the use of ne in L2 Japanese during study abroad. In
H. T. Nguyen &
G. Kasper (Eds.),
Talk-in-interaction: Multilingual perspective (pp. 351–385). National Foreign Language Resource Center, University of Hawai‘i.

Ishida, M.
(
2011)
Engaging in another person’s telling as a recipient in L2 Japanese: Development of interactional competence during one-year study abroad. In
G. Pallotti &
J. Wanger (Eds.),
L2 Learning as social practice: Conversation-analytic perspectives (pp. 45–85). National Foreign Language Resource Center, University of Hawai‘i.

Ishihara, N.
(
2014)
Assessment of pragmatics in the classroom. In
N. Ishihara &
A. D. Cohen (Eds.),
Teaching and learning pragmatics (pp. 287–317). Routledge.


Kane, M. T.
(
2006)
Validation. In
R. L. Brennan (Ed.),
Educational Measurement (4th ed., pp. 17–64). Greenwood Publishing.

Kasper, G., & Ross, S.
(
2013)
Assessing second language pragmatics: An overview and introductions. In
S. Ross &
G. Kasper (Eds.),
Assessing second language pragmatics (pp. 1–40). Palgrave Macmillan.

Kim, Y.
(
2009)
The Korean discourse markers-nuntey and kuntey in native-nonnative conversation: An acquisitional perspective. In
H. T. Nguyen &
G. Kasper (Eds.),
Talk-in-interaction: Multilingual perspective (pp. 317–350). National Foreign Language Resource Center, University of Hawai‘i.

Linacre, M.
(
2014a)
A user’s guide to FACETS Rasch-Model computer programs. [Computer software]. Winsteps.com.

Linacre, J. M.
(
2014b)
Facets computer program for many facet Rasch measurement (Version 3.71.4) [Computer software]. Winsteps.com.

McNamara, T. F.
(
1996)
Measuring second language performance. Longman.

McNamara, T. F., & Roever, C.
(
2006)
Language testing: The social dimension. Blackwell.

Okada, Y.
(
2010)
Role play in oral proficiency interviews: Interactive footing and interactional competencies.
Journal of Pragmatics, 42(16), 1647–1468.


Pekarek Doehler, S., & Pochon-Berger, E.
(
2011)
Developing ‘methods’ for interaction: A cross-sectional study of disagreement sequences in French L2. In
J. K. Hall,
J. Hellermann, &
S. Pekarek Doehler (Eds.),
L2 interactional competence and development (pp. 206–243). Multilingual Matters.


Roever, C.
(
2005)
Testing ESL pragmatics. Peter Lang.


Roever, C.
(
2011)
Testing of second language pragmatics: Past and future.
Language Testing, 28(4), 463–481.


Roever, C.
(
2013)
Testing implicature under operational conditions. In
S. Ross &
G. Kasper (Eds.),
Assessing second language pragmatics (pp. 43–64). Palgrave Macmillan.


Roever, C., Fraser, C., & Elder, C.
(
2014)
Testing ESL sociopragmatics: Development and validation of a web-based test battery. Peter Lang.


Schegloff, E. A.
(
2007)
Sequence organization in interaction: A primer in conversation analysis. Cambridge University Press.


Searle, J. R.
(
1969)
Speech acts: An essay in the philosophy of language. Cambridge University Press.


Taguchi, N.
(
2012)
Context, individual differences and pragmatic competence. Multilingual Matters.


Taguchi, N.
(
2019)
The Routledge handbook of second language acquisition and pragmatics. Routledge.


Taguchi, N., & Roever, C.
(
2017)
Second language pragmatics. Oxford University Press.

Timpe, V.
(
2013)
Assessing intercultural language learning. Peter Lang.

Walters, F. S.
(
2007)
A conversation-analytic hermeneutic rating protocol to assess L2 oral pragmatic competence.
Language Testing, 24(2), 155–183.


Youn, S. J.
(
2013)
Validating task-based assessment of L2 pragmatics in interaction using mixed methods [Unpublished doctoral dissertation]. University of Hawai‘i at Manoa.
Youn, S. J.
(
2015)
Validity argument for assessing L2 pragmatics in interaction using mixed methods.
Language Testing, 32(2), 199–225.


Youn, S.J., & Bogorevich, V.
(
2019)
Assessment in L2 pragmatics. In
N. Taguchi (Ed.),
The Routledge handbook of second language acquisition and pragmatics (pp. 308–321). Routledge.


Cited by
Cited by 3 other publications
Dai, David Wei & Michael Davey
2023.
On the Promise of Using Membership Categorization Analysis to Investigate Interactional Competence.
Applied Linguistics 
Roever, Carsten & Naoki Ikeda
2022.
What scores from monologic speaking tests can(not) tell us about interactional competence.
Language Testing 39:1
► pp. 7 ff.

This list is based on CrossRef data as of 10 september 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.