Rater variation in the assessment of speech acts
This study addresses variability among native speaker raters who evaluated pragmatic performance of learners of English as a foreign language. Using a five-point rating scale, four native English speakers of mixed cultural background (one African American, one Asian American, and two Australians) assessed the appropriateness of two types of speech acts (requests and opinions) produced by 48 Japanese EFL students. To explore norms and the reasoning behind the raters’ assessment practice, individual introspective verbal interviews were conducted. Eight students’ speech act productions (64 speech acts in total) were selected randomly, and the raters were asked to rate each speech act and then explain their rating decision. Interview data revealed similarities and differences in their use of pragmatic norms and social rules in evaluating appropriateness.
Keywords: Assessment, Rater variation, Pragmatic competence, Speech acts
Available under the Creative Commons Attribution-NonCommercial (CC BY-NC) 4.0 license.
Published online: 01 September 2011
Bachman, Lyle, and Adrian Palmer
Brown, James D.
Cohen, Andrew, and Elite Olshtain
Ducassee, Ana M.
Hudson, Thom, Emily Detmer, and James D. Brown
Johnson, Jeff, and Gad Lim
(1983) Pragmatics. Cambridge: Cambridge University Press. BoP
(2007) Interaction in a paired speaking test: The rater's perspective. Ph.D. dissertation: The University of Melbourne.
McNamara, Tim, and Tom Lumley
(1993) Pragmatics: An Introduction. Oxford: Blackwell. BoP
Poliitt, Alastair, and Neil Murray
(2006) Analysis of appropriateness in a speech act of request in L2 English. Pragmatics 16: 513-535. BoP
(1995) Meaning in Interaction: An Introduction to Pragmatics. London: Longman. BoP
(1998) Investigating EFL students' production of speech acts: A comparison of production questionnaires and role-plays. Journal of Pragmatics 30: 457-484. BoP
(2001) Using pictures for research in pragmatics-eliciting pragmatic strategies by picture response tests. In T. Hudson & J.D. Brown (eds.), A Focus on Language Test Development: Expanding the Language Proficiency Construct Across a Variety of Tests. (Technical Report #21). University of Hawaii, Second language Teaching and Curriculum Center, pp. 35-56.
(1997) Measuring interlanguage pragmatic competence of Japanese students of English as a foreign language: A multi-test framework evaluation. Unpublished Ph.D. thesis, Columbia Pacific University, Novata, CA.
Cited by 14 other publications
Ajabshir, Zahra Fakher
Alemi, Minoo, Zohreh R. Eslami & Atefeh Rezanejad
Susan Conrad, Alissa Hartig & Lynn Santelmann
Hernandez, Todd A.
Hernández, Todd A.
Kusevska, Marija, Biljana Ivanovska, Nina Daskalovska & Tatjana Ulanska
Li, Shuai, Naoko Taguchi & Feng Xiao
Santelmann, Lynn & Tetyana Sydorenko
Tajeddin, Zia & Minoo Alemi
This list is based on CrossRef data as of 19 april 2021. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.