Article published In:
PragmaticsVol. 31:1 (2021) ► pp.114–143
A Tale of four measures of pragmatic knowledge in an EFL institutional context
The upsurge of interest in L2 pragmatics studies has coincided with a growing interest in pragmatic assessment. Employing the most efficient measure of pragmatics has led many researchers to examine the existing measures to pinpoint the most useful ones. This study was an attempt to compare and contrast Written Discourse Completion Task (WDCT), Oral Discourse Completion Task (ODCT), and Role-play with Natural methodology in an EFL institutional context to see which measure approximated Natural methodology. To this end, data (requests) were collected from 27 intermediate–level Iranian EFL learners in a natural classroom institutional context over 15 weeks, and then the WDCTs, ODCTs, and Role-plays with the same contextual features were selected to elicit the intended data. The participants’ requests were transcribed and analyzed in terms of Schauer’s (2009) request head act strategy taxonomy and its internal and external modification devices. The results of Binominal tests indicated that, in spite of some minor similarities, none of the elicitation measures could approximate the natural data. The participants’ employment of direct, non-conventionally indirect request strategies, and internal and external modification devices were more conspicuous in the WDCTs, ODCTs, and Role-plays than those in the Natural methodology. The study implies that data collection methods should be selected based on researchers’ objectives and research questions.
Keywords: WDCT, ODCT, Role-play, Natural methodology, request speech act, internal and external modification devices
Article outline
- 1.Introduction
- 2.Literature review
- 2.1Pragmatic measurement
- 2.2Production measures of pragmatics
- 2.2.1Naturally-occurring data
- 2.2.2Written discourse completion task
- 2.2.3Oral discourse completion task
- 2.2.4Role-play
- 2.3Studies focusing on pragmatic measures
- 2.4Request speech act
- 3.Methodology
- 3.1Participants
- 3.2Instruments
- 3.3Data collection procedure
- 3.4Data analysis
- 4.Results
- 4.1Request strategies
- 4.2Internal modification devices
- 4.3External modification devices
- 5.Discussion
- 6.Conclusion
-
References
Available under the Creative Commons Attribution-NonCommercial (CC BY-NC) 4.0 license.
References
Bardovi-Harlig, Kathleen
2018 “Matching Modality in L2 Pragmatics Research Design”. System: 751, 13–22. 
Bardovi-Harlig, Kathleen, and Beverly Hartford
2005 Institutional Discourse and Interlanguage Pragmatics Research. In Interlanguage pragmatics: exploring institutional talk, ed. by Kathleen Bardovi-Harlig, and Beverly Hartford, 7–36. Mahwah, NJ: Lawrence Erlbaum.
Bardovi-Harlig, Kathleen, and Sun-Young Shin
2014 “Expanding Traditional Testing Measures with Tasks from L2 Pragmatics Research.” Iranian Journal of Language Testing, 4 (1): 26–49.
Brown, J. D.
2001 “Pragmatics Test: Different Purposes, Different Tests.” In Pragmatics in Language Teaching, ed. by Kenneth. R. Rose, and Gabriele Kasper, 301–325. Cambridge: Cambridge university press.
Brown, Penelope, and Steven, C. Levinson
1987 Politeness: Some Universals in Language Usage. Cambridge: Cambridge University Press.
Economidou-Kogetsidis, Maria
2013 “Strategies, Modification and Perspective in Native Speakers’ Requests: A Comparison of WDCT and Naturally Occurring Requests.” Journal of Pragmatics, 531: 21–38.
Ellis, Rod
2008 The Study of Second Language Acquisition (2nd ed.). Oxford: Oxford University Press.
Eslami, Zohreh. R., and Azizullah Mirzaei
2014 “Speech Act Data Collection in a Non-Western Context: Oral and Written DCTs in the Persian Language.” Iranian Journal of Language Testing, 4(1): 137–154.
Félix-Brasdefer, J. Cesar
2010 “Data Collection Methods in Speech Act Performance: DCTS, Role Plays, and Verbal Reports.” In Speech Act Performance: Theoretical, Empirical and Methodological Issues, ed. by Alicia Martínez-Flor and Esther Usó-Juan, 64–81. Amsterdam, Netherlands: John Benjamins.
2007 “Natural Speech vs. Elicited Data: A Comparison of Natural and Role Play Requests in Mexican Spanish.” Spanish in Context, 4(2): 159–185.
Golato, Andrea
2003 “Studying Compliment Responses: A Comparison of DCTs and Recordings of Naturally Occurring Talk.” Applied Linguistics, 24(1): 90–121.
House, Juliane
2018 “Authentic vs. Elicited Data and Qualitative vs. Quantitative Research Methods in Pragmatics: Overcoming Two Non-Fruitful Dichotomies.” System, 751:4–12.
Hudson, Thom
2001 “Indicators for Pragmatic Instruction.” In Pragmatics in Language Teaching, ed. by Kenneth R. Rose and Gabriele Kasper, 283–300. Cambridge: Cambridge university press.
Ishihara, Noriko
2010 “Instructional Pragmatics: Bridging Teaching, Research, and Teacher Education.” Language and Linguistics Compass, 4(10): 938–953.
Jalilifar, Alireza
2009 “Request Strategies: Cross-Sectional Study of Iranian EFL Learners and Australian Native Speakers.” English Language Teaching, 2(1): 46–61.
Kasper, Gabriele
2000 “Data Collection in Pragmatics Research.” In Culturally Speaking: Managing Rapport Through Talk across Cultures, ed. by Helen Spencery-Oatey, 316–41. London and New York: Continuum.
Kasper, Gabriele and Carsten Roever
2005 ”Pragmatics in Second Language Learning.” In Handbook of Research in Second Language Teaching and Learning, ed. by Eli Hinkel, 317–334. Mahwah/New Jersey: Lawrence Erlbaum Associates.
Labov, William
1972 Sociolinguistic Patterns. Philadelphia: University of Philadelphia.
Martínez-Flor, Alicia and Esther Usó-Juan
2011 “Research Methodologies in Pragmatics: Eliciting Refusals to Requests.” ELIA, 111: 47–87.
Norris, John. and Lourdes Ortega
2000 “Effectiveness of L2 Instruction: A Research Synthesis and Quantitative Meta-Analysis.” Language Learning, 50 (3): 417–528.
Rose, Kenneth. And Reiko Ono
1995 “Eliciting Speech Act Data in Japanese: The Effect of Questionnaire Type.” Language Learning, 45(2): 191–223.
Sasaki, Miyuki
1998 “Investigating EFL Students’ Production of Speech Acts: A Comparison of Production Questionnaires and Role Plays.” Journal of Pragmatics, 301: 457–484.
Schauer, Gila. A.
2009 Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.
Sifianou, Maria
1999 Politeness Phenomena in England and Greece. A Crosscultural Perspective. Oxford: Oxford University Press.
Taguchi, Naoko
2018 “Data Collection and Analysis in Developmental L2 Pragmatics Research: Discourse Completion Test, Role Play, and Naturalistic Recording.” In Critical Reflections on Data in Second Language Acquisition, ed. by Aarnes Gudmestad and Amanda Edmonds, 7–32. Amsterdam, Netherlands: John Benjamins.
Taguchi, Naoko, and Carsten Roever
2017 Second Language Pragmatics. Oxford: Oxford University Press.
Takahashi, Satomi
2010 “Assessing Learnability in Second Language Pragmatics.” In, Pragmatics across Languages and Cultures, ed. by Anna Trosborg, 391–423. Berlin, Germany: De Gruyter Mouton.
Turnbull, William
2001 “An Appraisal of Pragmatic Elicitation Techniques for the Social Psychological Study of Talk: The Case of Request Refusals.” Pragmatics, 11(1): 31–61.
Uso-Juan, Esther
2010 “Requests: A Sociopragmatic Approach.” In Speech Act Performance: Theoretical, Empirical and Methodological Issues ed. by Alicia Martinez-Flor and Esther Uso-Juan, 237–256. Amsterdam, Netherlands: John Benjamins.
Uso-Juan, Esther, and Alicia Martinez-Flor
2014 “Reorienting the Assessment of the Conventional Expressions of Complaining and Apologizing: From Single Response to Interactive DCTs.” Iranian Journal of Language Testing, 4(1): 113–136.
Woodfield, H.
2012 “I Think Maybe I Want to Lend the Notes from You: Development of Request Modification in Graduate Learners.” In Interlanguage Request Modification, ed. by Maria Economidou-Kogetsidis and Helen Woodfield, 9–49. Amsterdam, Netherlands: John Benjamins.
Yuan, Y.
2001 “An Inquiry into Empirical Pragmatics Data Gathering Methods: Written DCTs, Oral DCTs, and Natural Conversations.” Journal of pragmatics, 331: 271–92.