Chapter 7
Pragmatics
Assessing learning outcomes in instructional studies
The field of instructed SLA (ISLA) has grown rapidly in recent years to examine how systematic manipulations of instructional conditions can lead to the development of second language (L2) knowledge and use (Loewen & Sato, 2017). Following this trend, L2 pragmatics researchers have implemented various instructional methods and examined their effectiveness using experimental designs (Taguchi, 2015; Taguchi & Roever, 2017). The critical issue in this practice is how to assess learning outcomes in order to make a claim that certain instructional methods can produce robust pragmatic knowledge. To address this question, this chapter presents an overview of assessment methods used in analyzing pragmatics learning outcomes. The chapter surveys instructed L2 pragmatics studies published in the last four decades to identify common assessment methods (i.e., discourse completion tasks and role-play tasks). We provide step-by-step illustrations of expert studies in order to demonstrate how researchers design an assessment task, evaluate learning, and interpret results. The chapter concludes with a discussion of the challenges and limitations of current methods and provides direction for addressing such challenges in pragmatics research.
Article outline
- 1.Introduction: What is pragmatics and why is it important?
- 2.What we know and what we need to know about pragmatics in ISLA
- 2.1Current trends in instructed pragmatics
- 3.Data elicitation and interpretation options for pragmatics
- 3.1Discourse completion tasks (DCTs)
- 3.1.1Characteristics of DCTs
- 3.1.2Pros and cons of DCTs
- 3.1.3Step-by-step DCT illustration
- 3.2Role-play tasks
- 3.2.1Characteristics of role-play tasks
- 3.2.2Pros and cons of role-play tasks
- 3.2.3Step-by-step role play illustration
- 4.Advice to future pragmatics researchers in ISLA: How to improve assessment methods of learning outcomes
- 4.1Incorporating prosody as instructional targets and learning outcomes
- 4.2Improving authenticity of DCTs and role-plays via technology applications
- 4.3Enhancing consequentiality in role-play interaction
- 5.Troubleshooting ISLA pragmatics research
- Common question 1: How can we choose the most appropriate method to assess learning outcomes? Should we choose DCTs or role-plays?
- Common question 2: How can we determine assessment criteria and score bands when evaluating learning outcomes?
- 6.Conclusions
- 7.Further reading and additional resources
- 7.1Suggested books
- 7.2Suggested journals, professional organizations, and websites
-
Note
-
References
-
Appendix
References (47)
References
Alemi, M., & Haeri, N. S. (2020). Robot-assisted instruction of L2 pragmatics: Effects on young EFL learners’ speech act performance. Language Learning & Technology, 24(2), 86–103.
Al-Gahtani, S., & Roever, C. (2012). Proficiency and sequential organization of L2 requests. Applied Linguistics, 33(1), 42–65.
Al-Gahtani, S., & Roever, C. (2018). Proficiency and preference organization in second language refusals. Journal of Pragmatics, 129, 140–153.
Blum-Kulka, S., House, J., & Kasper, G. (1989). Cross-cultural pragmatics: Requests and apologies. Ablex.
Boersma, P., & Weenink, D. (2016). Praat: Doing phonetics by computer (Version 6.0.11) [Software]. Available from [URL]
Chen, S. (2019, October 4–5). Developing an L2 pragmatic speaking test using conversation analysis findings [Paper presentation]. Annual Midwest Association of Language Testers (MwALT) conference, Indiana University Bloomington.
Cohen, A. D., & Shively, R. L. (2007). Acquisition of requests and apologies in Spanish and French: Impact of study abroad and strategy-building intervention. Modern Language Journal, 91(2), 189–212.
Eslami, Z. R., Mirzaei, A., & Dini, S. (2015). The role of asynchronous computer mediated communication in the instruction and development of EFL learners’ pragmatic competence. System, 48, 99–111.
Félix-Brasdefer, J. C. (2007). Pragmatic development in the Spanish as a FL classroom: A cross-sectional study of learner requests. Intercultural Pragmatics, 4(2), 253–286.
Golato, A. (2003). Studying compliment responses: A comparison of DCTs and recordings of naturally occurring talk. Applied Linguistics, 24(1), 90–121.
Grabowski, K. C. (2009). Investigating the construct validity of a test designed to measure grammatical and pragmatic knowledge in the context of speaking [Unpublished doctoral dissertation]. Columbia University.
Gumperz, J. J. (1982). Discourse strategies. Cambridge University Press.
Gumperz, J. J., & Cook-Gumperz, J. (1982). Introduction: Language and the communication of social identity. In J. J. Gumperz (Ed.), Language and social identity (pp. 1–21). Cambridge University Press.
Halenko, N. (2013). Using computer animation to assess and improve spoken language skills. In Conference proceedings: ICT for language learning (p. 286). Libreriauniversitaria.
Halenko, N., & Jones, C. (2017). Explicit instruction of spoken requests: An examination of pre-departure instruction and the study abroad environment. System, 68, 26–37.
Hassall, T. (2020). Preference structure in request sequences: What about role-play? Journal of Pragmatics, 155, 321–333.
Huth, T. (2010). Can talk be inconsequential? Social and interactional aspects of elicited second-language interaction. Modern Language Journal, 94(4), 537–553.
Kasper, G. (2008). Data collection in pragmatics research. In H. Spencer-Oatey (Ed.), Culturally speaking: Culture, communication and politeness theory (pp. 279–302). Continuum.
Kasper, G., & Dahl, M. (1991). Research methods in interlanguage pragmatics. Studies in Second Language Acquisition, 13(2), 215–247.
Kasper, G., & Youn, S. J. (2018). Transforming instruction to activity: Roleplay in language assessment. Applied Linguistics Review, 9(4), 589–616.
Li, S. (2012). The effects of input-based practice on pragmatic development of requests in L2 Chinese. Language Learning, 62(2), 403–438.
Loewen, S., & Sato, M. (Eds.). (2017). The Routledge handbook of instructed second language acquisition. Routledge.
Nguyen, H. T. (2018). Interactional practices across settings: From classroom role-plays to workplace patient consultations. Applied Linguistics, 39(2), 213–235.
Okada, Y. (2010). Role-play in oral proficiency interviews: Interactive footing and interactional competencies. Journal of Pragmatics, 42(6), 1647–1668.
Plonsky, L., & Zhuang, J. (2019). A meta-analysis of second language pragmatics instruction. In N. Taguchi (Ed.), Routledge handbook of SLA and pragmatics (pp. 287–307). Routledge.
Pomerantz, A., & Heritage, J. (2013). Preference. In J. Sidnell & T. Stivers (Eds.), The handbook of conversation analysis (pp. 210–228). Wiley-Blackwell.
Rockey, C., Tiegs, J., & Fernández, F. (2020). Mobile application use in technology-enhanced DCTs. CALICO, 37(1), 85–108.
Schmidt, R. (1993). Consciousness, learning and interlanguage pragmatics. In G. Kasper & S. Blum-Kulka (Eds.), Interlanguage pragmatics (pp. 21–42). Oxford University Press.
Stokoe, E. (2013). The (in)authenticity of simulated talk: Comparing role-played and actual interaction and the implications for communication training. Research on Language and Social Interaction, 46(2), 165–185.
Swain, M., & Lapkin, S. (1998). Interaction and second language learning: Two adolescent French immersion students working together. Modern Language Journal, 82(1), 320–337.
Sydorenko, T., Jones, Z. W., Daurio, P., & Thorne, S. L. (2020). Beyond the curriculum: Extended discourse practice through self-access pragmatics simulations. Language Learning & Technology, 24(2), 48–69.
Sykes, J. M., & González-Lloret, M. (2020). Exploring the interface of interlanguage (L2) pragmatics and digital spaces. CALICO Journal, 37(1), i–xv.
Taguchi, N. (2015). Instructed pragmatics at a glance: Where instructional studies were, are, and should be going. Language Teaching, 48(1), 1–50.
Taguchi, N. (2021). Application of immersive virtual reality (VR) to pragmatics data collection methods: Insights from interviews. CALICO, 38(2), 181–201.
Taguchi, N., Hirschi, K., & Kang, O. (2021). Longitudinal L2 development in the prosodic marking of pragmatic meaning: Prosodic changes in L2 speech acts and individual factors. Studies in Second Language Acquisition, 1–16. Advance online publication.
Taguchi, N., & Kim, Y. (2016). Collaborative dialogue in learning pragmatics: Pragmatic-related episodes as an opportunity for learning request-making. Applied Linguistics, 37(3), 416–437.
Taguchi, N., Li, Q., & Tang, X. (2017). Learning Chinese formulaic expressions in a scenario-based interactive environment. Foreign Language Annals, 50(4), 641–660.
Taguchi, N., & Roever, C. (2017). Second language pragmatics. Oxford University Press.
Takimoto, M. (2020). Investigating the effects of cognitive linguistic approach in developing EFL learners’ pragmatic proficiency. System, 89.
Verschueren, J. (1999). Understanding pragmatics. Oxford University Press.
Youn, S. J. (2020). Pragmatic variables in role-play design for the context validity of assessing interactional competence. Papers in Language Testing and Assessment, 9(1), 95–127.
Cited by (2)
Cited by two other publications
Taguchi, Naoko
2024.
Technology-enhanced language learning and pragmatics: Insights from digital game-based pragmatics instruction.
Language Teaching 57:1
► pp. 57 ff.
Kim, YouJin, Sanghee Kang, Meredith D’Arienzo & Naoko Taguchi
2023.
Comparing traditional and task-based approaches to teaching pragmatics: Task design processes and learning outcomes.
Language Teaching Research
This list is based on CrossRef data as of 22 september 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.