Cover not available
Article published In:
Applied Pragmatics
Vol. 5:1 (2023) ► pp.1–40

Developing an empirically-driven aural multiple-choice DCT for conventional expressions in L2 pragmatics

ORCID logo | Indiana University
ORCID logo | The University of Utah

This paper reports on the development of an aural multiple-choice discourse completion task (aural MC-DCT) for use with EFL learners for large-scale instructional needs assessment. The original version of the aural MC-DCT and an oral DCT were administered to 134 EFL learners from three universities in China. The aural MC-DCT was revised based on the results of the initial administration and completed by 251 EFL students and 89 native speakers of American English in a second administration. Both versions of the task used learner-generated options. The first version used digital files of learner production from an earlier oral DCT taken by ESL learners; the second version re-recorded the learner-produced options using native speakers of American English. The results from the second version show that the aural MC-DCT is a feasible format for exploring learner knowledge of conventional expressions by comparing learner selections to production on the oral DCT, which has been established as a reliable measure of L2 English conventional expressions. The aural MC-DCT provides teachers with a practical alternative to the transcription and analysis required by the oral DCT, allowing them to easily assess knowledge of target conventional expressions and preferred alternatives in order to facilitate decisions about instruction.

Keywords: DCT, aural multiple-choice DCT, oral DCTs, conventional expressions, pragmatic routines, aural tasks
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1Assessing L2 speech acts using MC-DCTs
    • 2.2Assessing L2 formulaic language
    • 2.3The aural MC-DCT in pragmatics instruction
    • 2.4Methodological considerations: Authenticity and practicality
  • 3.Instrument development
    • 3.1Adapting the oral DCT
    • 3.2Developing the aural MC-DCT
  • 4.Study 1: Oral DCT with pilot MC-DCT
    • 4.1Method
    • 4.2Analysis
    • 4.3Pilot results
  • 5.Study 2: Revised aural MC-DCT
    • 5.1Revision of the aural MC-DCT
    • 5.2Method
      • 5.2.1Participants
      • 5.2.2Procedure
    • 5.3Analysis
    • 5.4Results
      • 5.4.1Learner performance matches
      • 5.4.2Interlanguage production informs good choices
      • 5.4.3Competitive distractors
  • 6.Discussion
    • 6.1Reliability
    • 6.2Pedagogical implications
  • 7.Conclusion
  • Acknowledgements
  • Notes
  • References
Published online: 3 March 2022
DOI logo
https://doi.org/10.1075/ap.20020.bar
References
Alcón-Soler, E., & Sánchez-Hernández, A.
(2017) Learning pragmatic routines during study abroad: A focus on type of routine. ATLANTIS: Journal of the Spanish Association of Anglo-American Studies, 39 (2), 191–210. DOI logoGoogle Scholar
Bardovi-Harlig, K.
(2009) Conventional expressions as a pragmalinguistic resource: Recognition and production of conventional expressions in L2 pragmatics. Language Learning, 59 (4), 755–795. DOI logoGoogle Scholar
(2010) Recognition of conventional expressions in L2 pragmatics. Pragmatics and Language Learning, 12 1, 141–162.Google Scholar
(2012) Formulas, routines, and conventional expressions in pragmatics research. ARAL, 32 1, 206–227. DOI logoGoogle Scholar
(2014) Awareness of meaning of conventional expressions in second language pragmatics. Language Awareness, 23 (1–2), 41–56. DOI logoGoogle Scholar
(2019) Routines in L2 pragmatics research. In N. Taguchi (Ed.), Handbook of SLA and pragmatics (pp. 47–62). Routledge. DOI logoGoogle Scholar
Bardovi-Harlig, K., Bastos, M.-T., Burghardt, B., Chappetto, E., Nickels, E., & Rose, M.
(2010) The use of conventional expressions and utterance length in L2 pragmatics. Pragmatics and Language Learning, 12 1, 163–186.Google Scholar
Bardovi-Harlig, K., & Bastos, M.-T.
(2011) Proficiency, length of stay, and intensity of interaction and the acquisition of conventional expressions in L2 pragmatics. Intercultural Pragmatics, 8(3), 347–384. DOI logoGoogle Scholar
Bardovi-Harlig, K., & Dörnyei, Z.
(1998) Do language learners recognize pragmatic violations? Pragmatic versus grammatical awareness in instructed L2 learning. TESOL Quarterly, 32 (2), 233–259. DOI logoGoogle Scholar
Bardovi-Harlig, K., Mossman, S., & Su, Y.
(2017) The effect of corpus-based instruction on pragmatic routines. Language Learning & Technology, 21 (3), 76–103. 10125/44622Google Scholar
Bardovi-Harlig, K., Mossman, S., & Vellenga, H. E.
(2015a) Developing corpus-based materials to teach pragmatic routines. TESOL Journal, 6 (3), 499–526. DOI logoGoogle Scholar
(2015b) The effect of instruction on pragmatic routines in academic discussion. Language Teaching Research, 19 (3), 324–350. DOI logoGoogle Scholar
Bardovi-Harlig, K., & Stringer, D.
(2017) Unconventional expressions: Productive syntax in the L2 acquisition of formulaic language. Second Language Research, 33 (1), 61–90. DOI logoGoogle Scholar
Bardovi-Harlig, K., & Su, Y.
(2018) The acquisition of conventional expressions as a pragmalinguistic resource in Chinese as a foreign language. Modern Language Journal, 102 (4), 732–757. DOI logoGoogle Scholar
(2021) The effect of learning environment on the selection of conventional expressions on an aural multiple-choice DCT. TESL-EJ, 25 (1). [URL]
(2019) Using corpus-linguistic methods to track longitudinal development: Routine apologies in the study abroad context. Journal of Pragmatics, 146 1, 87–105. DOI logoGoogle Scholar
Brown, J. D.
(2001) Pragmatics tests: Different purposes, different tests. In K. R. Rose & G. Kasper (Eds.), Pragmatics in language teaching (pp. 301–326). Cambridge University Press. DOI logoGoogle Scholar
Carr, N.
(2011) Designing and analyzing language tests. Oxford University Press.Google Scholar
Enochs, K., & Yoshitake-Strain, S.
(1996) Self-assessment and role plays for evaluating appropriateness in speech act realizations. ICU Language Research Bulletin, 11 1, 57–76.Google Scholar
Erman, B., & Warren, B.
(2000) The idiom principle and the open choice principle. Text & Talk, 20 (1), 20 1, 29–62. DOI logoGoogle Scholar
ETS
(2019) Examinee handbook: TOEIC listening and reading. Educational Testing Service.Google Scholar
Fuchs, R.
(2017) Do women (still) use More intensifiers than men? Recent change in the sociolinguistics of intensifiers in British English. International Journal of Corpus Linguistics, 22 (3), 345–374. DOI logoGoogle Scholar
Hudson, T., Detmer, E., & Brown, J. D.
(1992) A framework for testing cross-cultural pragmatics (Technical Report 2). University of Hawai‘i, Second Language Teaching and Curriculum Center.Google Scholar
(1995) Developing prototypic measures of cross-cultural pragmatics (Technical Report 7). University of Hawai‘i, Second Language Teaching and Curriculum Center.Google Scholar
Köylü, Y.
(2018) Comprehension of conversational implicatures in L2 English. Intercultural Pragmatics, 15 (3), 373–408. DOI logoGoogle Scholar
Liu, J.
(2007) Developing a pragmatics test for Chinese EFL learners. Language Testing, 24 (3), 391–415. DOI logoGoogle Scholar
Osuka, N.
(2017) Development of pragmatic routines by Japanese learners in a study abroad context. In I. Kecskes & S. Assimakopoulos (Eds.), Current issues in intercultural pragmatics (pp. 275–296). Benjamins. DOI logoGoogle Scholar
Papageorgiou, S., Wu, S., Hsieh, C., Tannenbaum, R. J., &. Cheng, M.
(2019) Mapping the TOEFL iBT® test scores to China’s standards of English language ability: Implications for score interpretation and use. (Research Report No. TOEFL-RR-89). Educational Testing Service. DOI logoGoogle Scholar
Roever, C.
(2005) Testing ESL Pragmatics: Development and validation of a web-based assessment battery. Peter Lang. DOI logoGoogle Scholar
(2012) What learners get for free: learning of routine formulae in ESL and EFL environments. ELT Journal, 66 (1), 10–21. DOI logoGoogle Scholar
Roever, C., Wang, S., & Brophy, S.
(2014) Learner background factors and learning of second language pragmatics. International Review of Applied Linguistics, 52 1, 377–401. DOI logoGoogle Scholar
Shin, S-Y., Lee, S. & Lidster, R.
(2021) Examining the effects of different English speech varieties on an L2 academic listening comprehension test at the item level. Language Testing, 38 (4), 580–601. DOI logoGoogle Scholar
Swain, M.
(1995) Three functions of output in second language learning In G. Cook & B. Seidlhofer (Eds.), Principle and practice in applied linguistics: Studies in the Honour of H. G. Widdowson (pp. 371–391). Oxford University Press.Google Scholar
Tagliamonte, S., & Roberts, C.
(2005) So weird; so cool; so innovative: The use of intensifiers in the television series Friends . American Speech, 80 (3), 280–300. DOI logoGoogle Scholar
Taguchi, N.
(2009) Corpus-informed assessment of comprehension of conversational implicatures in L2 English. TESOL Quarterly, 43 (4), 738–749. DOI logoGoogle Scholar
Taguchi, N., Li, S., & Xiao, F.
(2013) Production of formulaic expressions in L2 Chinese: A developmental investigation in a study abroad context. Chinese as a Second Language Research, 2 (1), 23–58. DOI logoGoogle Scholar
Taguchi, N., & Roever, C.
(2017) Second language pragmatics. Oxford University Press.Google Scholar
Teng, C., & Fei, F.
(2013) A consciousness-raising approach to pragmatics teaching: Web-based tasks for training study-abroad students. Journal of Technology and Chinese Language Teaching, 4 (1), 50–63.Google Scholar
Yamashita, S. O.
(1996) Six measures of JSL pragmatics. University of Hawaii at Manoa.Google Scholar
Cited by

Cited by 1 other publications

Bardovi-Harlig, Kathleen, Jesús Izquierdo & Yunwen Su
2022. Recognition of conventional expressions by EFL learners in Mexico and China. System 110 ► pp. 102918 ff. DOI logo

This list is based on CrossRef data as of 16 january 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.