Chapter published in:
Task-Based Approaches to Teaching and Assessing Pragmatics
Edited by Naoko Taguchi and YouJin Kim
[Task-Based Language Teaching 10] 2018
► pp. 160–190

Chapter 7
Independently measuring cognitive complexity in task design for interlanguage pragmatics development

| University of Barcelona
| University of Barcelona | International University of Catalonia
This study examines the effects of task design on interlanguage pragmatics, bringing together task complexity and L2 pragmatic outcomes measured holistically. Fifteen expert judges were asked independently to assess task complexity and to evaluate the pragmatic performance of 60 EFL learners who had written a response to four email messages at 4 different levels of complexity. On the basis of needs analysis, complexity had been manipulated by along the following parameters: +/−frequency of input, +/−familiarity with interlocutor, +/−intentional, +/−causal reasoning, +/−dependency of steps, +/−number of elements, and +/−dual task. In order to validate complexity independently, two techniques were applied. On the one hand, a subjective perception questionnaire evaluating mental effort and difficulty was answered by 15 experienced teachers. On the other hand, subjective judgments on a pragmatic scale by the same teachers followed by retrospective protocol analysis were used in the classification of tasks from simple to complex. Pragmatic outcomes were assessed on a holistic rating scale of pragmatic performance. While descriptive statistical results pointed in the direction of predictions, inferential statistics only confirm a difference between the first level of complexity and the other three. As for the effect of sequence, expert judges did not find the outcomes of the two sequences to differ. Results are discussed in light of task performance, task sequencing, and interlanguage pragmatics development.
Article outline
  • Introduction
  • Background
    • TBLT and pragmatics
    • The independent measurement of task complexity
    • Task grading and sequencing
    • Measuring pragmatic competence
    • Research questions
  • Methods
    • Participants
    • Tasks
    • Expert judgments via online questionnaires
    • Operationalization of task complexity and task sequencing
      • Input frequency
      • Causal reasoning
      • Intentional reasoning
      • Number of elements
    • Operationalization of task sequencing
    • Specific data collection procedures
    • Statistical tools and procedures
  • Results
    • Research Question 1
    • Research Question 2
  • Discussion
  • Conclusion
  • References
Published online: 15 August 2018
https://doi.org/10.1075/tblt.10.07gil
References
Alcón, E.
(2012) Teachability and bilingualism in effects on third language learners’ pragmatic knowledge. Intercultural Pragmatics, 9(4), 511–541.Google Scholar
(2013a) Mitigating e-mail requests in teenagers’ first and second language academic cyber-consultation. Multilingua, 32, 779–799.Google Scholar
Alcón-Soler, E.
(2015) Pragmatic learning and study abroad: Effects of instruction and length of stay. System, 48, 62–74.CrossrefGoogle Scholar
Alcón, E., & Martínez-Flor, A.
(Eds.) (2008) Investigating pragmatics in foreign language learning, teaching and testing. Clevedon: Multilingual Matters.CrossrefGoogle Scholar
Baralt, M.
(2013) The impact of cognitive complexity on feedback efficacy during online versus face-to-face interactive tasks. Studies in Second Language Acquisition, 35(4), 689–725. CrossrefGoogle Scholar
Barat, M.
(2014) Task complexity and task sequencing in traditional versus online language classes. In M. Baralt, R. Gilabert, P. Robinson (Eds). Task sequencing and second language learning (pp. 52–122). New York: BloomsburyGoogle Scholar
Baralt, M., Gilabert, R., Robinson, P.
(2014) Task sequencing and second language learning. New York: Bloomsbury.Google Scholar
Baralt, M., Gilabert, R., & Robinson, P.
(Eds.) (2014) Task sequencing and instructed second language learning. London: Bloomsbury.Google Scholar
Bardovi-Harlig, K., & Hartford, B. S.
(2005) Institutional discourse and interlanguage pragmatics. In K. Bardovi-Harlig & B. S. Hartford (Eds.), Interlanguage pragmatics: Exploring institutional talk (pp. 7–36). Mahwah, NJ: Erlbaum.Google Scholar
Bardovi-Harlig, K.
(2013) Developing L2 pragmatics. Language Learning, 63(1), 68–86. CrossrefGoogle Scholar
Barron, A.
(2012) Interlanguage pragmatics: From use to acquisition to second language pedagogy. Language Teaching, 45(1), 44–63. CrossrefGoogle Scholar
Barron, A., Gu, Y., & Steen, G.
(Eds.) (2016) Routledge handbook of pragmatics. London: Routledge.Google Scholar
Biesenbach-Lucas, S.
(2006) Making requests in email: Do cyber-consultations entail directness? Toward conventions in a new medium. Pragmatics and language learning, 11, 82–107.Google Scholar
(2007) Students writing emails to faculty: An examination of e-politeness among native and non-native speakers of English. Language Learning and Technology, 11(2), 59–81.Google Scholar
Chen, C-F. E.
(2001) Making e-mail requests to professors: Taiwanese vs. American students. Paper presented at the Annual Meeting of the American Association for Applied Linguistics in St. Louis, February 2001. (ERIC Document Reproduction Service No. ED 461 299).
Council of Europe
(2001) Common European framework of reference for languages: Learning, teaching, assessment. Cambridge: Press Syndicate of the University of Cambridge.Google Scholar
Economidou-Kogetsidis, M.
(2011) “Please answer me as soon as possible”: Pragmatic failure in non-native speakers’e-mail requests to faculty. Journal of Pragmatics, 43(13), 3193–3215.CrossrefGoogle Scholar
Economidou-kogetsidis, M.
(2015) Teaching E-mail Politeness in the EFL/ESL Classroom. ELT journal, 69(4), 415–424. Crossref.Google Scholar
Economidou-Kogetsidis, M.
(2015) Teaching email politeness in the EFL/ESL classroom. ELT Journal, 69, 415–424.CrossrefGoogle Scholar
Ellis, R.
(2003) Task-based language teaching and learning. Oxford: Oxford University Press.Google Scholar
Færch, C., & Kasper, G.
(1989) Internal and external modification in interlanguage request realization. In S. Blum-Kulka, J. House, & G. Kasper (Eds.), Cross-cultural pragmatics (pp. 221–247). Norwood, NJ: Ablex.Google Scholar
Félix-Brasdefer, J. C.
(2012) E-mail requests to faculty. In M. Economidou-Kogetsidis, & H. Woodfield (Eds), Interlanguage Request Modification (pp.217–287). Amsterdam: John Benjamins.CrossrefGoogle Scholar
Firth, A., & Wagner, J.
(2007) Second/Foreign language learning as a social accomplishment: elaborations on a reconceptualized SLA. The Modern Language Journal, 91(1), 800–819. CrossrefGoogle Scholar
Gilabert, R.
(2005) Evaluating the use of multiple sources and methods in needs analysis: A case study of journalists in the autonomous community of Catalonia (Spain). In M. H. Long (Ed.), Second language needs analysis (pp. 182–99). Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Gilabert, R., & Barón, J.
(2013) The impact of increasing task complexity on L2 pragmatic moves. In K. McDonough & A. Mackey (Eds.), Second language interaction in diverse educational contexts (pp. 45–69). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Gilabert, R., & Levkina, M.
(2018) The impact of task complexity and task sequencing on the development of pragmatic moves. Manuscript in preparation.Google Scholar
Gilabert, R., Barón, J., & Levkina, M.
(2011) Manipulating task complexity across task types and modes. In P. Robinson (Ed.), Second language task complexity: Researching the cognition hypothesis of language learning and performance (pp. 105–140). Amsterdam: John Benjamins.CrossrefGoogle Scholar
House, J.
(1996) Developing pragmatic fluency in English as a foreign language: routines and metapragmatic awareness. Studies in Second Language Acquisition, 18(2), 225–252. CrossrefGoogle Scholar
Hudson, T., Detmer, E., & Brown, J. D.
(1995) Developing prototypic measures of cross-cultural pragmatics. Honolulu, HI: Second Language Teaching and Curriculum Center, University of Hawaii at Manoa.Google Scholar
Jackson, D., & Suethanapornkul, S.
(2013) The cognition hypothesis: A synthesis and meta-analysis of research on second language task complexity. Language Learning, 63(2), 330–367.CrossrefGoogle Scholar
Kasper, G.
(1998) Interlanguage pragmatics. In H. Byrnes (Ed.), Learning Foreign and Second Languages: Perspectives in Research and Scholarship (pp. 183–208). New York, NY: The Modern Language Association of America.Google Scholar
Kasper, G., & Rose, K.
(2002) Pragmatic development in a second language. Malden, MA: Blackwell.Google Scholar
Kasper, G., & Schmidt, R.
(1996) Developmental issues in interlanguage pragmatics. Studies in Second Language Acquisition, 18, 149–69. CrossrefGoogle Scholar
Kasper, G., & Blum-Kulka, S.
(1993) Interlanguage pragmatics: An introduction. In G. Kasper & S. Blum-Kulka (Eds.), Interlanguage pragmatics (pp. 3–17). Oxford: Oxford University Press.Google Scholar
Kasper, G.
(2010) Interlanguage pragmatics. Variation and change: Pragmatic perspectives, 6, 141–154.CrossrefGoogle Scholar
Kim, Y., & Taguchi, N.
(2015) Promoting task-based pragmatics instruction in EFL classroom contexts: The role of task complexity. The Modern Language Journal, 99(4), 656–677. CrossrefGoogle Scholar
(2016) Learner–learner interaction during collaborative pragmatic tasks: The role of cognitive and pragmatic task demands. Foreign Language Annals, 49(1), 42–57.CrossrefGoogle Scholar
Kim, Y., & Tracy-Ventura, N.
(2011) Task complexity, language anxiety, and the development of the simple past. In P. Robinson (Ed.), Second language task complexity: Researching the cognition hypothesis of language learning and performance (pp. 287–306). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Koike, D. A., & Pearson, L.
(2005) The effect of instruction and feedback in the development of pragmatic competence. System, 33(3), 481–501. CrossrefGoogle Scholar
Levkina, M. & Gilabert, R.
(2014) Task Sequencing in the L2 Development of Spatial Expressions. In M. Baralt, R. Gilabert, & P. Robinson (Eds). Task sequencing and instructed second language learning (pp. 37–71). New York: Bloomsbury.Google Scholar
Liu, J.
(2006) Measuring interlanguage pragmatic knowledge of EFL learners. Frankfurt am Main: Peter Lang.Google Scholar
(2011) Testing interlanguage pragmatics knowledge. In Trosborg, A. (Ed.), Pragmatics across language and cultures (pp. 467–488). Berlin: Walter de Gruyter.Google Scholar
(2006) Assessing EFL learners’ interlanguage pragmatic knowledge: Implications for testers and teachers. Reflections on English Language Teaching, 5(1), 1–22.Google Scholar
Long, M. H.
(2005) Second language needs analysis. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Long, M.
(2015a) Second language acquisition and task-based language teaching. Malden, MA: Wiley-Blackwell.Google Scholar
Malika, A., & Levkina, M.
(2012) Measuring task complexity: Does EFL proficiency matter? In A. Shebadeh & C. Combee (Eds.), Task-based language teaching in foreign language contexts (pp.43–66). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Martínez-Flor, A., & Usó-Juan, E.
(2010) Pragmatics and speech act performance. In A. Marínez-Flor & E. Usó-Juan (Eds.), Speech act performance: Theoretical, empirical and methodological issues (pp. 3–20). Amsterdam: John Benjamins. CrossrefGoogle Scholar
Michel, M. C.
(2011) Effects of Task Complexity and Interaction on L2-Performance. In: Robinson, P. (ed.). Second Language Task Complexity: Researching the Cognition Hypothesis of Language Learning and Performance (pp. 141–174). John Benjamins Publishing Company, Amsterdam/Philadelphia.CrossrefGoogle Scholar
Michel, M., Gilabert, R., & Révész, A.
(2018) Eye-tracking and stimulated recall as means of investigating cognitive load and processes during L2 tasks. Manuscript in preparation.Google Scholar
Norris, J. M.
(2010) Understanding instructed SLA: Constructs, contexts, and consequences. Plenary address delivered at the annual conference of the European Second Language Association (EUROSLA), Reggio Emilia, Italy.Google Scholar
Norris, J. M., & Ortega, L.
(2003) Defining and measuring SLA. In C. J. Doughty & M. H. Long (Eds.), Handbook of second language acquisition (pp. 716–761). Malden, MA: Blackwell. CrossrefGoogle Scholar
Plonsky, L., & Kim, Y.
(2016) Task-based learner production: A substantive and methodological review. Annual Review of Applied Linguistics, 36, 73–97.CrossrefGoogle Scholar
Prabhu, N. S.
(1987) Second language pedagogy. Oxford: Oxford University Press.Google Scholar
Révész, A.
(2014) Towards a fuller assessment of cognitive models of task-based learning: Investigating task-generated cognitive demands and processes. Applied Linguistics, 35(1), 87–92. Crossref.Google Scholar
(2011) Task complexity, focus on L2 constructions, and individual differences: A classroom-based study. The Modern Language Journal, 95(1), 162–181. CrossrefGoogle Scholar
Révesz, A., Michel, M. & Gilabert, R.
(2016) Measuring cognitive task demands using dual task methodology, subjective self-ratings, and expert judgments: A validation study. Studies in Second Language Acquisition, 38, 703–737.CrossrefGoogle Scholar
Révész, A., & Gurzynski-Weiss, L.
(2016) Teachers’ perspectives on second language task difficulty: insights from think-alouds and eye tracking. Annual Review of Applied Linguistics 36 182–204. Crossref.Google Scholar
Révész, A., Michel, M. C., & Gilabert, R.
(2016) Measuring cognitive task demands using dual task methodology, subjective self-ratings, and expert judgments: a validation study. Studies in Second Language Acquisition, 38(4), 703–737. CrossrefGoogle Scholar
Révész, A., Sachs, R., & Hama, M.
(2014) The effects of task complexity and input frequency on the acquisition of the past counterfactual construction through recasts. Language Learning, 64, 615–650.CrossrefGoogle Scholar
Robinson, P.
(2001) Task complexity, task difficulty, and task production: Exploring interactions in a componential framework. Applied Linguistics, 22(1), 27–57. CrossrefGoogle Scholar
(2005) Cognitive complexity and task sequencing: Studies in a componential framework for second language task design. International Review of Applied Linguistics in Language Teaching, 43(1), 1–32. CrossrefGoogle Scholar
(2011) Second language task complexity: Researching the cognition hypothesis of language learning and performance. Amsterdam: John Benjamins. CrossrefGoogle Scholar
Robinson, P., & Gilabert, R.
(2007) Task complexity, the cognition hypothesis and second language learning and performance. IRAL-International Review of Applied Linguistics in Language Teaching, 45(3), 161–176. CrossrefGoogle Scholar
Robinson, P.
(2007) Task complexity, theory of mind, and intentional reasoning: Effects on speech production, interaction, uptake and perceptions of task difficulty. International Review of Applied Linguistics, 45, 195–215.CrossrefGoogle Scholar
Rose, K.
(2001) An exploratory cross-sectional study of interlanguage pragmatic development. Studies in Second Language Acquisition, 22(1), 27–67.Google Scholar
Skehan, P.
(1998) A cognitive approach to language learning. Oxford: Oxford University Press.Google Scholar
Sasayama, S., Malicka, A., & Norris, J.
(2015) Primary challenges in cognitive task complexity research: Results of a comprehensive research synthesis. Paper presented at the 2015 Task-Based Language Teaching Conference in Leuven, Belgium.
Sasayama, S.
(2016) Is a ‘complex’ task really complex? Validating the assumption of cognitive task complexity. Modern Language Journal, 100(1), 231–254. Crossref.Google Scholar
Safont, P.
(2005) Third language learners: pragmatic production and awareness (Vol. 12). Clevedon, England: Multilingual Matters.Google Scholar
Sasayama, S.
(2013) Is a “complex” task really complex? Measuring task complexity independently from linguistic production. Paper presented at the 5th Biennial International Conference on Task-Based Language Teaching, Banff, Alberta, Canada.
Sweller, J., Van Merriënboer, J. J. G., & Paas, F.
(1998) Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–295 CrossrefGoogle Scholar
Taguchi, N.
(2007) Task difficulty in oral speech act production. Applied linguistics, 28(1), 113–135. CrossrefGoogle Scholar
(2015) Instructed pragmatics at a glance: Where instructional studies were, are, and should be going. State-of-the-art article. Language Teaching, 48(1), 1–50. CrossrefGoogle Scholar
Tagouchi, N. & Roever, C.
(2017) Second language pragmatics. Oxford: Oxford University Press.Google Scholar
Takahashi, S.
(2001) The role of input enhancement in developing pragmatic competence. Pragmatics in language teaching, 171–199.CrossrefGoogle Scholar
Takimoto, M.
(2008) The effects of deductive and inductive instruction on the development of language learners’ pragmatic competence. The Modern Language Journal, 92(3), 369–386. CrossrefGoogle Scholar
Walters, F. S.
(2007) A conversation-analytic hermeneutic rating protocol to assess L2 oral pragmatic competence. Language Testing, 24(2), 155–183. CrossrefGoogle Scholar
Woodfield, H., & Economidou-Kogetsidis, M.
(2010) “I just need more time”: a study of native and non-native students’ requests to faculty for late submission. Multilingua, 1, 77–118.CrossrefGoogle Scholar
Cited by

Cited by 3 other publications

Barón, Júlia, M. Luz Celaya & Mayya Levkina
2020. Learning pragmatics through tasks. Applied Pragmatics 2:1 ► pp. 1 ff. Crossref logo
González-Lloret, Marta & Lourdes Ortega
2018.  In Task-Based Approaches to Teaching and Assessing Pragmatics [Task-Based Language Teaching, 10], ► pp. 192 ff. Crossref logo
Hayes, Heather, Marylee Demeter, John G. Morris & Goran Trajkovski
2021. Transparency of cognitive complexity in performance assessments: A validity study. The Journal of Competency-Based Education 6:2 Crossref logo

This list is based on CrossRef data as of 31 march 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.