A Tale of four measures of pragmatic knowledge in an EFL institutional context

Rasoul Mohammad Hosseinpur, Reza Bagheri Nevisi and Abdolreza Lowni

Abstract

The upsurge of interest in L2 pragmatics studies has coincided with a growing interest in pragmatic assessment. Employing the most efficient measure of pragmatics has led many researchers to examine the existing measures to pinpoint the most useful ones. This study was an attempt to compare and contrast Written Discourse Completion Task (WDCT), Oral Discourse Completion Task (ODCT), and Role-play with Natural methodology in an EFL institutional context to see which measure approximated Natural methodology. To this end, data (requests) were collected from 27 intermediate–level Iranian EFL learners in a natural classroom institutional context over 15 weeks, and then the WDCTs, ODCTs, and Role-plays with the same contextual features were selected to elicit the intended data. The participants’ requests were transcribed and analyzed in terms of Schauer’s (2009)Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar request head act strategy taxonomy and its internal and external modification devices. The results of Binominal tests indicated that, in spite of some minor similarities, none of the elicitation measures could approximate the natural data. The participants’ employment of direct, non-conventionally indirect request strategies, and internal and external modification devices were more conspicuous in the WDCTs, ODCTs, and Role-plays than those in the Natural methodology. The study implies that data collection methods should be selected based on researchers’ objectives and research questions.

Keywords:
Publication history
Table of contents

1.Introduction

Interlanguage pragmatics (ILP), as an inseparable aspect of learning or teaching a second/foreign language, has received a lot of attention over the past few decades. Different studies have employed various data elicitation methods to investigate how teaching appropriate use of language is to be approached in the most effective way (Ishihara 2010Ishihara, Noriko 2010 “Instructional Pragmatics: Bridging Teaching, Research, and Teacher Education.” Language and Linguistics Compass, 4(10): 938–953. CrossrefGoogle Scholar; Martinez-Flor and Uso-Juan 2011Martínez-Flor, Alicia and Esther Usó-Juan 2011 “Research Methodologies in Pragmatics: Eliciting Refusals to Requests.” ELIA, 11: 47–87.Google Scholar; Norris and Ortega 2000Norris, John. and Lourdes Ortega 2000 “Effectiveness of L2 Instruction: A Research Synthesis and Quantitative Meta-Analysis.” Language Learning, 50 (3): 417–528. CrossrefGoogle Scholar; Takahashi 2010Takahashi, Satomi 2010 “Assessing Learnability in Second Language Pragmatics.” In, Pragmatics across Languages and Cultures, ed. by Anna Trosborg, 391–423. Berlin, Germany: De Gruyter Mouton.Google Scholar).

Because of the intricacies inherent in pragmatics itself, the existence of several intervening and overlapping contextual factors, and the direct impact of the data collection method on the final performance of language users, the development of efficient and reliable methods to tap learners’ pragmatic competence seems to be crucial (Uso-Juan and Martinez-Flor 2014Uso-Juan, Esther, and Alicia Martinez-Flor 2014 “Reorienting the Assessment of the Conventional Expressions of Complaining and Apologizing: From Single Response to Interactive DCTs.” Iranian Journal of Language Testing, 4(1): 113–136.Google Scholar). As Bardovi-Harlig (2018Bardovi-Harlig, Kathleen 2018 “Matching Modality in L2 Pragmatics Research Design”. System: 75, 13–22. CrossrefGoogle Scholar, 13) put it, ILP “is a field of inquiry that has always reflected on its methodology”. Accordingly, quite a few studies have explored the effectiveness of various means of measuring ILP competence (Bardovi-Harlig and Shin 2014Bardovi-Harlig, Kathleen, and Sun-Young Shin 2014 “Expanding Traditional Testing Measures with Tasks from L2 Pragmatics Research.” Iranian Journal of Language Testing, 4 (1): 26–49.Google Scholar; Eslami-Rasekh and Mirzaei 2014Eslami, Zohreh. R., and Azizullah Mirzaei 2014 “Speech Act Data Collection in a Non-Western Context: Oral and Written DCTs in the Persian Language.” Iranian Journal of Language Testing, 4(1): 137–154.Google Scholar; Golato 2003Golato, Andrea 2003 “Studying Compliment Responses: A Comparison of DCTs and Recordings of Naturally Occurring Talk.” Applied Linguistics, 24(1): 90–121. CrossrefGoogle Scholar; Taguchi 2018Taguchi, Naoko 2018 “Data Collection and Analysis in Developmental L2 Pragmatics Research: Discourse Completion Test, Role Play, and Naturalistic Recording.” In Critical Reflections on Data in Second Language Acquisition, ed. by Aarnes Gudmestad and Amanda Edmonds, 7–32. Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar; Taguchi and Roever 2017Taguchi, Naoko, and Carsten Roever 2017Second Language Pragmatics. Oxford: Oxford University Press.Google Scholar; Uso-Juan and Martinez-Flor 2014Uso-Juan, Esther, and Alicia Martinez-Flor 2014 “Reorienting the Assessment of the Conventional Expressions of Complaining and Apologizing: From Single Response to Interactive DCTs.” Iranian Journal of Language Testing, 4(1): 113–136.Google Scholar; Yuan 2001Yuan, Y. 2001 “An Inquiry into Empirical Pragmatics Data Gathering Methods: Written DCTs, Oral DCTs, and Natural Conversations.” Journal of pragmatics, 33: 271–92. CrossrefGoogle Scholar).

However, few studies were comprehensive enough to include several pragmatic measures in one study in an EFL context. Moreover, most of these studies utilized every-day conversational interactions or conversational discourse, as opposed to institutional discourse, to conduct their investigations. Bardovi-Harlig and Hartford (2005Bardovi-Harlig, Kathleen, and Beverly Hartford 2005Institutional Discourse and Interlanguage Pragmatics Research. In Interlanguage pragmatics: exploring institutional talk, ed. by Kathleen Bardovi-Harlig, and Beverly Hartford, 7–36. Mahwah, NJ: Lawrence Erlbaum. CrossrefGoogle Scholar, 7–8) defined institutional discourse as:

Institutional talk may be understood as talk between an institutional representative and a client (e.g., a faculty advisor and a graduate student, or an interviewer at a job agency and an applicant) or between members of the same institution (also called workplace talk, such as talk between a nursing supervisor and a nurse, or among hotel or factory employees).

They maintained that institutional talks enjoy the three characteristics of comparability (there is control over various variables to make comparison or analogy easier), interactivity (there is turn-taking pattern in the process of interaction), and consequentiality (there is a real-life goal for communication). Conversational discourse, on the other hand, enjoys the two characteristics of interactivity and consequentiality. Ellis (2008)Ellis, Rod 2008The Study of Second Language Acquisition (2nd ed.). Oxford: Oxford University Press.Google Scholar also described institutional talks as talks that mostly transpire in an institutional context such as a court of law, even a surgeon’s room, or an educational context like a classroom.

Drawing upon request speech act, the present study was set up to shed more light on the similarities and differences between three frequently-used pragmatic measures: Written discourse completion task (WDCT), oral discourse completion task (ODCT), Role-play and Natural data produced as institutional discourse in an EFL language classroom context.

2.Literature review

2.1Pragmatic measurement

Various approaches have been developed to assess pragmatic awareness and production of language learners. Diaries, verbal protocols, and rank ordering tasks can be regarded as the major data collection instruments of pragmatic awareness (Kasper and Roever 2005Kasper, Gabriele and Carsten Roever 2005 ”Pragmatics in Second Language Learning.” In Handbook of Research in Second Language Teaching and Learning, ed. by Eli Hinkel, 317–334. Mahwah/New Jersey: Lawrence Erlbaum Associates.Google Scholar). The data collection tools used in measuring pragmatic production can be further subcategorized into two types: Naturally-occurring data collection procedures and intended data elicitation techniques (Félix-Brasdefer 2007 2007 “Natural Speech vs. Elicited Data: A Comparison of Natural and Role Play Requests in Mexican Spanish.” Spanish in Context, 4(2): 159–185. CrossrefGoogle Scholar).

2.2Production measures of pragmatics

2.2.1Naturally-occurring data

To obtain naturally-occurring data, the researcher observes the real interactions of the participants in the real contexts and records field notes online or with audio-visual equipment for later in-depth transcription (Taguchi 2018Taguchi, Naoko 2018 “Data Collection and Analysis in Developmental L2 Pragmatics Research: Discourse Completion Test, Role Play, and Naturalistic Recording.” In Critical Reflections on Data in Second Language Acquisition, ed. by Aarnes Gudmestad and Amanda Edmonds, 7–32. Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar). The potential strength of this method lies in the naturalness of the collected data and its being reflective of the true features of the real-life interactions (Schauer 2009Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar). However, the presence of the observer or audio-visual equipment can diminish the authenticity of the data and give rise to what Labov (1972)Labov, William 1972Sociolinguistic Patterns. Philadelphia: University of Philadelphia.Google Scholar dubbed the “observer’s paradox” (House 2018House, Juliane 2018 “Authentic vs. Elicited Data and Qualitative vs. Quantitative Research Methods in Pragmatics: Overcoming Two Non-Fruitful Dichotomies.” System, 75:4–12. CrossrefGoogle Scholar). Lack of control by the researcher over the variables under investigation (Bardovi-Harlig 2018Bardovi-Harlig, Kathleen 2018 “Matching Modality in L2 Pragmatics Research Design”. System: 75, 13–22. CrossrefGoogle Scholar) and the effect of extraneous variables on the ultimate results including social relationship, power, distance, status, gender, and age differences are the main limitations of this approach (Taguchi and Roever 2017Taguchi, Naoko, and Carsten Roever 2017Second Language Pragmatics. Oxford: Oxford University Press.Google Scholar).

2.2.2Written discourse completion task

In WDCT, respondents are presented the description of a situation or a scenario and are required to provide an appropriate answer for the described situation (Appendix A). This instrument is rather popular among pragmatic researchers because of some inherent features like ease of administration, possibility of eliciting a large amount of data within a relatively short span of time, and researchers’ control over the variables of the study (Bardovi-Harlig 2018Bardovi-Harlig, Kathleen 2018 “Matching Modality in L2 Pragmatics Research Design”. System: 75, 13–22. CrossrefGoogle Scholar; Golato 2003Golato, Andrea 2003 “Studying Compliment Responses: A Comparison of DCTs and Recordings of Naturally Occurring Talk.” Applied Linguistics, 24(1): 90–121. CrossrefGoogle Scholar). However, this data collection procedure has its own limitations. Authenticity and interactivity of the elicited data through WDCTs are the most important criticisms leveled against this data collection procedure. Some researchers hold the opinion that respondents’ written responses fail to truly reflect the authenticity of oral interactions and the written collected data are different from natural interactions in terms of length, number of repetitions, inversions, and omissions (Bardovi-Harlig 2018Bardovi-Harlig, Kathleen 2018 “Matching Modality in L2 Pragmatics Research Design”. System: 75, 13–22. CrossrefGoogle Scholar; Yuan 2001Yuan, Y. 2001 “An Inquiry into Empirical Pragmatics Data Gathering Methods: Written DCTs, Oral DCTs, and Natural Conversations.” Journal of pragmatics, 33: 271–92. CrossrefGoogle Scholar).

2.2.3Oral discourse completion task

As another pragmatic measure, ODCT is a special pragmatic instrument which requires participants to cooperatively listen to the scenarios from a tape recorder and then transfer their oral responses to a different tape recorder (Brown 2001Brown, J. D. 2001 “Pragmatics Test: Different Purposes, Different Tests.” In Pragmatics in Language Teaching, ed. by Kenneth. R. Rose, and Gabriele Kasper, 301–325. Cambridge: Cambridge university press. CrossrefGoogle Scholar; cf. Appendix B). ODCT cannot reflect all features of natural interactions because it is only an oral response to an oral question. In fact, there is no real interaction between someone who asks the question and the respondent. Therefore, ODCT shares all the limitations of the WDCT except its delivery mode and the ability to analyze some speech features such as pause length and speech rate which are the main rationales behind the existence of this type of data-elicitation tool (Taguchi 2018Taguchi, Naoko 2018 “Data Collection and Analysis in Developmental L2 Pragmatics Research: Discourse Completion Test, Role Play, and Naturalistic Recording.” In Critical Reflections on Data in Second Language Acquisition, ed. by Aarnes Gudmestad and Amanda Edmonds, 7–32. Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar). The necessity of employing recording devices and the time-consuming process of gathering data through ODCT are among other limitations of this pragmatic measure.

2.2.4Role-play

Role-plays can be regarded as a suitable or a well-balanced alternative since they can make up for the inherent problems of the naturally-occurring data collection methods such as broadness of scope in studying a particular variable and lack of authenticity and adaptability of the elicited data through WDCTs and ODCTs (Bardovi-Harlig 2018Bardovi-Harlig, Kathleen 2018 “Matching Modality in L2 Pragmatics Research Design”. System: 75, 13–22. CrossrefGoogle Scholar; Taguchi 2018Taguchi, Naoko 2018 “Data Collection and Analysis in Developmental L2 Pragmatics Research: Discourse Completion Test, Role Play, and Naturalistic Recording.” In Critical Reflections on Data in Second Language Acquisition, ed. by Aarnes Gudmestad and Amanda Edmonds, 7–32. Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar; cf. Appendix C). Accordingly, these problems are more likely to mitigate researchers’ control over the intended variables. Furthermore, role-plays provide the participants with an opportunity to be well- aware of or familiarized with the details of each scenario containing the intention of the conversation and the interlocutors’ social status. Finally, the participants are invited to play roles (Félix-Brasdefer 2010Félix-Brasdefer, J. Cesar 2010 “Data Collection Methods in Speech Act Performance: DCTS, Role Plays, and Verbal Reports.” In Speech Act Performance: Theoretical, Empirical and Methodological Issues, ed. by Alicia Martínez-Flor and Esther Usó-Juan, 64–81. Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar).

Although this type of pragmatic measure has the advantage of being close to the natural data and enjoys high degree of researcher control over the intended variables, the time-consuming process of this method can be considered a limitation (Eslami-Rasekh and Mirzaei 2014Eslami, Zohreh. R., and Azizullah Mirzaei 2014 “Speech Act Data Collection in a Non-Western Context: Oral and Written DCTs in the Persian Language.” Iranian Journal of Language Testing, 4(1): 137–154.Google Scholar). The ability of the interactants to play roles is another serious limitation of this method. Visualizing oneself in a specific context and assuming roles in that particular context usually calls for an acting ability which might be absent in most participants, if not all of them (Kasper 2000Kasper, Gabriele 2000 “Data Collection in Pragmatics Research.” In Culturally Speaking: Managing Rapport Through Talk across Cultures, ed. by Helen Spencery-Oatey, 316–41. London and New York: Continuum.Google Scholar; Schauer 2009Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar).

2.3Studies focusing on pragmatic measures

Comparing and contrasting different measures of pragmatics has been the concern of several studies in the field of Interlanguage Pragmatics (ILP). Rose and Ono (1995)Rose, Kenneth. And Reiko Ono 1995 “Eliciting Speech Act Data in Japanese: The Effect of Questionnaire Type.” Language Learning, 45(2): 191–223. CrossrefGoogle Scholar, for example, targeted request speech act and investigated the impact of data collection methods on the ultimate elicited data. Having collected data through Multiple Choice Questionnaires (MCQs) and Discourse Completion Tests (DCTs), they found that the collected data through these two methods did not match or correspond. Their Japanese participants employed more non-conventionally indirect requests or hints in completing MCQs than DCTs.

Drawing upon request and refusal responses, Sasaki (1998)Sasaki, Miyuki 1998 “Investigating EFL Students’ Production of Speech Acts: A Comparison of Production Questionnaires and Role Plays.” Journal of Pragmatics, 30: 457–484. CrossrefGoogle Scholar compared Role-play and WDCT in a Japanese EFL context and reported that Role-plays tended to elicit longer data and that more strategies were used in Role-plays in comparison with WDCTs.

Considering five dependent variables of response length, the number of exclamation particles, the number of repetitions, the number of inversions, and the number of omissions in the elicited data, Yuan (2001)Yuan, Y. 2001 “An Inquiry into Empirical Pragmatics Data Gathering Methods: Written DCTs, Oral DCTs, and Natural Conversations.” Journal of pragmatics, 33: 271–92. CrossrefGoogle Scholar compared ODCT, WDCT, field notes, and recorded conversations. The data obtained through compliment and compliment responses revealed that while ODCTs shared some drawbacks of WDCTs, they elicited data containing more features of natural conversation.

Naturally-occurring data and DCTs were investigated by Golato (2003)Golato, Andrea 2003 “Studying Compliment Responses: A Comparison of DCTs and Recordings of Naturally Occurring Talk.” Applied Linguistics, 24(1): 90–121. CrossrefGoogle Scholar. Having collected compliment responses during natural interactions, she designed a kind of DCT which reflected the context of natural interactions. The results indicated divergences between these two data collection procedures. She concluded that the data gathered through interactions allowed the researcher to study the organization and realization of language in natural interactions. However, the data elicited through DCTs mainly represented metapragmatic information.

Eslami-Rasekh and Mirzaie (2014)Eslami, Zohreh. R., and Azizullah Mirzaei 2014 “Speech Act Data Collection in a Non-Western Context: Oral and Written DCTs in the Persian Language.” Iranian Journal of Language Testing, 4(1): 137–154.Google Scholar examined the validity of the elicited data by WDCT and ODCT in terms of the response length, range and content of the expressions, formality level, and spoken versus written forms of language. They ultimately found that the two measures yielded different production responses. ODCTs mostly yielded longer and more elaborate samples in comparison with WDCTs. They concluded that data elicitation through WDCT in a language like Persian, with striking differences between spoken and written genres, was quite inappropriate.

A general survey of previously-done studies in the existing literature points to the dearth and paucity of research on the simultaneous comparison and contrast of WDCT, ODCT, and Role-play with Natural methodology as different measures of pragmatic competence in an EFL institutional context with a specific emphasis on the request speech act and its internal and external modification devices. This study is an attempt to fill the above-mentioned gap in the literature.

2.4Request speech act

Requests, as one of the most face-threatening acts, require considerable cultural and linguistic expertise on the part of the speaker. Due to this complexity, many language learners deviate from target norms and use them inappropriately (Uso-Juan 2010Uso-Juan, Esther 2010 “Requests: A Sociopragmatic Approach.” In Speech Act Performance: Theoretical, Empirical and Methodological Issues ed. by Alicia Martinez-Flor and Esther Uso-Juan, 237–256. Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar). The degree of imposition associated with the request act, the relative power of the interlocutors, and the social distance between them, according to Brown and Levinson (1987)Brown, Penelope, and Steven, C. Levinson 1987Politeness: Some Universals in Language Usage. Cambridge: Cambridge University Press. CrossrefGoogle Scholar, are the three important variables in the act of requesting.

The two main components of requesting are the request head act or the utterance that serves the function of requesting and the optional modification devices which precede or follow the request head act to modify its illocutionary force (Sifianou 1999Sifianou, Maria 1999Politeness Phenomena in England and Greece. A Crosscultural Perspective. Oxford: Oxford University Press.Google Scholar). These modification devices can either appear within the request head act (internal modifiers) or in the immediate linguistic context that surrounds it (external modifiers).

Numerous attempts have been made by researchers to present different taxonomies for the classification of request strategies. In her taxonomy, Schauer (2009)Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar classified requests into direct, conventionally indirect, and non-conventionally indirect requests. Imperatives, performatives, want statements, and locution derivables can be considered the subcategories of direct requests. In this type of request, the requester’s intention lies in the exact sentence which is uttered, and the requestee can easily grasp the illocutionary force of the request. Conventionally indirect requests include suggestory formula, availability, prediction, permission, willingness and ability. Through this strategy, the requester utilizes conventionalized linguistic items to soften the illocutionary force of an utterance for request formulation. In non-conventionally indirect request strategy or hint, the request is not clear enough to be interpreted easily and maximum level of responsibility is assumed for the hearer to interpret it (Schauer 2009Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar).

Table 1.Taxonomy of request strategies (Borrowed from Schauer 2009Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar, 86)
Direct Requests
Imperatives Tell me the way to X!
Performatives
unhedged I’m asking you to tell me the way to X.
hedged I want to ask you the way to X.
Want statements I wish you’d tell me the way to X.
Locution derivable Where is X?
Conventionally Indirect requests
Suggestory formula How about telling me the way to X?
Availability Have you got time to tell me the way to X?
Prediction Is there any chance to tell me the way to X?
Permission Could I ask you about the way to X?
Willingness Would you mind telling me the way to X?
Ability Could you tell me the way to X?
Non-Conventionally Indirect Requests
Hints I have to meet someone in X.

Schauer (2009)Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar further subclassified internal modification devices as lexical modifiers (downtoners, politeness markers, understaters, past tense modals, consultative devices, hedges, aspect, and marked modalities) and syntactic modifiers (conditional clauses, appreciative embeddings, tentative embeddings, tag questions, and negations).

Table 2.Taxonomy of internal modifiers: Lexical downgraders (Borrowed from Schauer 2009Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar, 90)
Name Function Example
Downtoner sentence adverbial that is used to reduce the force of the request Could I maybe have some of them or could you bring a copy or something?
Politeness marker employed by the speakers to bid for their interlocutors’ cooperation Could you open the window a little bit, please?
Understater adverbial modifier that is employed to decrease the imposition of the request by underrepresenting theproposition of the request Can you speak up a bit, please?
Past Tense Modals past tense forms such as could instead of can make the request appear more polite Professor Jones, could you show me the direction to the Trent Building?
Consultative Device used to consult the interlocutor’s opinion on the proposition of the request Erm, Lucy, would you mind filling in this questionnaire for me?
Hedge adverbial that is used by the speaker to make the request more vague Is it possible if we can arrange a meeting during the holidays somehow?
Aspect progressive form of verb that is used deliberately by the speaker I was wondering if maybe you could give them to me tomorrow?
Marked Modality might and may make the request appear more tentative. Excuse me, may I just pass?

She also subcategorized external modifiers into alerters, preparators, grounders, disarmers, imposition minimizers, sweeteners, promise of reward, small talks, appreciators, and considerators.

Table 3.Taxonomy of external modifiers (Borrowed from Schauer 2009Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar, 92)
Name Function Example
Alerter linguistic device that is used to get the interlocutor’s attention; precedes the Head Er; excuse me; hello; Peter
Preparator short utterance that intends to prepare the interlocutor for the request; can follow or substitute the Alerter May I ask you a favor?
Head the actual request Do you know where the Portland Building is?
Grounder provides an explanation for request the Erm, unfortunately, I really don’t understand this topic here
Disarmer used to pre-empt the interlocutor’s potential objections I know you are really busy but maybe you’ve got some minutes for me.
Imposition Minimizer employed to decrease the imposition of the request I will return them immediately, the next day.
Sweetener employed to flatter the interlocutor and to put them into a positive mood I think you are the perfect person to do it
Promise of Reward the requester offers the interlocutor a reward for fulfilling the request I would fill in yours [the questionnaire] as well, if you need one, one day
Small talk short utterance at the beginning of the request that is intended to establish a positive atmosphere Good to see you
Appreciator usually employed at the end of the request to positively reinforce it That would be very nice
Considerator employed at the end of the request; intends to show consideration towards the interlocutor’s situation Only if you’ve got the time of course

Drawing upon Schauer’s (2009)Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar taxonomy of request strategy and its internal and external modification devices, the present study is an attempt to compare the elicited data through WDCT, ODCT, and Role-play with Natural methodology. More specifically, the following research questions were formulated:

  1. How do requests elicited by the WDCTs compare to the naturally-occurring requests in institutional communication in terms of internal and external modification devices?

  2. How do requests elicited by the ODCTs compare to the naturally-occurring requests in institutional communication in terms of internal and external modification devices?

  3. How do requests elicited by the Role-plays compare to the naturally-occurring requests in institutional communication in terms of internal and external modification devices?

3.Methodology

3.1Participants

To conduct the study, 56 students’ conversations and their talk-in interactions with their teachers in an English language institute in the Iranian EFL context were recorded and transcribed as the natural data. Having analyzed the recorded data, the researchers found that 27 participants met the required conditions to be included in the study, and they agreed to accompany the researchers in the course of this study by allowing the researchers to record their interactions in the classroom. The required condition to include the participants in the study was to make two requests with contextual features of low-status, low-imposition, and high-status, low- imposition respectively. The participants were intermediate in terms of their overall language proficiency. To determine their language proficiency, the researchers administered the Michigan Test of English Language Proficiency (MTELP). The results of the test were indicative of the fact that the participants were at level two or intermediate. Their age varied from 19 to 28 and basic demographic characteristics such as L1 and culture were common among them.

3.2Instruments

Four methods of pragmatic data collection (Natural methodology, Role-play, ODCT, and WDCT) were utilized to carry out this study. To gather naturally-occurring data, the researchers recorded the students’ institutional interactions in the classroom wherein the instructor primarily intended to improve and promote the students’ overall English language proficiency in general and teach them how to make proper L2 requests in particular. Having gathered the data, the researchers analyzed the data to spot the most important features of the gathered data and to select the most appropriate scenarios for the Role-play, ODCT, and WDCT. This process was undertaken to ensure whether the selected scenarios reflected the true features of the collected natural conversations and to make the comparison among these four measures more plausible and tangible.

During the data analysis, it came to light that most of the requests produced by the participants in the natural institutional context were low-status, low-imposition, and high-status, low- imposition. This prevalence led the researchers to choose requests with these contextual features in the natural data. In the same vein, the scenarios of the other measures (Role-plays, ODCTs, and WDCTs) were selected based on the same contextual features. Therefore, four requests of each individual participant, two low-status, low-imposition and two high-status, low-imposition, were picked up for later in-depth analysis. Social status is the degree of the interactants’ social leverage and influence over one another and imposition refers to the load or burden of a request. Two values of social status and imposition, high and low, were taken into account in the present study.

The scenarios for the Role-plays, ODCTs, and WDCTs were mainly borrowed from Schauer (2009)Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar and Jalilifar (2009)Jalilifar, Alireza 2009 “Request Strategies: Cross-Sectional Study of Iranian EFL Learners and Australian Native Speakers.” English Language Teaching, 2(1): 46–61. CrossrefGoogle Scholar. In the Role-play phase, the participants were presented with two low-status, low-imposition and two high-status, low-imposition scenarios. They were invited to play the specified roles which culminated in the production of four more requests by the other students. The ODCTs and WDCTs were administered in the following weeks. Congruent with natural data, the participants responded to two low-status, low-imposition and two high-status, low-imposition scenarios in each methodology. In other words, each individual participant formulated 16 requests in throughout the implementation process of this study.

3.3Data collection procedure

All the participants had to go through a fifteen-week instructional period to complete the study. Data collection took about a whole semester including 15 weeks. To collect the data, the researchers took several methodological steps at different times during the semester. The institutional interactions of the students in the classroom were recorded during the first twelve weeks. The first three weeks were ignored due to the observer paradox. From among the 56 participants, 27 students had produced at least two low-status, low-imposition and two high-status, low-imposition requests in the natural data. Therefore, these 27 participants finally met the required condition to accompany the researchers in the course of this study.

Having collected the natural data, the researchers engaged the participants in role-plays in the 13th week. All the participants were called to a classroom in pairs and the printed scenarios were distributed to one of them to be read prior to making a request. The other participant was just there to answer the requesters’ request impromptu. For the first two scenarios, both participants were students because the social status of the first two scenarios was low. However, for the next two scenarios, one of the parties was the instructor. He played the role of a requestee because the social status of the other two requests was high. The requester read each scenario then he/she made his/her request one by one. The participants’ interactions were then recorded and transcribed later on.

The ODCTs were administered in the 14th week. The researchers asked a non-native proficient language user of English with a sound knowledge or good command of English, especially pronunciation and accent, to read the scenarios out loud to be recorded. The scenarios were not the same, but they shared the same contextual features of status and imposition as the Role-plays. Having recorded the scenarios, the researchers directed the participants to a classroom one by one to make requests based on the recorded scenarios of the ODCTs. They listened to the scenarios and made their requests based upon those scenarios. Their responses were recorded and were transcribed later on. Finally, in the 15thweek and in the final phase, the WDCTs were administered. The WDCT scenarios were not the same as those incorporated in the Role-play and ODCT, but they shared the same contextual features of status and imposition. The participants were then asked to read the scenarios and write their answers in the blank right below them.

3.4Data analysis

To analyze the data, the recorded the Role-plays and the ODCTs were transcribed. Then, instances of request strategies: Direct, conventionally indirect, and non-conventionally indirect along with internal and external modifiers in the natural data, Role-plays, ODCTs, and WDCTs were counted and the total number of each request strategy type or modification device was determined. The most frequent internal modifiers used in the participants’ requests were downtoners, politeness markers, and past tense modals. Likewise, alerters, considerators, grounders, and appreciators were the most common external modification devices in the data. Finally, having considered Schauer’s (2009)Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar taxonomy of request head act and its internal and external modification devices, the researchers employed Chi-Square and binomial tests to analyze the data.

4.Results

4.1Request strategies

Table 4 summarizes the frequencies and percentages of the direct, indirect, and non-conventionally indirect request strategies in the four pragmatic measurements. As it can be readily discerned, WDCT, ODCT, and Role-Play share striking similarities and in all these three measures, conventionally indirect request strategies are the most-frequently used one. WDCs and Role-Plays enjoy a total number of 78 conventionally indirect request strategies accounting for 28.3% of the total number of requests, while the same strategies also occur in ODCTs with a higher number (82) and a higher percentage (29.7%) as well. The case for the Natural methodology is rather different as more direct request strategies (a total number of 48 accounting for 39.3% of the total number of requests) are employed by the interactants compared to the conventionally indirect request strategies with a total number of 38 accounting for 13.8% of the overall occurrences.

Table 4.Descriptive statistics of request strategies
Request Total
Direct request Conventionally Indirect Request Non-Conventio-nall Indirect Request
Method WDCT Count  28  78  2 108
% within Request     23.0%     28.3%     5.9%     25.0%
ODCT Count  22  82  4 108
% within Request     18.0%     29.7%    11.8%     25.0%
Role-play Count  24  78  6 108
% within Request     19.7%     28.3%    17.6%     25.0%
Natural Count  48  38 22 108
% within Request     39.3%     13.8%    64.7%     25.0%
Total Count 122 276 34 432
% within Request    100.0%    100.0%   100.0%    100.0%

To answer research questions one to three and to compare and contrast the results obtained from the WDCTs, ODCTs, and Role-plays in terms of the request strategies with the data elicited through the Natural methodology, the researchers ran binomial tests.

Table 5.Binomial test data in terms of request strategies
Category N Observed Prop. Test Prop. Exact sig. (2-tailed)
WDCT & Natural Direct Request

WDCT

Natural

Total

 28.00

 48.00

28

78

106

 .26

 .74

1.00

.50 .00

Conventionally

Indirect Request

WDCT

Natural

Total

 78.00

 38.00

28

78

106

 .26

 .74

1.00

.50 .00

Non-conventionally

Indirect Request

WDCT

Natural

Total

  2.00

 22.00

28

78

106

 .26

 .74

1.00

.50 .00
ODCT & Natural Direct Request

ODCT

Natural

Total

 22.00

 48.00

28

78

106

 .26

 .74

1.00

.50 .00

Conventionally

Indirect Request

ODCT

Natural

Total

 82.00

 38.00

28

78

106

 .26

 .74

1.00

.50 .00

Non-conventionally

Indirect Request

ODCT

Natural

Total

  4.00

 22.00

28

78

106

 .26

 .74

1.00

.50 .00
Role-play & Natural Direct Request

Role-play

Natural

Total

 24.00

 48.00

28

78

106

 .26

 .74

1.00

.50 .00

Conventionally

Indirect Request

Role-play

Natural

Total

 78.00

 38.00

28

78

106

 .26

 .74

1.00

.50 .00

Non-conventionally

Indirect Request

Role-play

Natural

Total

  6.00

 22.00

28

78

106

 .26

 .74

1.00

.50 .00

Table 5 indicates that the WDCTs, ODCTs, and Role-plays significantly differed from Natural methodology in terms of request strategies. Direct and non-conventionally indirect request strategies showed up more frequently in the natural data. However, conventionally indirect request strategy stood out more conspicuously in the WDCTs, ODCTs, and Role-plays as the main strategy to make requests. Therefore, it came to light that the WDCTs, ODCTs, and Role-plays did not elicit the same data as naturally-occurring elicitation methodology did in terms of the request strategies.

A Pearson Chi-Square test was administered to compare the participants’ production of the request strategies in the WCDTs, ODCTs, and Role-plays. The results revealed the value of 2.891 with 4 df and Asymp. Sig. 0.576 suggesting that the difference among these three measures of pragmatics was insignificant.

Figure 1.Request strategy use pattern
Figure 1.

Figure 1 depicts the request strategy use pattern in the four measures of pragmatics and indicates that the WDCTs, ODCTs, and Role-plays shared striking similarities but differed markedly from the Natural methodology in terms of request strategy type elicitation.

4.2Internal modification devices

Downtoners, politeness markers, and past tense modals were the most frequently-used internal modifiers employed by the participants to soften the illocutionary force of their requests in this study. Table 6 summarizes the percentages and frequencies of the data for internal modifiers elicited through the four measures of pragmatics utilized in the current study.

Table 6.Descriptive statistics of request internal modifiers
Internal Modification Devices Total
Downtoner Politeness marker Past tense modal
Method WDCT Count  66  38 29 133
% within Internal     29.2%     32.5%    35.8%     31.4%
ODCT Count  70  32 22 124
% within Internal     31.0%     27.4%    27.2%     29.2%
Role-play Count  74  37 24 135
% within Internal     32.7%     31.6%    29.6%     31.8%
Natural Count  16  10  6  32
% within Internal      7.1%      8.5%     7.4%      7.5%
Total Count 226 117 81 424
% within Internal    100.0%    100.0%   100.0%    100.0%

Binomial tests were employed to see whether the WDCTs, ODCTs, and Role-plays elicited similar data as the Natural data did in terms of the request internal modifiers.

Table 7.Binomial tests in terms of request internal modifiers
Category N Observed Prop. Test Prop. Exact sig. (2-tailed)
WDCT & Natural Direct Request

WDCT

Natural

Total

1.00

 .00

66

16

82

 .80

 .20

1.00

.50 .00
Politeness Marker

WDCT

Natural

Total

1.00

 .00

38

10

48

 .79

 .21

1.00

.50 .00
Past Tense Modal

WDCT

Natural

Total

1.00

 .00

29

 6

35

 .83

 .17

1.00

.50 .00
ODCT & Natural Downtoner

ODCT

Natural

Total

1.00

 .00

70

16

86

 .89

 .19

1.00

.50 .00
Politeness Marker

ODCT

Natural

Total

1.00

 .00

32

10

42

 .76

 .24

1.00

.50 .00
Past Tense Modal

ODCT

Natural

Total

1.00

 .00

22

 6

28

 .79

 .21

1.00

.50 .04
Role-play & Natural Downtoner

Role-play

Natural

Total

1.00

 .00

74

16

90

 .82

 .18

1.00

.50 .00
Politeness Marker

Role-play

Natural

Total

1.00

 .00

37

10

47

 .79

 .21

1.00

.50 .00
Past Tense Modal

Role-play

Natural

Total

1.00

 .00

24

 6

30

 .80

 .20

1.00

.50 .00

Table 7 demonstrates that the internal modifiers in the WDCTs, ODCTs, and Role-plays occurred about three times more than the Natural data and suggests that these three measures differed significantly from the Natural methodology in terms of the internal modifiers.

To compare the WCDTs, ODCTs, and Role-plays in terms of the participants’ use of internal modifiers, the researchers ran a Pearson Chi-Square test. The Chi-square statistics stood at 1.550 with 4 df and Asymp. Sig. 0.818 which was indicative of the fact that no significant differences existed among the three above-mentioned pragmatic measures.

Figure 2.Request internal modification devices use pattern
Figure 2.

Although Figure 2 reveals the WDCTs, ODCTs, Role-plays, and the Natural methodology display the same pattern in terms of downtoners, politeness markers, and past modals, the pattern of elicited data through the WDCTs, ODCTs, and Role-plays differs considerably from that of the Natural data in terms of the frequencies of occurrences in each with regard to the utilization and employment of downtoners, politeness markers, and past modals.

4.3External modification devices

Table 8 displays the descriptive statistics of the request external modifiers (appreciators, grounders, alerters, and considerators) for the WDCTs, ODCTs, Role-plays, and Natural methodology. From among different external modifications, grounders with a total number of 82, 85, and 40 occurrences were the most frequently-used external modifiers in the WDCTs, ODCTs, and Natural methodology respectively whereas alerters with a total number of 92 occurrences were the most frequently-employed external modifiers in the Role-plays.

Table 8.Descriptive statistics of request external modifiers
External Modification Devices Total
Appreciator Grounder Alerter Considerator
Method WDCT Count  65  82  72   6 225
% within Method     28.9%     36.4%     32.0%      2.7%    100.0%
ODCT Count  58  85  75   4 222
% within Method     26.1%     38.3%     33.8%      1.8%    100.0%
Role-play Count  78  89  92  64 323
% within Method     24.1%     27.6%     28.5%     19.8%    100.0%
Natural Count  13  40   6  29  88
% within Method     14.8%     45.5%      6.8%     33.0%    100.0%
Total Count 214 296 245 103 858
% within Method     24.9%     34.5%     28.6%     12.0%    100.0%

Binomial tests were conducted to determine whether the WDCTs, ODCTs, and Role-plays elicited the same data as the Natural data did in terms of request external modifiers.

Table 9.Binomial tests in terms of request external modifiers
Category N Observed Prop. Test Prop. Exact sig. (2-tailed)
WDCT & Natural Appreciator

WDCT

Natural

Total

1.00

 .00

 65

 13

 78

 .83

 .17

1.00

.50 .00
Grounder

WDCT

Natural

Total

1.00

 .00

 82

 40

122

 .67

 .33

1.00

.50 .00
Alerter

WDCT

Natural

Total

1.00

 .00

 72

  6

 78

 .92

 .08

1.00

.50 .00
Considerator

WDCT

Natural

Total

1.00

 .00

  6

 29

 35

 .17

 .83

1.00

.50 .00
ODCT & Natural Appreciator

ODCT

Natural

Total

1.00

 .00

 58

 13

 71

 .82

 .18

1.00

.50 .00
Grounder

ODCT

Natural

Total

1.00

 .00

 85

 40

125

 .68

 .32

1.00

.50 .00
Alerter

ODCT

Natural

Total

1.00

 .00

 75

  6

 81

 .93

 .07

1.00

.50 .04
Considerator

ODCT

Natural

Total

1.00

 .00

  4

 29

 33

 .86

 .14

1.00

.50 .04
Role-play & Natural Appreciator

Role-play

Natural

Total

1.00

 .00

 78

 13

 91

 .86

 .14

1.00

.50 .00
Grounder

Role-play

Natural

Total

1.00

 .00

 89

 40

129

 .69

 .31

1.00

.50 .00
Alerter

Role-play

Natural

Total

1.00

 .00

 92

  6

 98

 .94

 .06

1.00

.50 .00
Considerator

Role-play

Natural

Total

1.00

 .00

 64

 29

 93

 .69

 .31

1.00

.50 .00

Table 9 demonstrates that the WDCTs, ODCTs, and Role-plays elicited different data from the Natural methodology in terms of external modification devices. The WDCTs, ODCTs, and Role-plays elicited more appreciators, grounders, and alerters. However, considerators appeared more frequently in natural interactions. The results of a Chi-Square test with the value = 68.332 and DF = 6 and Asymp. Sig. = 0.000 indicated that the WDCTs, ODCTs, and Role-plays differed significantly from one another in terms of eliciting request external modifiers.

Figure 3.Request external modification devices use pattern
Figure 3.

5.Discussion

The present study was an attempt to compare and contrast three common measures of pragmatic knowledge, Written Discourse Completion Task (WDCT), Oral Discourse Completion Task (ODCT), and Role-play, with Natural methodology in an EFL institutional context. To this end, the participants’ employment of the request speech act and its internal and external modification devices were investigated in these four measures of pragmatic knowledge. The results of the study demonstrated marked differences between the three common pragmatic data elicitation methods and Natural methodology. In accordance with the findings of many other studies (e.g., Economidou-Kogetsidis 2013Economidou-Kogetsidis, Maria 2013 “Strategies, Modification and Perspective in Native Speakers’ Requests: A Comparison of WDCT and Naturally Occurring Requests.” Journal of Pragmatics, 53: 21–38. CrossrefGoogle Scholar; Golato 2003Golato, Andrea 2003 “Studying Compliment Responses: A Comparison of DCTs and Recordings of Naturally Occurring Talk.” Applied Linguistics, 24(1): 90–121. CrossrefGoogle Scholar; Turnbull 2001Turnbull, William 2001 “An Appraisal of Pragmatic Elicitation Techniques for the Social Psychological Study of Talk: The Case of Request Refusals.” Pragmatics, 11(1): 31–61. CrossrefGoogle Scholar; Yuan 2001Yuan, Y. 2001 “An Inquiry into Empirical Pragmatics Data Gathering Methods: Written DCTs, Oral DCTs, and Natural Conversations.” Journal of pragmatics, 33: 271–92. CrossrefGoogle Scholar), the results of the present study also suggest that, in spite of some similarities, common pragmatic elicitation methods such as WDCT, ODCT, and Role-play cannot elicit similar data to the ones gathered through Natural methodology.

In terms of request head act strategies, the participants were more inclined to apply more direct request and non-conventionally indirect request strategies to make requests in natural conversations. However, conventionally indirect request strategies were more popular in the WDCTs, ODCTs, and Role-plays. This may have been due to the spontaneous nature of the interactions and the velocity of the ongoing events under naturally-occurring circumstances, the participants may have been relying upon their implicit or unconscious and fully-automatized socio-pragmatic knowledge. In the same vein, because the participants were provided with ample and sufficient time to ponder upon their responses for the WDCTs, ODCTs, and Role-plays, they had largely relied upon their explicit and conscious pragmalinguistic knowledge, and their responses had mainly represented their pragmatic knowledge of request expressions rather than their spontaneity or off-the-cuff (impromptu) ability to make English requests. The participants displayed an entirely different pattern in using internal and external modification devices in their requests. They utilized remarkably more internal and external modification devices in the WDCTs, ODCTs, and Role-plays. This frequent utilization of internal and external modification devices in the WDCTs, ODCTs, and Role-plays could be reasonably justified by the participants’ dependence and heavy reliance upon their explicit pragmalinguistic knowledge which, in turn, might have led to such overuse of internal and external modification devices. In the Iranian EFL context, requesters typically try to be polite and do facework during requesting. It seems that, as the following extract from the data pool (the WDCT) reveals, the respondents have paid special attention to embellish the form of the requests to show politeness and this was possible because of the off-line nature of the tasks which has enabled them to draw on their pragmaliguistic knowledge.

(1)

Requester:
Ehsan! Could you open the window, please? It’s really hot in here. If no problem, of course.

Example (1) indicates that one of the participants has employed different internal (politeness markers and past tense modals) and external modification devices (alerters, grounders, and considerators) to produce a low-imposition, low-status request in a WDCT. In line with Turnbull (2001)Turnbull, William 2001 “An Appraisal of Pragmatic Elicitation Techniques for the Social Psychological Study of Talk: The Case of Request Refusals.” Pragmatics, 11(1): 31–61. CrossrefGoogle Scholar, Oral and Written DCTs are appropriate choices to study forms of address. This study also confirms the appropriateness of these methods along with Role-plays to study the form of language in general.

However, in natural contexts, due to the spontaneous nature of the interactions, the requesters do not find enough time to draw on their explicit knowledge to form requests. They merely meet the sociopragmatic demand of requesting by exploiting their automatized interlanguage pragmatic knowledge. This can be ascribed to the fact that the interactants’ requests are communicatively-oriented in nature and this might have prevented them to go through superfluous and redundant details like the ones prevalent and abundant in the WDCTs, ODCTs, and Role-plays. Therefore, a combination of both time pressure the interactants feel they are under and the communicative orientation of the interaction (the speedy negotiation of meaning) could have been the explanatory factors that could possibly account for the learners’ drawing upon their fully automatized implicit knowledge of request speech acts.

Another likely reason for the overuse of internal and external modification devices in the WDCTs, ODCTs, and Role-plays could stem from the lack of sufficient and appropriate context. In natural interactions, interactants can convey their purpose through body language or a single word, but they have to perform linguistically even in unnecessary situations in other measures of pragmatic knowledge. In natural settings, the respondents can enjoy their “opting out” options to say nothing in proper situations. As in the following Example (2), the requester asked for the pen and left the sentence incomplete by taking advantage of the real-life context and pointing to the pen.

(2)

Requester:
Ali?
Requestee:
Just looking at the requester.
Requester:
Give me your………

In the WDCTs, the situation is even more obscured due to the mode of delivery. Writing is considerably different from speaking (Eslami-Rasekh and Mirzaie 2014Eslami, Zohreh. R., and Azizullah Mirzaei 2014 “Speech Act Data Collection in a Non-Western Context: Oral and Written DCTs in the Persian Language.” Iranian Journal of Language Testing, 4(1): 137–154.Google Scholar), and in this data elicitation technique, respondents have to fill in the blanks out of obligation. Consistent with the Yuan (2001)Yuan, Y. 2001 “An Inquiry into Empirical Pragmatics Data Gathering Methods: Written DCTs, Oral DCTs, and Natural Conversations.” Journal of pragmatics, 33: 271–92. CrossrefGoogle Scholar, the present study also confirmed the dissimilarity and disparity of the elicited data through WDCTs with the gathered data by Natural methodology. However, in sharp contrast with Yuan’s (2001)Yuan, Y. 2001 “An Inquiry into Empirical Pragmatics Data Gathering Methods: Written DCTs, Oral DCTs, and Natural Conversations.” Journal of pragmatics, 33: 271–92. CrossrefGoogle Scholar findings, it was uncovered that ODCTs do not show or present enough potentiality to elicit similar data to Natural data.

Situation of data collection also might account for yielding different data features through methods of pragmatic data elicitation and data collection. In making requests in the WDCTs, ODCTs, and Role-plays, the participants should imagine and visualize the intended situations and contexts, and not all of the participants enjoy this ability equally (Schauer 2009Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar). In real conversations, the interactants are actually engaged in conversing and there is no need for imagination to be involved. They have a real purpose in their mind during request-making, and they want to make some future changes. Under naturally-occurring conversational circumstances, interactants are more goal-directed and purposeful whereas the interactants in the WDCTs, ODCTs, and Role-plays tend to resort to their imaginative power to act. Actually, they are not involved in conversations, and this lack of involvement in real conversations can lead them to talk more (Turnbull 2001Turnbull, William 2001 “An Appraisal of Pragmatic Elicitation Techniques for the Social Psychological Study of Talk: The Case of Request Refusals.” Pragmatics, 11(1): 31–61. CrossrefGoogle Scholar). This could have resulted in the overuse of external and internal modification devices in the WDCTs, ODCTs, and Role-plays to enhance and enrich the formal structure of the requests. Therefore, an element of artificiality is present which could have undermined the naturalness of such measures.

The test/task-like nature of the WDCTs, ODCTs, and Role-plays could also be another likely explanation for the marked differences between such measures and Natural methodology in terms of the utilization of internal and external modification devices (Sasaki 1998Sasaki, Miyuki 1998 “Investigating EFL Students’ Production of Speech Acts: A Comparison of Production Questionnaires and Role Plays.” Journal of Pragmatics, 30: 457–484. CrossrefGoogle Scholar). These measures are typically conceived as formal tests by language learners. It seems that the formality of these tests/tasks could have made the participants feel greatly compelled to account for their requests by prolonging them.

The results also revealed that the pattern of usage was different for the internal and external modifiers. As Figure 2 depicts, apart from the frequency of occurrence, the usage pattern is the same among the four measures of pragmatic knowledge for the internal modifiers. However, Figure 3 displays that no such consistent pattern can be observed for the external modifiers. One possible explanation for the consistency observed in the patterns of usage for the internal modifiers could be attributed to the fact that the learners might have tended to transfer their L1 strategies to their L2 since downtoners, politeness markers “please” and past tense modals are frequently used in Persian both in formal and informal requests. Simplicity of these structures that are acquired very early on by language learners could be another reason behind their over-utilization in almost all situations and contexts.

The pattern of usage for external modifiers was consistent in the WDCTs and ODCTs, but different in the Role-plays and Natural methodology. The reason for the high frequency of the alerters in the participants’ requests in Role-plays could be described by the very nature of this measure that requires the respondents to promptly take action in order to attract the interlocutor’s attention through endearment terms and solidarity particles. Furthermore, the grounders and considerators’ high frequency in natural interactions can be likely justified by the fact that there might have been a possible influence from the learners’ L1. As demonstrated in the following extract from the participants’ natural interactions, Iranian native speakers usually provide reasons, justifications and explanations for their requests in high-status situations to show both their deferential attitude and make sure that they are producing correct and appropriate requests.

(3)

Requester:
Excuse me! Could you repeat the sentence again? I couldn’t write it. I was busy writing your previous statement.

It seems that the formality of the situation made these learners feel greatly compelled to justify their requests by prolonging them (Woodfield 2012Woodfield, H. 2012 “I Think Maybe I Want to Lend the Notes from You: Development of Request Modification in Graduate Learners.” In Interlanguage Request Modification, ed. by Maria Economidou-Kogetsidis and Helen Woodfield, 9–49. Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar).

Broadly speaking, WDCTs might be more appropriate for tapping into explicit/ declarative knowledge, whereas Natural methodology might be more appropriate for tapping into automated/procedural knowledge. If the purpose is to delve into the socio-pragmatic layers of the language, Natural methodology, regarded as the most complex data collection method (House, 2018House, Juliane 2018 “Authentic vs. Elicited Data and Qualitative vs. Quantitative Research Methods in Pragmatics: Overcoming Two Non-Fruitful Dichotomies.” System, 75:4–12. CrossrefGoogle Scholar), is then highly recommended and best serves the purpose because as it was revealed, the participants tried to abide by social rules rather than pragmalinguistic considerations during natural interactions. The participants took into account request imposition and interlocutor status in natural interactions while making requests. However, this does not mean that Natural methodology is the best method to gather pragmatic data in all contexts and under every circumstance. Accordingly, Natural methodology sometimes cannot be accurate enough (Yuan 2001Yuan, Y. 2001 “An Inquiry into Empirical Pragmatics Data Gathering Methods: Written DCTs, Oral DCTs, and Natural Conversations.” Journal of pragmatics, 33: 271–92. CrossrefGoogle Scholar) and gathering data and controlling the contextual variables could be very difficult through this methodology (Economidou-Kogetsidis 2013Economidou-Kogetsidis, Maria 2013 “Strategies, Modification and Perspective in Native Speakers’ Requests: A Comparison of WDCT and Naturally Occurring Requests.” Journal of Pragmatics, 53: 21–38. CrossrefGoogle Scholar). Role-plays, considered the middle position data collection method (House 2018House, Juliane 2018 “Authentic vs. Elicited Data and Qualitative vs. Quantitative Research Methods in Pragmatics: Overcoming Two Non-Fruitful Dichotomies.” System, 75:4–12. CrossrefGoogle Scholar), if contextually-rich, can also be used to elicit both pragmalinguistic and sociolinguistic data. Moreover, Role-plays are reliable measures of pragmatic knowledge (Brown 2001Brown, J. D. 2001 “Pragmatics Test: Different Purposes, Different Tests.” In Pragmatics in Language Teaching, ed. by Kenneth. R. Rose, and Gabriele Kasper, 301–325. Cambridge: Cambridge university press. CrossrefGoogle Scholar) that can cater for both the social status of the interlocutors and the imposition and grammatical accuracy of the requests.

It goes without saying that the present study has its own limitations. One potential limitation that has to be mentioned is the possibility of a sequencing effect present in the research method applied. This could have probably impacted the frequencies between the target measures displayed in Table 4. Moreover, it might be worth mentioning that the obtained results could be limited to the Iranian culture, the classroom contexts taken advantage of, and generalizable to specifically-determined proficiency levels of the already-mentioned participants of the study. Such crucial factors might, in turn, either mitigate or undermine the generalizability of the findings to some extent.

Contextual information plays an important role in pragmatic elicitation techniques (Schauer 2009Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar). The amount of contextual information incorporated into the pragmatic measure can have a direct influence on the type of data which can be elicited. One possible research area to be thoroughly investigated is to compare two different versions of WDCT, ODCT, or Role-play with differing levels of contextual information to discern if the same or different data can be elicited. This area can be a promising one to determine the important role contextual information plays in eliciting different type of data. The employment of both quantitative and qualitative research methods or mixed method approaches (House 2018House, Juliane 2018 “Authentic vs. Elicited Data and Qualitative vs. Quantitative Research Methods in Pragmatics: Overcoming Two Non-Fruitful Dichotomies.” System, 75:4–12. CrossrefGoogle Scholar) and matched modality tasks (Bardovi-Harlig 2018Bardovi-Harlig, Kathleen 2018 “Matching Modality in L2 Pragmatics Research Design”. System: 75, 13–22. CrossrefGoogle Scholar) to tap language learners’ pragmatic knowledge are promising areas that demand more attention by interested researchers.

6.Conclusion

This study was conducted in an EFL institutional context and request speech act was picked up to carry out this study. These variables might have had their possible bearings on the findings. It is believed that some speech acts can be measured more effectively through certain methods (Eslami-Rasekh and Mirzaei 2014Eslami, Zohreh. R., and Azizullah Mirzaei 2014 “Speech Act Data Collection in a Non-Western Context: Oral and Written DCTs in the Persian Language.” Iranian Journal of Language Testing, 4(1): 137–154.Google Scholar; Hudson 2001Hudson, Thom 2001 “Indicators for Pragmatic Instruction.” In Pragmatics in Language Teaching, ed. by Kenneth R. Rose and Gabriele Kasper, 283–300. Cambridge: Cambridge university press. CrossrefGoogle Scholar). If another type of speech act had been used, it might have produced different results. Moreover, the design of the data collection procedure can also have a direct effect on the type of data that the pragmatic method elicits. The investigation of the elicited data through the WDCTs, ODCTs, and Role-plays and gathered data through the Natural methodology demonstrated marked differences. This finding highlights the fact that none of the elicitation measures could approximate the natural data. However, the closeness of one method of pragmatic data collection to Natural method does not mean that it is advantageous over other existing measures of pragmatics. Data collection methods should be chosen based on research questions and researchers’ objectives (Yuan 2001Yuan, Y. 2001 “An Inquiry into Empirical Pragmatics Data Gathering Methods: Written DCTs, Oral DCTs, and Natural Conversations.” Journal of pragmatics, 33: 271–92. CrossrefGoogle Scholar). In other words, each of them suits one particular research purpose though suffering from some specific limitations. In the WDCTs and ODCTs, for instance, the participants of this study cared more about surface pragmalinguistic features, and were overly concerned with making grammatically-correct requests. Therefore, despite lack of control over some discourse features such as paralinguistic and non-verbal elements, it could be claimed that DCTs lend themselves well to elicit pragmalinguistic and metapragmatic information (House 2018House, Juliane 2018 “Authentic vs. Elicited Data and Qualitative vs. Quantitative Research Methods in Pragmatics: Overcoming Two Non-Fruitful Dichotomies.” System, 75:4–12. CrossrefGoogle Scholar; Martinez-Flor and Uso-Juan, 2011Martínez-Flor, Alicia and Esther Usó-Juan 2011 “Research Methodologies in Pragmatics: Eliciting Refusals to Requests.” ELIA, 11: 47–87.Google Scholar).

References

Bardovi-Harlig, Kathleen
2018 “Matching Modality in L2 Pragmatics Research Design”. System: 75, 13–22. CrossrefGoogle Scholar
Bardovi-Harlig, Kathleen, and Beverly Hartford
2005Institutional Discourse and Interlanguage Pragmatics Research. In Interlanguage pragmatics: exploring institutional talk, ed. by Kathleen Bardovi-Harlig, and Beverly Hartford, 7–36. Mahwah, NJ: Lawrence Erlbaum. CrossrefGoogle Scholar
Bardovi-Harlig, Kathleen, and Sun-Young Shin
2014 “Expanding Traditional Testing Measures with Tasks from L2 Pragmatics Research.” Iranian Journal of Language Testing, 4 (1): 26–49.Google Scholar
Brown, J. D.
2001 “Pragmatics Test: Different Purposes, Different Tests.” In Pragmatics in Language Teaching, ed. by Kenneth. R. Rose, and Gabriele Kasper, 301–325. Cambridge: Cambridge university press. CrossrefGoogle Scholar
Brown, Penelope, and Steven, C. Levinson
1987Politeness: Some Universals in Language Usage. Cambridge: Cambridge University Press. CrossrefGoogle Scholar
Economidou-Kogetsidis, Maria
2013 “Strategies, Modification and Perspective in Native Speakers’ Requests: A Comparison of WDCT and Naturally Occurring Requests.” Journal of Pragmatics, 53: 21–38. CrossrefGoogle Scholar
Ellis, Rod
2008The Study of Second Language Acquisition (2nd ed.). Oxford: Oxford University Press.Google Scholar
Eslami, Zohreh. R., and Azizullah Mirzaei
2014 “Speech Act Data Collection in a Non-Western Context: Oral and Written DCTs in the Persian Language.” Iranian Journal of Language Testing, 4(1): 137–154.Google Scholar
Félix-Brasdefer, J. Cesar
2010 “Data Collection Methods in Speech Act Performance: DCTS, Role Plays, and Verbal Reports.” In Speech Act Performance: Theoretical, Empirical and Methodological Issues, ed. by Alicia Martínez-Flor and Esther Usó-Juan, 64–81. Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar
2007 “Natural Speech vs. Elicited Data: A Comparison of Natural and Role Play Requests in Mexican Spanish.” Spanish in Context, 4(2): 159–185. CrossrefGoogle Scholar
Golato, Andrea
2003 “Studying Compliment Responses: A Comparison of DCTs and Recordings of Naturally Occurring Talk.” Applied Linguistics, 24(1): 90–121. CrossrefGoogle Scholar
House, Juliane
2018 “Authentic vs. Elicited Data and Qualitative vs. Quantitative Research Methods in Pragmatics: Overcoming Two Non-Fruitful Dichotomies.” System, 75:4–12. CrossrefGoogle Scholar
Hudson, Thom
2001 “Indicators for Pragmatic Instruction.” In Pragmatics in Language Teaching, ed. by Kenneth R. Rose and Gabriele Kasper, 283–300. Cambridge: Cambridge university press. CrossrefGoogle Scholar
Ishihara, Noriko
2010 “Instructional Pragmatics: Bridging Teaching, Research, and Teacher Education.” Language and Linguistics Compass, 4(10): 938–953. CrossrefGoogle Scholar
Jalilifar, Alireza
2009 “Request Strategies: Cross-Sectional Study of Iranian EFL Learners and Australian Native Speakers.” English Language Teaching, 2(1): 46–61. CrossrefGoogle Scholar
Kasper, Gabriele
2000 “Data Collection in Pragmatics Research.” In Culturally Speaking: Managing Rapport Through Talk across Cultures, ed. by Helen Spencery-Oatey, 316–41. London and New York: Continuum.Google Scholar
Kasper, Gabriele and Carsten Roever
2005 ”Pragmatics in Second Language Learning.” In Handbook of Research in Second Language Teaching and Learning, ed. by Eli Hinkel, 317–334. Mahwah/New Jersey: Lawrence Erlbaum Associates.Google Scholar
Labov, William
1972Sociolinguistic Patterns. Philadelphia: University of Philadelphia.Google Scholar
Martínez-Flor, Alicia and Esther Usó-Juan
2011 “Research Methodologies in Pragmatics: Eliciting Refusals to Requests.” ELIA, 11: 47–87.Google Scholar
Norris, John. and Lourdes Ortega
2000 “Effectiveness of L2 Instruction: A Research Synthesis and Quantitative Meta-Analysis.” Language Learning, 50 (3): 417–528. CrossrefGoogle Scholar
Rose, Kenneth. And Reiko Ono
1995 “Eliciting Speech Act Data in Japanese: The Effect of Questionnaire Type.” Language Learning, 45(2): 191–223. CrossrefGoogle Scholar
Sasaki, Miyuki
1998 “Investigating EFL Students’ Production of Speech Acts: A Comparison of Production Questionnaires and Role Plays.” Journal of Pragmatics, 30: 457–484. CrossrefGoogle Scholar
Schauer, Gila. A.
2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar
Sifianou, Maria
1999Politeness Phenomena in England and Greece. A Crosscultural Perspective. Oxford: Oxford University Press.Google Scholar
Taguchi, Naoko
2018 “Data Collection and Analysis in Developmental L2 Pragmatics Research: Discourse Completion Test, Role Play, and Naturalistic Recording.” In Critical Reflections on Data in Second Language Acquisition, ed. by Aarnes Gudmestad and Amanda Edmonds, 7–32. Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar
Taguchi, Naoko, and Carsten Roever
2017Second Language Pragmatics. Oxford: Oxford University Press.Google Scholar
Takahashi, Satomi
2010 “Assessing Learnability in Second Language Pragmatics.” In, Pragmatics across Languages and Cultures, ed. by Anna Trosborg, 391–423. Berlin, Germany: De Gruyter Mouton.Google Scholar
Turnbull, William
2001 “An Appraisal of Pragmatic Elicitation Techniques for the Social Psychological Study of Talk: The Case of Request Refusals.” Pragmatics, 11(1): 31–61. CrossrefGoogle Scholar
Uso-Juan, Esther
2010 “Requests: A Sociopragmatic Approach.” In Speech Act Performance: Theoretical, Empirical and Methodological Issues ed. by Alicia Martinez-Flor and Esther Uso-Juan, 237–256. Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar
Uso-Juan, Esther, and Alicia Martinez-Flor
2014 “Reorienting the Assessment of the Conventional Expressions of Complaining and Apologizing: From Single Response to Interactive DCTs.” Iranian Journal of Language Testing, 4(1): 113–136.Google Scholar
Woodfield, H.
2012 “I Think Maybe I Want to Lend the Notes from You: Development of Request Modification in Graduate Learners.” In Interlanguage Request Modification, ed. by Maria Economidou-Kogetsidis and Helen Woodfield, 9–49. Amsterdam, Netherlands: John Benjamins. CrossrefGoogle Scholar
Yuan, Y.
2001 “An Inquiry into Empirical Pragmatics Data Gathering Methods: Written DCTs, Oral DCTs, and Natural Conversations.” Journal of pragmatics, 33: 271–92. CrossrefGoogle Scholar

Appendix A.Scenarios for WDCT (Borrowed from Jalilifar 2009Jalilifar, Alireza 2009 “Request Strategies: Cross-Sectional Study of Iranian EFL Learners and Australian Native Speakers.” English Language Teaching, 2(1): 46–61. CrossrefGoogle Scholar and Schauer 2009Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar)

Scenario 1.

For registration you need to fill out a couple of forms. You search all of your pockets and cannot find a pen. You want to ask another student who is sitting next to you in the department hall. What would you say?

You say:

Scenario 2.

You are attending a seminar. The sun is shining into the classroom and it is very hot. A friend of yours is sitting next to the window. You turn to your friend and ask him to open it.

You say:

Scenario 3.

You are attending a seminar. The professor is explaining a new concept, but you cannot hear her very well. You ask her to speak louder.

You say:

Scenario 4.

You have to hand in an essay to the secretary. The secretary’s office is closing soon and you are already running late. When you get to her office, two professors are standing in front of it. You ask them to let you through.

You say:

Appendix B.Scenarios for ODCT (Borrowed from Jalilifar 2009Jalilifar, Alireza 2009 “Request Strategies: Cross-Sectional Study of Iranian EFL Learners and Australian Native Speakers.” English Language Teaching, 2(1): 46–61. CrossrefGoogle Scholar and Schauer 2009Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar)

Scenario 1.

You are going to visit your friend, who lives in the college dormitory. You are on the campus, but you don’t know where the room is. You are going to ask a student for the location of the dorm. How would you ask the student?

You say:

Scenario 2.

You are studying at home. Your younger brother opens the window and the cold wind blows right into your face and bothers you. You want to ask him to close it. What would you say?

You say:

Scenario 3.

It’s Sunday afternoon. You with your father are in the living room in your house watching TV. Your father has just stood up to make himself a cup of coffee. Since he has stood up, you want to ask him to get you the TV remote.

You say:

Scenario 4.

You are having trouble with your computer; it keeps crashing. You know your older neighbor knows a lot about computers and you ask her to help you even though the two of you don’t know each other well. What do you say?

You say:

Appendix C.Scenarios for Role-play (Borrowed from Jalilifar 2009Jalilifar, Alireza 2009 “Request Strategies: Cross-Sectional Study of Iranian EFL Learners and Australian Native Speakers.” English Language Teaching, 2(1): 46–61. CrossrefGoogle Scholar and Schauer 2009Schauer, Gila. A. 2009Interlanguage Pragmatic Development: The Study Abroad Context. London: Continuum.Google Scholar)

Scenario 1.

Your friend and you go to a restaurant to eat. You want to order and need to ask the waiter for the menu. What would you say?

Scenario 2.

You are trying to study in your room and hear loud music coming from another student’s room down the hall. You don’t know the student, but you decide to ask him/her to turn the music down. What would you say?

Scenario 3.

You are attending a seminar. It is a very sunny day and the classroom is hot. The professor is standing near the window. You ask him to open it.

Scenario 4.

You are in the corridor of you department. Your next seminar is taking place in the Trent building, but you don’t know where the trend building is. One of your professors, Professor Jones, is walking down the corridor towards you. You ask him for the directions to the trend building.

Address for correspondence

Rasoul Mohammad Hosseinpur

University of Qom

P.O. box 37185-396

Qom

Iran

rmhosseinpur@gmail.com

Biographical notes

Rasoul Mohammad Hosseinpur is an assistant professor of TEFL at University of Qom, Iran. He has published several articles in ELT-related journals. His research areas include second language writing pedagogy, interlanguage pragmatics, and language assessment.

Reza Bagheri Nevisi is an assistant professor of TEFL at University of Qom, Iran. His research interests include teaching methodologies, language assessment, and interlanguage pragmatics. He has published different articles in ELT- related journals.

Abdolreza Lowni is an MA Graduate from University of Qom, Iran. He is mainly interested in pragmatics-related research.