Using a corpus in creating and evaluating a DCT
Discourse Completion Tasks (DCTs) have been one of the most popular tools in pragmatics research. Yet, many have criticized DCTs for their lack of authenticity (e.g., Culpeper, Mackey, & Taguchi, 2018; Nguyen, 2019). We propose that corpora can serve as resources in designing and evaluating DCTs. We created a DCT using advice-seeking prompts from the Q+A corpus (Baker & Egbert, 2016). Then, we administered the DCT to 33 participants. We evaluated the DCT by (1) comparing the linguistic form and the semantic content of the participants’ DCT responses (i.e., advice-giving expressions) with authentic data from the corpus; and (2) interviewing the participants about the instrument quality. Chi-square tests between DCT data and corpus data revealed no significant differences in advice-giving expressions in terms of both the overall level of directness (χ2 [2, N = 660] = 6.94, p = .03, V = .10) and linguistic realization (χ2 [8, N = 660] = 17.75, p = .02, V = .16), and showed a significant difference but small effect size in terms of semantic content (χ2 [6, N = 512] = 30.35, p < .01, V = .24). Taken together with the interview data, our findings indicate that corpora are useful in designing DCTs.
Keywords: discourse completion task, corpus-based instrument, advice-giving, technology-mediated communication, authenticity, pragmatics
- 2.Literature review
- 2.2Criticism of DCTs
- 2.3Applications of corpora in instrument design and pragmatics research
- 2.4The speech act of advice-giving
- 3.The present study
- 4.3DCT instrument
- 4.4Comparison of advice-giving strategies between DCT and corpus
- 4.4.1Coding schemes for advice-giving strategies
- 4.4.2Statistical analysis of advice-giving strategies
- 4.5.1Interview procedures
- 4.5.2Thematic analysis of interview data
- 5.1Thematic analysis of interview data
- 5.2Statistical analysis of coded advice-giving strategies in the DCT and corpus data
- 6.1Similarities and differences between DCT elicited and naturalistic advice-giving strategies
- 6.2Designing a DCT with characteristics consistent with the target register to produce an authentic instrument
- 7.Implications for pragmatics research
- 7.2Future directions
Published online: 10 March 2020
Alcón, E., & Safont, P.
Bachman, L. F.
Bachman, L. F., & Palmer, A. S.
Baker, P., & Egbert, J.
Bardovi-Harlig, K., Mossman, S., & Su, Y.
Bardovi-Harlig, K., Mossman, S., & Vellenga, H. E.
Beebe, L. M., & Cummings, M. C.
Biber, D., Conrad, S., Reppen, R., Byrd, P., Helt, M., Clark, V., … Urzua, A.
Billmyer, K., & Varghese, M.
Boyatzis, R. E.
Braun, V., & Clarke, V.
Brown, P., & Levinson, S. C.
Canli, Z., & Canli, B.
Chapelle, C. A.
Cohen, A. D., & Olshtain, E.
Culpeper, J., Mackey, A., & Taguchi, N.
Cutrona, C. E. & Suhr, J. A.
DeCapua, A., & Dunham, J. F.
Flöck, I., & Geluykens, R.
Grabowski, K. C.
Halenko, N., & Jones, C.
Hartford, B. S., & Bardovi-Harlig, K.
Hong, C. Y., & Shih, S. C.
Johnston, B., Kasper, G., & Ross, S.
Kasper, G., & Dahl, M.
Martínez-Flor, A. M.
Nguyen, T. T. M.
Parvaresh, V., & Tavakoli, M.
Plonsky, L., & Oswald, F. L.
R Core Team
Schauer, G. A., & Adolphs, S.
Staples, S., & Fernández, J.
Cited by 3 other publications
Hashimoto, Brett, Daniel Keller, Ekaterina Sudina, Katherine Yaw, Jesse Egbert & Luke Plonsky
This list is based on CrossRef data as of 03 april 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.