Edited by Naoko Taguchi and YouJin Kim
[Task-Based Language Teaching 10] 2018
► pp. 218–246
Chapter 9Task design and validity evidence for assessment of L2 pragmatics in interaction
This study examines how clear communicative goals and authentic interaction are ensured in task-based pragmatic assessment practices, particularly in designing role-play assessment tasks and developing task-appropriate rating criteria. Employing a concurrent mixed methods design, conversation analysis (CA) and multi-faceted Rasch measurement were used to investigate whether task-independent interactional features are elicited from role-play interactions and how interaction-specific rating categories function quantitatively. The CA findings indicated that various interactional organizations emerged from role-play task interactions as examinees oriented to the different degree of imposition and contextual variables embedded in the role-plays. Further, the interactional features included in the rating criteria created a substantial amount of variance in distinguishing varying degrees of pragmatic performance. The findings are discussed in terms of designing valid pragmatic assessment tasks and what features need to be included in developing rating criteria to ensure the validity evidence of task-based pragmatic assessment.
Article outline
- Introduction
- Background
- Assessing pragmatic competence in interaction and validity challenges
- Role-plays as assessment tasks and eliciting validity evidence
- The present study
- Methods
- Participants
- Examinees
- Interlocutors
- Raters
- Test instruments
- Role-plays
- Rating criteria
- Procedures
- Test administration
- Rater training
- Rating
- Data analysis
- Conversation analysis
- Multi-faceted Rasch measurement
- Participants
- Results
- Conversation Analysis (CA) findings
- Recommendation letter request role-play
- Additional advising time request role-play
- Refusing a professor’s request role-play
- Deciding a meeting time
- Deciding a meeting mode
- Multi-faceted Rasch Measurement analysis
- Conversation Analysis (CA) findings
- Discussion
- Conclusion
-
References
https://doi.org/10.1075/tblt.10.09you
References
Cited by
Cited by 1 other publications
This list is based on CrossRef data as of 05 january 2023. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.