Cover not available
Article published In:
Interaction Studies
Vol. 25:1 (2024) ► pp.1–35

Exploring the construct of interactional competence in different types of oral communication assessment

ORCID logo | University of Foreign Language Studies–The University of Danang | The University of UtahSalt Lake City
Research on interaction in speaking assessment suggests that both verbal and nonverbal interaction are integral parts of the construct of interactional competence (Galaczi & Taylor, 2018; Plough et al., 2018; Young, 2011). However, little has been done to investigate which features significantly contribute to interactional competence scores. This study, therefore, examined which interaction features that raters noticed in individual scripted interview and paired discussion tasks to gain an insight into the interactional competence construct, providing validity evidence for an inclusion of interactional competence in speaking assessment. Sixty-eight student performances were rated based on interaction rating scales. Exploratory factor analysis revealed four factors: nonverbal communication, topic management, interactional management, and interactive listening. Logistic regressions showed that while raters attended to more topic management features in the individual scripted interview task, they noticed more interactional management features in the paired discussion task. Simple regressions showed that nonverbal communication and topic management features predicted interactional competence scores in the individual scripted interview task, whereas nonverbal communication, topic management, interactional management, and interactive listening features were predictors of scores in the paired discussion task. The findings suggest that both nonverbal and verbal interaction features are important in the interactional competence construct with the paired task providing test-takers with more opportunities to demonstrate their interactional ability.
Keywords: interactional competence, oral communication, interaction features, interview, paired discussion
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1Raters and test-takers’ perceptions toward IC
    • 2.2IC in different types of oral communication assessment
    • 2.3Methodology used in IC research
  • 3.Methods
    • 3.1Participants
    • 3.2Materials
      • 3.2.1Test-takers’ video-recorded performances
      • 3.2.2Interactional competence rating scale (ICS)
      • 3.2.3Interaction ability scale (IAS)
    • 3.3Procedures
      • 3.3.1Rater training
      • 3.3.2Rating
      • 3.3.3Data analysis
        • 3.3.3.1Data screening for RQ1
        • 3.3.3.2Data screening for RQ2
          • 3.3.3.2.1Data screening for the individual scripted interview task
          • 3.3.3.2.2Data screening for the paired discussion task
  • 4.Results
    • 4.1The extent to which the individual scripted interview task shared interaction features with the paired discussion task
      • 4.1.1Exploratory factor analysis
      • 4.1.2Logistic regression
        • 4.1.2.1Topic management (TM)
        • 4.1.2.2Nonverbal communication (NC)
        • 4.1.2.3Interactional management (IM)
        • 4.1.2.4Interactive listening (IL)
    • 4.2The extent to which the nonverbal and the verbal interaction features contribute to variances in the ICS scores in the two tasks
      • 4.2.1Individual scripted interview task
      • 4.2.2Paired discussion task
  • 5.Discussion
    • 5.1The extent to which the individual scripted interview task shared interaction features with the paired discussion task
    • 5.2The extent to which the interaction features contributed to variance in the ICS scores across the two tasks
  • 6.Limitations
  • 7.Conclusion
  • Acknowledgements
  • References
Published online: 7 June 2024
DOI logo
https://doi.org/10.1075/is.00022.vo
References (43)
References
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43 1, 561–573. DOI logoGoogle Scholar
Brooks, L. (2009). Interacting in pairs in a test of oral proficiency: Co-constructing a better performance. Language Testing, 26 (3), 342–366. DOI logoGoogle Scholar
Chapelle, C. A., Enright, M. K., & Jamieson, J. (Eds.). (2008). Building a validity argument for the Test of English as a Foreign Language. London, English: Routledge.Google Scholar
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Davies, A., Brown, A., Elder, C., Hill, K., Lumley, T., & McNamara, T. (1999). Dictionary of language testing. Cambridge University Press.Google Scholar
De Winter, J. C., Dodou, D., & Wieringa, P. A. (2009). Exploratory factor analysis with small sample sizes. Multivariate Behavioral Research, 44 (2), 147–181. DOI logoGoogle Scholar
Ducasse, A. M. (2014). Interaction in paired oral proficiency assessment in Spanish. Frankfurt/Main, Germany: Peter Lang.Google Scholar
Ducasse, A. M., & Brown, A. (2009). Assessing paired orals: Raters’ orientation to interaction. Language Testing, 26 1, 423–443. DOI logoGoogle Scholar
Eckes, T. (2015). Introduction to Many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments. Peter Lang Pub. Inc.Google Scholar
EPT Oral Communication Test. Retrieved on February 18, 2020 from [URL]
Fabrigar, L. R. (2012). Exploratory factor analysis: Understanding statistics. Oxford.Google Scholar
Galaczi, E. D. (2014). IC across proficiency levels: How do learners manage interaction in paired speaking tests? Applied Linguistics, 35 (5), 553–574. DOI logoGoogle Scholar
Galaczi, E. D., & Taylor, L. (2018). IC: Conceptualisations, operationalisations, and outstanding questions. Language Assessment Quarterly, 15 (3), 219–236. DOI logoGoogle Scholar
He, L., & Dai, Y. (2006). A corpus-based investigation into the validity of the CET–SET group discussion. Language Testing, 23 1, 370–401. DOI logoGoogle Scholar
Heinze, G., & Schemper, M. (2002). A solution to the problem of separation in logistic regression. Statistics in Medicine, 21 (16), 2409–2419. DOI logoGoogle Scholar
Kline, R. (2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford.Google Scholar
Kramsch, C. (1986). From language proficiency to IC. Modern Language Journal, 70 (4), 366–372. DOI logoGoogle Scholar
Lam, D. M. K. (2018). What counts as ‘responding’? Contingency on previous speaker contribution as a feature of IC. Language Testing, 35 (3), 377–401. DOI logoGoogle Scholar
Leech, N. L., Barrett, K. C., & Morgan, G. A. (2011). IBM SPSS for intermediate statistics: Use and interpretation (4th ed.). New York, NY: Taylor & Francis.Google Scholar
Linacre, J. M. (2014). Facets (3.71.4) [Computer software]. [URL]. [URL]
(1994). Many-facet Rasch measurement (2nd ed.). MESA Press.Google Scholar
Matsumoto, D. (2006). Culture and nonverbal behavior. In V. L. Manusov & M. L. Patterson (Eds.), The SAGE handbook of nonverbal communication (pp. 219–235). Sage Publications, Inc.. DOI logoGoogle Scholar
May, L. (2011). Interactional competence in a paired speaking test: Features salient to raters. Language Assessment Quarterly, 8 (2),127–145. DOI logoGoogle Scholar
McNeill, D. (1985). So you think gestures are nonverbal? Psychological Review, 92 1, 350–371. DOI logoGoogle Scholar
Ockey, G. J. (2014). The potential of the L2 group oral to elicit discourse with a mutual contingency pattern and afford equal speaking rights in an ESP context. English for Specific Purposes, 35 1, 17–29. DOI logoGoogle Scholar
(2013). Exploratory factor analysis and structural equation modeling. In A. J. Kunnan (Ed.), The companion to language assessment. Vol. III: Evaluation, Methodolody, and Interdisciplinary Themes (pp. 1224–1244, Part 101, Chapter 73). Malden, MA: John Wiley & Sons. DOI logoGoogle Scholar
Ockey, G. J., & Chukharev-Hudilainen, E. (2021). Human versus computer partner in the paired oral discussion test. Applied Linguistics, 42(5), 924–944. DOI logoGoogle Scholar
Ockey, G. J., & Li, Z. (2015). New and not so new methods for assessing oral communication. Language Value, 71, 1–21. DOI logoGoogle Scholar
Ockey, G. J., & Wagner, E. (2018). Assessing L2 listening: Moving towards authenticity, vol. 50, John Benjamins, Philadelphia, PA. DOI logoGoogle Scholar
O’Sullivan, B., Weir, C. J., & Saville, N. (2002). Using observation checklists to validate speaking test tasks. Language Testing, 19 1, 33–56. DOI logoGoogle Scholar
Pedhazur, E. J. (1997). Multiple regression in behavioral research: Explanation and prediction. Belmont, Ca.: Wadsworth.Google Scholar
Plough, I., Banerjee, J., & Iwashita, N. (2018). Interactional competence: Genie out of the bottle. Language Testing, 35 (3), 427–445. DOI logoGoogle Scholar
Roever, C., & Kasper, G. (2018). Speaking in turns and sequences: IC as a target construct in testing speaking. Language Testing, 35 (2), 331–355. DOI logoGoogle Scholar
Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics. 4th Edition, Allyn and Bacon, Boston.Google Scholar
Van Lier, L. (1989). Reeling, writhing, drawling, stretching, and fainting in coils: Oral proficiency interviews as conversation. TESOL Quarterly, 23 1, 489–508. DOI logoGoogle Scholar
Van Moere, A. (2007). Group oral test: How does task affect candidate performance and test score? (Unpublished Ph.D. Dissertation). The University of Lancaster.
Vo, S. (2021). Evaluating interactional competence in interview and paired discussion tasks: A rater cognition study. TESOL Journal, 12 (2), 1–18. DOI logoGoogle Scholar
Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8 (3), 370. [URL]
Youn, S. J. (2020). Interactional features of L2 pragmatic interaction in role-play speaking assessment. TESOL Quarterly, 54 (1), 201–233. DOI logoGoogle Scholar
(2015). Validity argument for assessing L2 pragmatics in interaction using mixed methods. Language Testing, 32 (2), 199–225. DOI logoGoogle Scholar
Young, R. (2011). Interactional competence in language learning, teaching, and testing. In E. Hinkel (Ed.), Handbook of Research in Second Language Teaching and Learning (pp. 426–443). London, UK & New York, NY: Routledge.Google Scholar
Yuki, M., Maddux, W. W., & Masuda, T. (2007). Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology, 43 (2), 303–311. DOI logoGoogle Scholar