Cover not available
Article published In:
Journal of Second Language Pronunciation
Vol. 4:1 (2018) ► pp.73–102

Pronunciation’s role in English speaking-proficiency ratings

| Brigham Young University
| Brigham Young University
| Brigham Young University
| Brigham Young University

Although pronunciation is an integral part of speaking, the role pronunciation plays in determining speaking-proficiency levels is unclear (Higgs & Clifford, 1982; Kang, 2013). To contribute to our understanding of this area, the research reported here investigated the relationship between English as a Second Language (ESL) learners’ pronunciation ability and their speaking-proficiency ratings. At an intensive English program (IEP) in the United States, a speaking test was administered to 223 ESL students. Their speaking proficiency was rated using an oral proficiency assessment based on standardized guidelines. In addition, their pronunciation was rated in six categories (vowels, consonants, word stress, sentence stress, intonation, and rhythm) by 11 raters using a rubric specifically developed and validated for this study. Many-Facet Rasch Measurement (MFRM) was used to estimate the students’ pronunciation ability, which was then compared to their speaking ability. The study found that sentence stress, rhythm, and intonation accounted for 41% of the variance in the speaking-proficiency test scores with sentence stress being the most powerful factor.

Keywords: ESL, English pronunciation, speaking proficiency, assessment, pronunciation rubric, suprasegmentals, sentence stress
Article outline
  • Pronunciation’s role in speaking proficiency tests
  • Literature review
  • Method
  • Setting
  • Procedure Phase One: Selecting and developing instruments
    • Speaking proficiency test
    • Pronunciation rubric development
  • Procedure Phase Two: Evaluating and adjusting the pronunciation rubric
  • Procedure Phase Three: Answering the research questions
    • Participants
    • Minimizing raters’ personal and language bias
    • Rater training
    • Pronunciation ratings
  • Data analysis
  • Results and discussion
  • Functionality and reliability of the rubric as a whole
  • The relationship between pronunciation and proficiency ratings
  • Relative influence of different pronunciation subskills
  • Conclusions, limitations, and implications
  • Limitations
  • Implications for future research
  • Conclusion
  • Note
  • References
Published online: 31 May 2018
DOI logo
https://doi.org/10.1075/jslp.00004.ma
References
American Council on the Teaching of Foreign Languages
(2012) ACTFL proficiency guidelines 2012. Retrieved from [URL]
British Council, IDP: IELTS Australia and Cambridge English Language Assessment. (n.d.)
IELTS speaking: Band descriptors (public version). Retrieved from [URL]
Brown, A.
(1988) Functional load and the teaching of pronunciation. TESOL Quarterly, 221, 593–606. DOI logoGoogle Scholar
Carey, M. D., Mannell, R. H., & Dunn, P. K.
(2011) Does a rater’s familiarity with a candidate’s pronunciation affect the rating in oral proficiency interviews? Language Testing, 281, 201–219. DOI logoGoogle Scholar
Catford, J. C.
(1987) Phonetics and the teaching of pronunciation: A systemic description of English phonology. In J. Morley (Ed.), Current perspectives on pronunciation: Practices anchored in theory (pp.83–100). Washington, DC: TESOL.Google Scholar
Dalton, P., & Hardcastle, W. J.
(1977) Disorders of fluency and their effects on communication. London: Edward Arnold.Google Scholar
Eckes, T.
(2011) Introduction to Many-facet Rasch Measurement: Analyzing and evaluating rater-mediated assessments. Frankfurt: Peter Lang. DOI logoGoogle Scholar
Educational Testing Service
(2006) Rubric for rubrics. Retrieved from [URL]
(2014) TOEFL iBT Test. Retrieved from [URL]
Hahn, L.
(2004) Primary stress and intelligibility: Research to motivate the teaching of suprasegmentals. TESOL Quarterly, 381, 201–223. DOI logoGoogle Scholar
Henrichsen, L. E., Green, B. A., Nishitani, A., & Bagley, C. L.
(1999) Pronunciation matters: Communicative, story-based activities for mastering the sounds of North American English. Ann Arbor: University of Michigan Press. (electronic version available at [URL])
Higgs, T. V., & Clifford, R.
(1982) The push toward communication. In T. V. Higgs (Ed.), Curriculum, competence and the foreign language teacher (pp. 51–79). Lincolnwood, IL: National Textbook.Google Scholar
Isaacs, T., & Trofimovich, P.
(2011) Phonological memory, attention control, and musical ability: Effects of individual differences on rater judgments of second language speech. Applied Psycholinguistics, 321, 113–140. DOI logoGoogle Scholar
(2012) Deconstructing comprehensibility: Identifying the linguistic influences on listeners’ L2 comprehensibility ratings. Studies in Second Language Acquisition, 341, 475–505. DOI logoGoogle Scholar
Jenkins, J.
(2006) Current perspectives on teaching world Englishes and English as a lingua franca. TESOL Quarterly, 401, 157–181. DOI logoGoogle Scholar
Kang, O.
(2013) Relative impact of pronunciation features on ratings of non-native speakers’ oral proficiency. In J. Levis & K. LeVelle (Eds.), Proceedings of the 4th Pronunciation in Second Language Learning and Teaching Conference (Aug. 2012 pp. 10–15). Ames, IA: Iowa State University.Google Scholar
Kang, O., Rubin, D., & Pickering, L.
(2010) Suprasegmental measures of accentedness and judgments of language learner proficiency in oral English. The Modern Language Journal, 941, 554–566. Retrieved from [URL]
Larson-Hall, J.
(2010) A guide to doing statistics in second language research using SPSS. London: Routledge.Google Scholar
Lee, Y. J.
(2012) Software to facilitate language assessment: Focus on Quest, Facets, and Turnitin. In C. Coombe, P. Davidson, B. O’Sullivan, & S. Stoynoff (Eds.), The Cambridge guide to second language assessment (pp. 280–288). New York, NY: Cambridge University Press.Google Scholar
Levis, J. M.
(2006) Pronunciation and the assessment of spoken language. In R. Hughes (Ed.), Spoken English, TESOL and applied linguistics: Challenges for theory and practice (pp. 245–270). New York, NY: Palgrave Macmillan. DOI logoGoogle Scholar
Lincare, J. M.
(2004) Optimizing rating scale category effectiveness. In E. V. Smith & R. M. Smith (Eds.), Introduction to Rasch measurement (pp.258–278). Maple Grove, MN: JAM Press.Google Scholar
Munro, M. J., & Derwing, T. M.
(2006) The functional load principle in ESL pronunciation instruction: An exploratory study. System, 341, 520–531. DOI logoGoogle Scholar
Myford, C. M., & Wolfe, E. W.
(2003) Detecting and measuring rater effects using Many-Facet Rasch Measurement: Part I. Journal of Applied Measurement, 4(4), 386–422.Google Scholar
(2004) Detecting and measuring rater effects using many-facet Rasch measurement: Part II. Journal of Applied Measurement, 5(2), 189–227.Google Scholar
Nicolosi, L., Harryman, E., & Keresheck, J.
(1989) Terminology of communication disorders. Baltimore, MD: William & Wilkins. DOI logoGoogle Scholar
Rose, M.
n.d.). Make room for rubrics. Retrieved from [URL]
Wennerstrom, A.
(1997) Discourse intonation and second language acquisition: Three genre based studies (Unpublished doctoral dissertation). University of Washington, Seattle.Google Scholar
Winke, P., Gass, S., & Myford, C.
(2012) Raters’ L2 background as a potential source of bias in rating oral performance. Language Testing, 301, 231–252. DOI logoGoogle Scholar
Cited by

Cited by 3 other publications

Lima, Edna F.
2020. The Supra Tutor. Journal of Second Language Pronunciation 6:1 ► pp. 39 ff. DOI logo
Lima, Edna F.
2023. A closer look at the Supra Tutor’s prosody lessons and the pedagogical and technological rationale behind them. Journal of Second Language Pronunciation 9:3 ► pp. 350 ff. DOI logo
Mroz, Aurore
2020. Aiming for advanced intelligibility and proficiency using mobile ASR. Journal of Second Language Pronunciation 6:1 ► pp. 12 ff. DOI logo

This list is based on CrossRef data as of 2 april 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.