Article published in:Variation within and across Romance Languages: Selected papers from the 41st Linguistic Symposium on Romance Languages (LSRL), Ottawa, 5–7 May 2011
Edited by Marie-Hélène Côté and Eric Mathieu
[Current Issues in Linguistic Theory 333] 2014
► pp. 401–416
French oral proficiency assessment
Elicited imitation with speech recognition
Testing oral proficiency is an important but challenging part of foreign language assessment: currently accepted methods are time-consuming and expensive. Our recent efforts have led to the implementation of new assessment methods for English (Graham et al. 2008), Japanese, and Spanish. We demonstrate that this work is also relevant for French oral proficiency testing. We begin by describing and historically situating an oral proficiency testing methodology called elicited imitation (EI). Then, we document the development, implementation and evaluation of a French EI test. We also detail the incorporation of the use of automatic speech recognition to score French EI items. Last, we substantiate with correlation analyses that carefully engineered, automatically scored French EI items correlate to a high degree with human scoring.
Published online: 17 December 2014
Bernstein, Jared, Alistair Van Moere & Jian Cheng
Bley-Vroman, Robert & Craig Chaudron
Chaudron, Craig, Matthew Prior & U. Kozok
2005 “Elicited Imitation as an Oral Proficiency Measure”. Paper presented to the 14th World Congress of Applied Linguistics, Madison, Wisc., July 2005.
Christensen, Carl, Ross Hendrickson & Deryle Lonsdale
2010 “Principled Construction of Elicited Imitation Tests”. Proceedings of the 7th Conference on Language Resources and Evaluation (LREC’10) , 233–238.
Connell, Phil J. & Catherine Myles-Zitzer
Daelemans, Walter & Antal van den Bosch
De Calmès, Martine & Guy Pérennou
1998 “BDLEX: A lexicon for spoken and written French”. Proceedings of the 1st International Conference on Language Resources & Evaluation , 1129–1136.
Deléglise, Paul, Yannick Estève, Sylvain Meignier & Teva Merlin
2009 “Improvements to the LIUM French ASR System Based on CMU Sphinx: What helps to significantly reduce the word error rate?”. Proceedings of INTERSPEECH-2009 , 2123–2126.
Graham, C. Ray, Troy Cox, Jeremiah McGhee & Matthew LeGare
2011 “Examining the Validity of an Elicited Imitation Instrument to Test Oral Language in Spanish”. Paper presented at the Language Testing Research Colloquium (LTRC), University of Michigan, June 2011.
Graham, C. Ray, Deryle Lonsdale, Casey Kennington, Aaron Johnson & Jeremiah McGhee
2008 “Elicited Imitation as an Oral Proficiency Measure with ASR Scoring”. Proceedings of the 6th International Language Resources and Evaluation Conference (LREC’08) ed. by Nicoletta Calzolari, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis & Daniel Tapias, 1604–1610. ELRA.
Graham, C. Ray, Jeremiah McGhee & Benjamin Millard
Hood, Lois & Patsy Lightbrown
Klein, Dan & Christopher D. Manning
2003 “A* parsing: Fast exact viterbi parse selection”. Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology . [http://acl.ldc.upenn.edu/N/N03/N03-1016.pdf]
Lamere, Paul, Philip Kwok, Evandro B. Gouvêa, Bhiksha Raj, Rita Singh, William Walker & Peter Wolf
2003 The CMU SPHINX-4 speech recognition system.
Levy, Roger & Galen Andrew
2006 “Tregex and Tsurgeon: Tools for querying and manipulating tree data structures”. 5th International Conference on Language Resources and Evaluation (LREC 2006) .
Liskin-Gasparro, Judith E.
Lonsdale, Deryle & Carl Christensen
2011 “Automating the Scoring of Elicited Imitation Tests”. Proceedings of the ACL-HLT/ICML/ISCA Joint Symposium on Machine Learning in Speech and Language Processing . [http://www.ttic.edu/sigml/symposium2011/papers/Lonsdale+Christensen_Imitation.pdf]
Lonsdale, Deryle, Dan Dewey, Jerry McGhee, Aaron Johnson & Ross Hendrickson
2009 “Methods of Scoring Elicited Imitation Items: An empirical study”. Paper presented at the annual conference of the American Association for Applied Linguistics, Denver, Col., March 2009.
Lonsdale, Deryle & Yvon LeBras
Matsushita, Hitokazu, Deryle Lonsdale & Dan Dewey
Mendonça, Ângelo, David Graff & Denise DiPersio
Müller, Pieter F.
2010 Automatic Oral Proficiency Assessment of Second Language Speakers of South African English. Master’s thesis, Stellenbosch University.
New, Boris, Christophe Pallier, Marc Brysbaert & Ludovic Ferrand
Radloff, Carla F.
1994: “Probabilistic Part-of-Speech Tagging Using Decision Trees”. Proceedings of the International Conference on New Methods in Language Processing , Manchester, UK.
Slobin, Dan I. & Charles A. Welsh
Spitze, Kimett & Susan D. Fischer
Tomita, Yasuyo, Wataru Suzuki & Lorena Jessop
2007 “Fips, a “Deep” Linguistic Multilingual Parser”. Proceedings of the ACL 2007 Workshop on Deep Linguistic Processing. Association for Computational Linguistics , 120–127. Prague, Czech Republic.