Cover not available
Part of
Variation within and across Romance Languages: Selected papers from the 41st Linguistic Symposium on Romance Languages (LSRL), Ottawa, 5–7 May 2011
Edited by Marie-Hélène Côté and Eric Mathieu
[Current Issues in Linguistic Theory 333] 2014
► pp. 401–416

French oral proficiency assessment

Elicited imitation with speech recognition

| Brigham Young University
| Brigham Young University
Testing oral proficiency is an important but challenging part of foreign language assessment: currently accepted methods are time-consuming and expensive. Our recent efforts have led to the implementation of new assessment methods for English (Graham et al. 2008), Japanese, and Spanish. We demonstrate that this work is also relevant for French oral proficiency testing. We begin by describing and historically situating an oral proficiency testing methodology called elicited imitation (EI). Then, we document the development, implementation and evaluation of a French EI test. We also detail the incorporation of the use of automatic speech recognition to score French EI items. Last, we substantiate with correlation analyses that carefully engineered, automatically scored French EI items correlate to a high degree with human scoring.
Published online: 17 December 2014
DOI logo
https://doi.org/10.1075/cilt.333.26mil
References (33)
References
Bernstein, Jared, Alistair Van Moere & Jian Cheng. 2010. “Validating Automated Speaking Tests”. Language Testing 27.355–377 DOI logoGoogle Scholar
Bley-Vroman, Robert & Craig Chaudron. 1994. “Elicited Imitation as a Measure of Second-Language Competence”. Research Methodology in Second-Language Acquisition ed. by Elaine E. Tarone, Susan M. Gass & Andrew D. Cohen, 245–261. Northvale N.J.: L. Erlbaum.Google Scholar
Chaudron, Craig, Matthew Prior & U. Kozok. 2005. “Elicited Imitation as an Oral Proficiency Measure”. Paper presented to the 14th World Congress of Applied Linguistics, Madison, Wisc., July 2005.
Christensen, Carl, Ross Hendrickson & Deryle Lonsdale. 2010. “Principled Construction of Elicited Imitation Tests”. Proceedings of the 7th Conference on Language Resources and Evaluation (LREC’10) , 233–238.
Connell, Phil J. & Catherine Myles-Zitzer. 1982. “An Analysis of Elicited Imitation as a Language Evaluation Procedure”. Journal of Speech and Hearing Disorders 47.390–396. DOI logoGoogle Scholar
Daelemans, Walter & Antal van den Bosch. 2005. Memory-Based Language Processing. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
De Calmès, Martine & Guy Pérennou. 1998. “BDLEX: A lexicon for spoken and written French”. Proceedings of the 1st International Conference on Language Resources & Evaluation , 1129–1136.
Deléglise, Paul, Yannick Estève, Sylvain Meignier & Teva Merlin. 2009. “Improvements to the LIUM French ASR System Based on CMU Sphinx: What helps to significantly reduce the word error rate?”. Proceedings of INTERSPEECH-2009 , 2123–2126.
Erlam, Rosemary. 2006. “Elicited Imitation as a Measure of L2 Implicit Knowledge: An empirical validation study”. Applied Linguistics 27.464–491. DOI logoGoogle Scholar
Graham, C. Ray, Troy Cox, Jeremiah McGhee & Matthew LeGare. 2011. “Examining the Validity of an Elicited Imitation Instrument to Test Oral Language in Spanish”. Paper presented at the Language Testing Research Colloquium (LTRC), University of Michigan, June 2011.
Graham, C. Ray, Deryle Lonsdale, Casey Kennington, Aaron Johnson & Jeremiah McGhee. 2008. “Elicited Imitation as an Oral Proficiency Measure with ASR Scoring”. Proceedings of the 6th International Language Resources and Evaluation Conference (LREC’08) ed. by Nicoletta Calzolari, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis & Daniel Tapias, 1604–1610. ELRA.Google Scholar
Graham, C. Ray, Jeremiah McGhee & Benjamin Millard. 2010. “The Role of Lexical Choice in Elicited Imitation Item Difficulty”. Selected Proceedings of the 2008 Second Language Research Forum ed. by Matthew T. Prior, et al., 57–72. Somerville, Mass.: Cascadilla Proceedings Project.Google Scholar
Hood, Lois & Patsy Lightbrown. 1978. “What Children Do When Asked to “Say What I Say”: Does elicited imitation measure linguistic knowledge:. Allied Health and Behavioral Sciences 1:2.195–219.Google Scholar
Klein, Dan & Christopher D. Manning. 2003. “A* parsing: Fast exact viterbi parse selection”. Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology . [[URL]]
Lamere, Paul, Philip Kwok, Evandro B. Gouvêa, Bhiksha Raj, Rita Singh, William Walker & Peter Wolf. 2003. The CMU SPHINX-4 speech recognition system.
Levy, Roger & Galen Andrew. 2006. “Tregex and Tsurgeon: Tools for querying and manipulating tree data structures”. 5th International Conference on Language Resources and Evaluation (LREC 2006) .
Liskin-Gasparro, Judith E. 1982. Educational Testing Service Oral Proficiency Testing Manual. Princeton, N. J.: Educational Testing Service.Google Scholar
Lonsdale, Deryle & Carl Christensen. 2011. “Automating the Scoring of Elicited Imitation Tests”. Proceedings of the ACL-HLT/ICML/ISCA Joint Symposium on Machine Learning in Speech and Language Processing . [[URL]]
Lonsdale, Deryle, Dan Dewey, Jerry McGhee, Aaron Johnson & Ross Hendrickson. 2009. “Methods of Scoring Elicited Imitation Items: An empirical study”. Paper presented at the annual conference of the American Association for Applied Linguistics, Denver, Col., March 2009.
Lonsdale, Deryle & Yvon LeBras. 2009. A Frequency Dictionary of French: Core vocabulary for learners. New York: Routledge.Google Scholar
Luoma, Sari. 2004. Assessing Speaking. Cambridge: Cambridge University Press. DOI logoGoogle Scholar
Matsushita, Hitokazu, Deryle Lonsdale & Dan Dewey. 2010. “Japanese Elicited Imitation: ASR-based oral proficiency test and optimal item creation”. Corpus, ICT, and Language Education ed. by George R.S. Weir & Shinichiro Ishikawa, 161–172. University of Strathclyde Publishing.Google Scholar
Mendonça, Ângelo, David Graff & Denise DiPersio. 2009. French Gigaword. 2nd ed. Philadelphia: Linguistic Data Consortium.Google Scholar
Müller, Pieter F. 2010. Automatic Oral Proficiency Assessment of Second Language Speakers of South African English. Master’s thesis, Stellenbosch University.
Naiman, Neil. 1974. “The Use of Elicited Imitation in Second Language Acquisition Research”. Working Papers on Bilingualism 2:1.1–37.Google Scholar
New, Boris, Christophe Pallier, Marc Brysbaert & Ludovic Ferrand. 2004. “Lexique 2: A new French lexical database”. Behavior Research Methods, Instruments, & Computers 36.516–524. DOI logoGoogle Scholar
Radloff, Carla F. 1991. Sentence Repetition Testing for Studies of Community Bilingualism. Dallas Summer Institute of Linguistics & the University of Texas at Arlington: Summer Institute of Linguistics & the University of Texas at Arlington Publications in Linguistics.Google Scholar
Schmid, Helmut. 1994: “Probabilistic Part-of-Speech Tagging Using Decision Trees”. Proceedings of the International Conference on New Methods in Language Processing , Manchester, UK.
Slobin, Dan I. & Charles A. Welsh. 1971. “Elicited Imitation as a Research Tool in Developmental Psycholinguistics”. Language Training in Early Childhood Education ed. by Celia B. Lavatelli, 170. Urbana: University of Illinois Press.Google Scholar
Spitze, Kimett & Susan D. Fischer. 1981. “Short-Term Memory as a Test of Language Proficiency”. ESL Talk 12:4.32–41.Google Scholar
Tomita, Yasuyo, Wataru Suzuki & Lorena Jessop. 2009. “Elicited Imitation: Toward valid procedures to measure implicit second language grammatical knowledge. TESOL Quarterly 43.345–350.Google Scholar
Wehrli, Eric. 2007. “Fips, a “Deep” Linguistic Multilingual Parser”. Proceedings of the ACL 2007 Workshop on Deep Linguistic Processing. Association for Computational Linguistics , 120–127. Prague, Czech Republic.
Zechner, Klaus, Derrick Higgins, Xiaoming Xi & David M. Williamson. 2009. “Automatic Scoring of Non-Native Spontaneous Speech in Tests of Spoken English”. Speech Communication 51.883–895. DOI logoGoogle Scholar