Chapter published in:
Learner Corpora and Language Teaching
Edited by Sandra Götz and Joybrato Mukherjee
[Studies in Corpus Linguistics 92] 2019
► pp. 29–48

To automated generation of test questions on the basis of error annotations in EFL essays

A time-saving tool?

| National Research University Higher School of Economics, Moscow

The paper presents computer tools that were designed to increase the pedagogical value of a learner corpus for teachers and learners of English as a foreign language. The corpus used in the study is a collection of essays written by Russian learners of English – students from four different cities in Russia, studying at different Bachelor programmes at the Higher School of Economics. The paper describes how the tools we offer make use of manually annotated errors for the process of automated generation of questions for English tests, what editing the generated tests requires, and in the final section there is a discussion to what degree this process appears to be useful for EFL instructors.

Article outline
  • 1.Introduction
    • 1.1Previous research in corpus annotation
    • 1.2Description of the learner corpus and annotation practices adopted in it
  • 2.Benefits of error annotations in REALEC for teaching and learning English
    • 2.1Pedagogical directions in using a learner corpus in university EFL classes
    • 2.2REALEC English Test Maker: Related works, the main principles and stages of work of the program
    • 2.3REALEC English Test Maker: Special cases
    • 2.4RETM (REALEC English Test Maker): Editing automatically generated questions
  • 3.Discussion
  • Acknowledgements
  • Notes
  • References
Published online: 06 May 2019
https://doi.org/10.1075/scl.92.03vin
References
Bakota, Tibor, Beszédes, Árpád, Gergely, Tamás, Gyalai, Milán Imre, Gyimóthy, Tibor & Füleki, Dániel
2009 Semi-automatic test case generation from business process models. In Proceedings of the 11th Symposium on Programming Languages and Software Tools and 7th Nordic Workshop on Model Driven Software Engineering, 5–18. Tampere, Finland: Tampere University of Technology. https://​pdfs​.semanticscholar​.org​/1037​/6b92645ba824a4ef5815b4a3e16d3287bcfc​.pdf> (15 February 2018).
Bayerl, Petra S.
2007 Identifying sources of disagreement: Generalizability theory in manual annotation studies. Computational Linguistics 33(1): 3–8. CrossrefGoogle Scholar
Fort, Karën, Adda, Gilles & Cohen, K. Bretonnel
2011 Amazon mechanical turk: Gold mine or coal mine? Computational Linguistics 37(2): 413–420. CrossrefGoogle Scholar
Gerasimenko, Ekaterina
2018 Automated Error Detection in English Examination Essays Written by Russian Students. BA dissertation, School of Linguistics, Research University Higher School of Economics, Moscow.Google Scholar
Glaznieks, Aivars, Nicolas, Lionel, Stemle, Egon, Abel, Andrea & Lyding, Verena
2014 Establishing a standardised procedure for building learner corpora. Journal of Applied Language Studies 8(3): 5–20.Google Scholar
Granger, Sylviane
2003 The International Corpus of Learner English: A new resource for foreign language learning and teaching and second language acquisition research. TESOL Quarterly 37(3): 538–546. CrossrefGoogle Scholar
2004 Computer learner corpus research: Current status and future prospects. In Applied Corpus Linguistics: A Multidimensional Perspective, Ulla Connor & Thomas A. Upton (eds), 123–145. Amsterdam: Rodopi. CrossrefGoogle Scholar
Gierl, Mark J. & Haladyna, Thomas M.
2012 Automatic Item Generation: Theory and Practice. New York NY: Routledge. CrossrefGoogle Scholar
Heilman, Michael
2011 Automatic Factual Question Generation from Text. PhD dissertation, Carnegie Mellon University. CMU-LTI-11-004. https://​lti​.cs​.cmu​.edu​/sites​/default​/files​/research​/thesis​/2011​/michael​_heilman​_automatic​_factual​_question​_generation​_for​_reading​_assessment​.pdf> (25 February 2018).
Hoshino, Ayako & Nakagawa, Hiroshi
2007 Assisting cloze test making with a web application. In Proceedings of SITE 2007--Society for Information Technology & Teacher Education International Conference, Roger Carlsen, Karen McFerrin, Jerry Price, Roberta Weber & Dee Anna Willis (eds), 2807–2814. Chesapeake VA: AACE. http://​www​.r​.dl​.itc​.u​-tokyo​.ac​.jp​/~hoshino​/publications​/site2007​.pdf> (25 February 2018).
Hovy, Eduard & Lavid, Julia
2010 Towards a ‘Science’ of corpus annotation: A new methodological challenge for corpus linguistics. International Journal of Translation 22(1): 13–36.Google Scholar
Huddleston, Rodney & Pullum, Geoffrey K.
2002 The Cambridge Grammar of the English Language. Cambridge: CUP. CrossrefGoogle Scholar
Leech, Geoffrey
2015 Adding linguistic annotation. In Developing Linguistic Corpora: A Guide to Good Practice. Oxford: Oxbow Books.Google Scholar
Lyashevskaja, Olga & Plungian, Vladimir
2003 Morphological annotation in Russian National Corpus: A theoretical feedback. In 5th International Conference on Formal Description of Slavic Languages (FDSL-5). Frankfurt: Peter Lang.Google Scholar
Plugian, Vladimir
2005 Zachem nuzhen Natsional’ny korpus russkogo yazyka? Neformal’noye vvedeniye (Why do we need Russian National Corpus? Informal introduction). In Russian National Corpus: 2003–2005, 6–20. Moscow: Indrik. http://​ruscorpora​.ru​/sbornik2005​/02plu​.pdf> (25 February 2018).
Proshina, Zoya G. & Eddy, Anna A.
2017 Russian English: History, Functions, and Features. Cambridge: CUP.Google Scholar
Saraceni, Mario
2015 World Englishes: A Critical Analysis. London: Bloomsbury.Google Scholar
Vinogradova, Olga
2016 The role and applications of expert error annotation in a corpus of English learner texts. In Computational Linguistics and Intellectual Technologies (Proceedings of International Conference “Dialogue 2016”) issue 15(22), 740–751.Google Scholar
Vinogradova, Olga, Lyashevskaya, Olga & Panteleeva, Irina
2017 Multi-level student essay feedback in a learner corpus. In Computational Linguistics and Intellectual Technologies (Papers from the Annual conference “Dialogue”) Issue 16(23), Vol. 1, 382–396. Moscow: RSUH.Google Scholar
Vinogradova, Olga & Login, Nikita
2017 The design of tests with multiple choice questions automatically generated from essays in a learner corpus. Higher School of Economics Research Paper No. WP BRP 60/LNG/2017 13 December 2017 <https://​papers​.ssrn​.com​/sol3​/papers​.cfm​?abstract​_id​=3087215> (25 February 2018) Crossref