To automated generation of test questions on the basis of error annotations in EFL essays

A time-saving tool?

Olga Vinogradova | National Research University Higher School of Economics, Moscow

The paper presents computer tools that were designed to increase the pedagogical value of a learner corpus for teachers and learners of English as a foreign language. The corpus used in the study is a collection of essays written by Russian learners of English – students from four different cities in Russia, studying at different Bachelor programmes at the Higher School of Economics. The paper describes how the tools we offer make use of manually annotated errors for the process of automated generation of questions for English tests, what editing the generated tests requires, and in the final section there is a discussion to what degree this process appears to be useful for EFL instructors.

Published online: 06 May 2019
https://doi.org/10.1075/scl.92.03vin

References

Bakota, Tibor, Beszédes, Árpád, Gergely, Tamás, Gyalai, Milán Imre, Gyimóthy, Tibor & Füleki, Dániel

2009 Semi-automatic test case generation from business process models. In Proceedings of the 11th Symposium on Programming Languages and Software Tools and 7th Nordic Workshop on Model Driven Software Engineering, 5–18. Tampere, Finland: Tampere University of Technology. https://pdfs.semanticscholar.org/1037/6b92645ba824a4ef5815b4a3e16d3287bcfc.pdf (15 February 2018).

Bayerl, Petra S.

2007 Identifying sources of disagreement: Generalizability theory in manual annotation studies. Computational Linguistics 33(1): 3–8.

Fort, Karën, Adda, Gilles & Cohen, K. Bretonnel

2011 Amazon mechanical turk: Gold mine or coal mine? Computational Linguistics 37(2): 413–420.

Gerasimenko, Ekaterina

2018 Automated Error Detection in English Examination Essays Written by Russian Students. BA dissertation, School of Linguistics, Research University Higher School of Economics, Moscow.

Glaznieks, Aivars, Nicolas, Lionel, Stemle, Egon, Abel, Andrea & Lyding, Verena

2014 Establishing a standardised procedure for building learner corpora. Journal of Applied Language Studies 8(3): 5–20.

Granger, Sylviane

2003 The International Corpus of Learner English: A new resource for foreign language learning and teaching and second language acquisition research. TESOL Quarterly 37(3): 538–546.

2004 Computer learner corpus research: Current status and future prospects. In Applied Corpus Linguistics: A Multidimensional Perspective, Ulla Connor & Thomas A. Upton (eds), 123–145. Amsterdam: Rodopi.

Gierl, Mark J. & Haladyna, Thomas M.

2012 Automatic Item Generation: Theory and Practice. New York NY: Routledge.

Heilman, Michael

2011 Automatic Factual Question Generation from Text. PhD dissertation, Carnegie Mellon University. CMU-LTI-11-004. https://lti.cs.cmu.edu/sites/default/files/research/thesis/2011/michael_heilman_automatic_factual_question_generation_for_reading_assessment.pdf (25 February 2018).

Hoshino, Ayako & Nakagawa, Hiroshi

2007 Assisting cloze test making with a web application. In Proceedings of SITE 2007--Society for Information Technology & Teacher Education International Conference, Roger Carlsen, Karen McFerrin, Jerry Price, Roberta Weber & Dee Anna Willis (eds), 2807–2814. Chesapeake VA: AACE. http://www.r.dl.itc.u-tokyo.ac.jp/~hoshino/publications/site2007.pdf (25 February 2018).

Hovy, Eduard & Lavid, Julia

2010 Towards a ‘Science’ of corpus annotation: A new methodological challenge for corpus linguistics. International Journal of Translation 22(1): 13–36.

Huddleston, Rodney & Pullum, Geoffrey K.

2002 The Cambridge Grammar of the English Language. Cambridge: CUP.

Leech, Geoffrey

2015 Adding linguistic annotation. In Developing Linguistic Corpora: A Guide to Good Practice. Oxford: Oxbow Books.

Lyashevskaja, Olga & Plungian, Vladimir

2003 Morphological annotation in Russian National Corpus: A theoretical feedback. In 5th International Conference on Formal Description of Slavic Languages (FDSL-5). Frankfurt: Peter Lang.

Plugian, Vladimir

2005 Zachem nuzhen Natsional’ny korpus russkogo yazyka? Neformal’noye vvedeniye (Why do we need Russian National Corpus? Informal introduction). In Russian National Corpus: 2003–2005, 6–20. Moscow: Indrik. http://ruscorpora.ru/sbornik2005/02plu.pdf (25 February 2018).

Proshina, Zoya G. & Eddy, Anna A.

2017 Russian English: History, Functions, and Features. Cambridge: CUP.

Saraceni, Mario

2015 World Englishes: A Critical Analysis. London: Bloomsbury.

Vinogradova, Olga

2016 The role and applications of expert error annotation in a corpus of English learner texts. In Computational Linguistics and Intellectual Technologies (Proceedings of International Conference “Dialogue 2016”) issue 15(22), 740–751.

Vinogradova, Olga, Lyashevskaya, Olga & Panteleeva, Irina

2017 Multi-level student essay feedback in a learner corpus. In Computational Linguistics and Intellectual Technologies (Papers from the Annual conference “Dialogue”) Issue 16(23), Vol. 1, 382–396. Moscow: RSUH.

Vinogradova, Olga & Login, Nikita

2017 The design of tests with multiple choice questions automatically generated from essays in a learner corpus. Higher School of Economics Research Paper No. WP BRP 60/LNG/2017, 13 December 2017 <https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3087215 (25 February 2018).