Quantitative research methods and study quality in learner corpus research
This study aims to provide the first empirical assessment of quantitative research methods and study quality in learner corpus
research. We systematically review quantitative primary studies referenced in the Learner Corpus Bibliography
(LCB), a representative bibliography of learner corpus research maintained by the Learner Corpus Association
which contained 1,276 references when the current study began. Each primary study in the LCB was coded for over fifty features
representing six dimensions: (a) publication type (i.e. conference paper, book chapter, journal article), (b) research focus (e.g.
lexis, grammar), (c) methodological features (e.g. keyword analysis, error analysis, use of reference corpus), (d) statistical
analyses (e.g. X², t-test, regression analysis), and (e) reporting practices (e.g. reliability coefficients,
means). Results point to several systematic strengths as well as many flaws, such as the absence of research questions, incomplete
and inconsistent reporting practices (e.g. means without standard deviations), and lack of statistical literacy (i.e. LCR studies
generally overrely on tests of statistical significance, do not report effect sizes, rarely check or report whether statistical
assumptions have been met, and rarely use multivariate analyses). Improvements over time, however, are clearly noted and there are
signs that, like other related disciplines, learner corpus research is slowly undergoing methodological reform.
Keywords: study quality, quantitative research methods, reporting practices, research synthesis
This article is currently available as a sample article.