Evaluating locally-developed language testing
A predictive study of ‘direct entry’ language programs at an Australian University
Nicholas Cope | Centre for Macquarie English, Macquarie University
The study reported here investigates the predictive validity of language assessments by ‘Direct Entry’ programs at an Australian University – programs developed on site for Non English Speaking Background international students, principally to provide (i) pre-entry academic and language preparation and (ii) language assessment for university admissions purposes. All 138 students in the sample had entered degree studies via one of the three programs that made up the locally-developed Direct Entry pathway. Inferential statistics (correlation and regression) showed the assessments awarded by two programs to satisfactorily predict academic outcomes, while predictive validity for one was not demonstrated. Descriptive statistics (mean pass rates and academic averages) then revealed a pattern of relatively poor academic performance in certain university disciplines to which particular Direct Entry programs were dedicated. Informed by principles of language program evaluation, the study’s outcomes were seen as both summative and formative: remedial strategies are accordingly recommended. While the specific relevance of the study’s findings is to the particular institutional context in which the study was conducted, the study instantiates a perspective on language assessment validation of broader relevance in an Australian context where locally-developed Direct Entry programs – about which the research literature is largely silent – are increasingly widespread.
References
Alderson, J., Clapham, C. & Wall, D.
(
1995)
Language test construction and evaluation. Cambridge: CUP.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bachman, L.
(
1990)
Fundamental considerations in language testing. Oxford: OUP.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Bellingham, L.
(
1993)
The relationship of language proficiency to academic success of international students.
New Zealand Journal of Educational Studies, 30(2), 229–232.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Black, J.
(
1991)
Performance in English skills courses and overall academic achievement.
TESL Canada Journal, 9(1), 42–53.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Burns, R.
(
1997)
Introduction to research methods. Melbourne: Longman.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cotton, F. & Conrow, F.
(
1995)
An investigation of the predictive validity of IELTS amongst a group of international students studying at the University of Tasmania. IELTS Research Reports (Vol. 11). Canberra: IELTS Australia.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Dooey, P. & Oliver, R.
(
2002)
An investigation into the predictive validity of the IELTS test as an indicator of future academic success.
Prospect, 17(1), 36–54.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Elder, C.
(
1993)
Language proficiency as a predictor of performance in teacher education.
Melbourne Papers in Applied Linguistics, 2(1), 68–85.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Feast, V.
(
2002)
The impact of IELTS scores on performance at university.
International Education Journal, 3(4), 70–85.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ferguson, G. & White, E.
(
1998)
A predictive study of IELTS. Institute for Applied Language Studies, University of Edinburgh.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Graham, J.
(
1987)
English language proficiency and the prediction of academic success.
TESOL Quarterly, 21(3), 505–521.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hale, G., Stansfield, C. & Duran, R.
(
1984)
Summaries of studies involving the Test of English as a Foreign Language, 1963—1982 (Report no. RR-84-03, TOEFL-RR-16). Princeton, N. J.: Educational Testing Service.
![DOI logo](https://benjamins.com/logos/doi-logo.svg)
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Hill, K., Storch, N. & Lynch, B.
(
1999)
A comparison of IELTS and TOEFL as predictors of academic performance. IELTS Research Reports (Vol. 21). Canberra: IELTS Australia.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Huong, T.
(
2001)
The predictive validity of the International English Language Testing System (IELTS) test.
Post-Script, 2(1), 66–94.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Ingram, D. & Bayliss, A.
(
2007)
IELTS as a predictor of academic language performance, Part 1. IELTS Research Reports (Vol. 71). Canberra: IELTS Australia.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Kerstjens, M. & Nery, C.
(
2000)
Predictive validity in the IELTS test: A study of the relationship between IELTS scores and students’ subsequent academic performance. IELTS Research Reports (Vol. 31). Canberra: IELTS Australia.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lynch, B.
(
1996)
Language program evaluation: Theory and practice. Cambridge: Cambridge University Press.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lynch, T.
(
1994)
The University of Edinburgh Test of English at Matriculation: Validation report.
Edinburgh Working Papers in Applied Linguistics, No. 51. Scotland: Edinburgh University.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Lynch, T.
(
2000)
An evaluation of the revised Test of English at Matriculation at the University of Edinburgh.
Edinburgh Working Papers in Applied Linguistics, 101, 61–71.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Macquarie University
(
2002)
Macquarie University handbook of undergraduate studies. Sydney: Macquarie University.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
McGovern, P. & Walsh, S.
(Eds.) (
2007)
IELTS research reports (Vol. 71). Canberra: IELTS Australia.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Tulloh, R.
(Ed.) (
1999)
IELTS research reports (Vol. 21). Canberra: IELTS Australia.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Tulloh, R.
(Ed.) (
2000)
IELTS research reports (Vol. 31). Canberra: IELTS Australia.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Wood, S.
(Ed.) (
1998)
IELTS research reports (Vol. 11). Sydney: ELICOS Association.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Woodrow, L.
(
2006)
Academic success of international postgraduate education students and the role of English proficiency.
University of Sydney Papers in TESOL, 11, 51–70.
![Google Scholar](https://benjamins.com/logos/google-scholar.svg)
Cited by
Cited by 2 other publications
O'Dwyer, John, Elif Kantarcıoğlu & Carole Thomas
2018.
An Investigation of the Predictive Validity of the TOEFL iBT® Test at an English‐Medium University in Turkey.
ETS Research Report Series 2018:1
► pp. 1 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
Oliver, Rhonda, Honglin Chen & Stephen Moore
2016.
Review of selected research in applied linguistics published in Australia (2008–2014).
Language Teaching 49:4
► pp. 513 ff.
![DOI logo](//benjamins.com/logos/doi-logo.svg)
This list is based on CrossRef data as of 18 may 2024. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers.
Any errors therein should be reported to them.