The French CAT
An assessment of its empirical validity
This paper investigates the empirical validity of the Monash-Melbourne computer adaptive test for French (French CAT), a single parameter Rasch model measurement of underlying morphosyntactic proficiency. It focuses, in particular, on the accuracy of the French CAT as a tool for streaming incoming university students into three levels of a first year (post high school) French course. Psychometric ability estimations of the Rasch model are compared against instructors’ assessment of students’ overall linguistic competence. A comparison is also made between the theoretical confidence interval of predicted abilities and the actual distribution of testee scores. Finally, individual student French CAT scores are correlated with end-of-semester language examination results. In all instances, Item Response Theory, upon which the French CAT is based, is shown to provide a highly valid means of determining linguistic ability for the purposes of course placement. Moreover, given the significant correlation between initial streaming and end-of-semester results, the French CAT is also demonstrated to be a good predictor of short-term achievement.
References (15)
Burston, J. (1993a) The validity of French VCE results in tertiary level language placement. Carrefours 11, 1: 27–30.
Burston, J. (1993b) The validity of French VCE results in tertiary level language placement: Part 2. Carrefours 11, 2: 42–47.
Burston, J., M. Monville-Burston and J. Harfouch (1994) Comparison of evaluation measures in first year students’ placement. Paper presented at the Second Conference of the Australian Society of French Studies Conference, Melbourne, 12–14 July 1994.
Burston J. and M. Monville-Burston (forthcoming) Practical design and implementation considerations of a computer-adaptive foreign language test: the Monash/Melbourne French CAT. CALICO Journal.
Hambleton, R. K. and L. L. Cook (1977) Latent trait models and their use in the analysis of educational test data. Journal of Educational Measurement 14,2:75–96.
Hatch, E. and A. Lazaraton. (1991) The research manual: Design and statistics for Applied Linguistics. New York, Newbury House.
Lord, F. M. (1980) Applications of item response theory to practical testing problems. Hillsdale, N.J., Lawrence Erlbaum.
Lunz, M. E. and B. A. Bergstrom (1991) Comparability of decisions for computer adaptive and written examinations. Journal of Allied Health 20,1:15–23.
Lunz, M. E., J. A. Stahl, and B. A. Bergstrom (1993) Targeting, test length, test precision and decision accuracy for computerized adaptive tests. Paper presented at the annual meeting of the American Educational Research Association, Atlanta, April 1993.
Rasch, G. (1960) Probabilistic model for some intelligence and attainment tests. Chicago, University of Chicago Press.
Traub, R. E. and R. G. Wolfe (1981) Latent trait theories and the assessment of educational achievement. In D. C. Berliner (ed.) Review of research in education. Washington, D.C., Educational Research Association.
Weiss, D. J. (1982) Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement 6,4:473–492.
Weiss, D. J. and G. G. Kinsbury (1984) Application of computerized adaptive testing to educational problems. Journal of Educational Measurement 21,4:361–375.
Wright, B. D. and M. H. Stone (1979) Best test design. Chicago, Ill., MESA.
Wright, B. D. and G. Masters (1982) Rating scale analysis. Chicago, Ill., MESA.