Chapter published in:
Learning to Read in a Digital World
Edited by Mirit Barzillai, Jenny Thomson, Sascha Schroeder and Paul van den Broek
[Studies in Written Language and Literacy 17] 2018
► pp. 205–224

Chapter 9
Digitisation of reading assessment

| University of Stavanger
| University of Stavanger
| University of Oslo
| University of Sheffield
As screen reading becomes the new standard, valid measures for capturing the defining features of reading ability as it moves from paper to screens must be developed. With the ongoing digitisation of many international and national large scale assessments, questions about the role of testing mode become especially pertinent. This chapter explores the question of how testing mode impacts the design of digital reading tests as well as children’s performance on them. We discuss how findings from empirical research on mode effects can inform the design of reading assessment and consider the pedagogical implications of a move to digital assessment.
Article outline
  • 1.Introduction
  • 2.Assessing skills digitally
  • 3.Digitisation of reading assessment: Cognitive considerations
  • 4.Digitisation of reading assessment: Navigating issues of text selection
  • 5.High stakes assessments: ILSA International assessment
    • 5.1The IEA-assessments PIRLS, ePIRLS
    • 5.2The OECD-assessments PISA and PIAAC
  • 6.Changing definitions, changing texts
  • 7.What large-scale assessment has taught us about mode-differences
    • 7.1Pedagogical implications 1: Designing assessments sensitive to reading across modes
    • 7.2Pedagogical implications 2: The influence of digitised reading assessment on classroom practice
  • 8.Concluding perspectives
  • References
Published online: 09 August 2018
https://doi.org/10.1075/swll.17.09sto
References
Ackerman, R., & Goldsmith, M.
(2011) Metacognitive regulation of text learning: On screen versus on paper. Journal of Experimental Psychology: Applied, 17(1), 18–32. CrossrefGoogle Scholar
Alexander, P. A., & The Disciplined Reading Learning Research Laboratory
(2012) Reading into the future: Competence for the 21st century. Educational Psychologist, 47(4), 259–280. CrossrefGoogle Scholar
Baccino, T.
(2004) La lecture électronique: De la vision à la compréhension. Grenoble: Presses Universitaires de Grenoble.Google Scholar
Baccino, T., & Pynte, J.
(1994) Spatial coding and discourse models during text reading. Language and Cognitive Processes, 9, 143–155. CrossrefGoogle Scholar
Ball, R., & Hourcade, J. P.
(2011) Rethinking reading for age from paper and computers. International Journal of Human-Computer Interaction, 27(11), 1066–1082. CrossrefGoogle Scholar
Baron, N. S.
(2015) Words onscreen: The fate of reading in a digital world. New York: Oxford University Press.Google Scholar
Baumert, J., Lüdtke, O., Trautwein, U., & Brunner M.
(2009) Large-scale student assessment studies measure the results of processes of knowledge acquisition: Evidence in support of the distinction between intelligence and student achievement. Educational Research Review, 4(3), 165–176. CrossrefGoogle Scholar
Benedetto, S., Drai-Zerbib, V., Pedrotti, M., Tissier, G., & Baccino, T.
(2013) E-readers and visual fatigue. PLoS ONE, 8(12), 1–7. CrossrefGoogle Scholar
Blehm, C., Vishnu, S., Khattak, A., Mitra, S., & Yee, R. W.
(2005) Computer vision syndrome: A review. Survey of Ophthalmology, 50(3), 253–262. CrossrefGoogle Scholar
Cataldo, M. G., & Oakhill, J.
(2000) Why are poor comprehenders inefficient searchers? An investigation into the effects of text. Journal of Educational Psychology, 92(4), 791–799. CrossrefGoogle Scholar
Cunningham, A. E., & Stanovich, K. E.
(1997) Early reading acquisition and its relation to reading experience and ability 10 years later. Developmental Psychology, 33(6), 934–945. CrossrefGoogle Scholar
Eyre, J., Berg, M., Mazengarb, J., & Lawes, E.
(2017) Mode equivalence in PAT: Reading Comprehension. Wellington: New Zealand Council for Educational Research.Google Scholar
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E.
(2014) Preparing for life in a digital age: The IEA International Computer and Information Literacy Study International Report: ICILS 2013 IEA. Australia: Springer Open.Google Scholar
Frønes, T. S., Narvhus, E. K., & Aasebø, M. C.
(2013) Nordic results from the PISA digital reading assessment. Nordic Journal of Digital Literacy, 1(2), 13–31.Google Scholar
Guthrie, J. T., Wigfield, A., Metsala, J. L., & Cox, K. E.
(1999) Motivational and cognitive predictors of text comprehension and reading amount. Scientific Studies of Reading, 3(3), 231–256. CrossrefGoogle Scholar
Haldane, S.
(2009) Delivery platforms for national and international computer-based surveys. In F. Scheuermann, & J. Björnsson (Eds.), The transition to computer-based assessment: New approaches to skills assessment and implications for large-scale testing (pp. 63–67). Luxembourg: Office for Official Publications of the European Communities.Google Scholar
Hopfenbeck, T., & Baird, J-A.
(2014) International tests. In J-A. Baird, T. N. Hopfenbeck, P. Newton, G. Stobart, & A. T. Steen-Utheim (Eds.), State of the field review: Assessment and learning (pp. 60–77). Lysaker: Norwegian Knowledge Centre for Education.Google Scholar
Jerrim, J.
(2016) PISA 2012: How do results for the paper and computer tests compare? Assessment in Education: Principles, Policy & Practice, 23(4), 495–518. CrossrefGoogle Scholar
Kerr, M. A., & Symons, S. E.
(2006) Computerized presentation of text: Effects on children’s reading of informational material. Reading and writing, 19(1), 1–19. CrossrefGoogle Scholar
Kintsch, W.
(1998) Comprehension: A paradigm for cognition. Cambridge: Cambridge University Press.Google Scholar
Kim, H., & Kim, J.
(2013) Reading from an LCD monitor versus paper: Teenagers’ reading performance. International Journal of Research Studies in Educational Technology (IJRSET), 2(1), 15–24. CrossrefGoogle Scholar
Kretzschmar, F., Pleimling, D., Hosemann, J., Füssel, S., Bornkessel-Schlesewsky, I., & Schlesewsky, M.
(2013) Subjective impressions do not mirror online reading effort: Concurrent EEG-eyetracking evidence from the reading of books and digital media. PLoS ONE, 8(2). CrossrefGoogle Scholar
Liu, Z.
(2005) Reading behavior in the digital environment: Changes in reading behavior over the past ten years. Journal of Documentation, 61(6), 700–712. CrossrefGoogle Scholar
Livingstone, S., Haddon, L., Vincent, J., Mascheroni, G., & Olafsson, K.
(2014) Net children go mobile: The UK report. London: London School of Economics and Political Science.Google Scholar
Mangen, A., Walgermo, B. R., & Brønnick, K.
(2013) Reading linear texts on paper vs. computer screens: Effects on reading comprehension. International Journal of Educational Research, 58, 61–68. CrossrefGoogle Scholar
Margolin, S. J., Driscoll, C., Toland, M. J., & Kegler, J. L.
(2013) E-readers, computer screens, or paper: Does reading comprehension change across media platforms? Applied Cognitive Psychology, 27(4), 512–519. CrossrefGoogle Scholar
Martin, R., & Binkley, M.
(2009) Gender differences in cognitive tests: A consequence of gender-dependent preferences for specific information presentation formats? In F. Scheuermann, & J. Björnsson (Eds.), The transition to computer-based assessment: New approaches to skills assessment and implications for large-scale testing (pp. 75–82). Luxembourg: Office for Official Publications of the European Communities.Google Scholar
Mehrens, W. A.
(1992) Using performance assessment for accountability purposes. Educational Measurement: Issues and Practice, 11(1), 3–9. CrossrefGoogle Scholar
Mullis, I. V. S., & Martin, M. O.
(Eds.) (2015) PIRLS 2016 assessment framework (2nd ed.). TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College and International Association for the Evaluation of Educational Achievement (IEA). Retrieved from https://​timssandpirls​.bc​.edu​/pirls2016​/framework​.html.Google Scholar
National Center for Educational Statistics
(2016) Digitally based assessments (DBA). Retrieved from https://​nces​.ed​.gov​/nationsreportcard​/dba/
Newton, P.
(2014) Validity and the cultivation of valuable learning. In J-A. Baird, T. N. Hopfenbeck, P. Newton, G. Stobart, A. T. Steen-Utheim (Eds.), State of the field review: Assessment and learning (pp. 78–99). Lysaker: Norwegian Knowledge Centre for Education.Google Scholar
Norman, E., & Furnes, B.
(2016) The relationship between metacognitive experiences and learning: Is there a difference between digital and non-digital study media? Computers in Human Behavior, 54, 301–309. CrossrefGoogle Scholar
OECD
(2013) The Survey of Adult Skills: Reader’s Companion. OECD Publishing.Google Scholar
(2015a) Students, Computers and Learning. Making the Connection. OECD Publishing. CrossrefGoogle Scholar
(2015b) PISA 2018 Draft Reading Literacy Framework. 40th meeting of the PISA Governing Board.Google Scholar
(2016a) PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic and Financial Literacy. Paris: PISA, OECD Publishing.CrossrefGoogle Scholar
(2016b) PISA 2015 Results (Volume I): Excellence and Equity in Education. Paris: OECD Publishing.CrossrefGoogle Scholar
Pfost, M., Dörfler, T., & Artelt, C.
(2013) Students’ extracurricular reading behaviour and the development of vocabulary and reading comprehension. Learning and Individual Differences, 26, 89–102. CrossrefGoogle Scholar
Piolat, A., Roussey, J.-Y., & Thunin, O.
(1997) Effects of screen presentation on text reading and revising. International Journal of Human-Computer Studies, 47(4), 565–589. CrossrefGoogle Scholar
Porion, A., Aparicio, X., Megalakaki, O., Robert, A., & Baccino, T.
(2016) The impact of paper-based versus computerized presentation on text comprehension and memorization. Computers in Human Behavior, 54, 569–576. CrossrefGoogle Scholar
Rasmusson, M.
(2015) Reading paper – reading screen. Nordic Studies in Education, 35, 3–19.Google Scholar
Rockinson-Szapkiw, A. J., Courduff, J., Carter, K., & Bennett, D.
(2013) Electronic versus traditional print textbooks: A comparison study on the influence of university students’ learning. Computers & Education, 63, 259–266. CrossrefGoogle Scholar
Russel, M., Goldberg, A., & O’connor, K.
(2003) Computer-based testing and validity: A look back into the future. Assessment in Education: Principles, Policy & Practice, 10(3), 279–293. CrossrefGoogle Scholar
Sanchez, C. A., & Wiley, J.
(2009) To scroll or not to scroll: Scrolling, working memory capacity, and comprehending complex texts. Human Factors: The Journal of the Human Factors and Ergonomics Society, 51(5), 730–738. CrossrefGoogle Scholar
Scheuermann, F., & Björnsson, J.
(Eds.) (2009) The transition to computer-based assessment: New approaches to skills assessment and implications for large-scale testing. Luxembourg: Office for Official Publications of the European Communities.Google Scholar
Singer, L. M., & Alexander, P. A.
(2016) Reading across mediums: Effects of reading digital and print texts on comprehension and calibration. The Journal of Experimental Education, 85(1), 155–172. CrossrefGoogle Scholar
Solheim, R. G.
(2013) PIRLS 2011 – oppbygging og metode. In E. Gabrielsen, & R. G. Solheim (Eds.), Over kneiken? Leseferdighet på 4. og 5. trinn i et tiårsperspektiv (pp. 13–25). Oslo: Akademika.Google Scholar
Støle, H., Mangen, A., & Schwippert, K.
in prep.) Digitisation of 5th grade national reading assessment in Norway: A mode-effect study.
Wagemaker, H.
(2014) International large-scale assessments: From research to policy. In L. Rutkowski, M. von Davie, & D. Rutkowski (Eds.), Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis (pp.11–36). Boca Raton: Taylor & Francis.Google Scholar
Walgermo, B. R., Mangen, A., & Brønnick, K.
(2013) Lesing av sammenhengende tekster på skjerm og papir: Apropos digitalisering av leseprøver. Conference paper presentation, Skriv! Les! Trondheim, Norway, May 6–8.
Wolf, M.
(2016) Tales of literacy for the 21st century. Oxford: Oxford University Press.Google Scholar
Wolf, M., & Barzillai, M.
(2009) The importance of deep reading. Educational Leadership, Literacy 2.0, 6(66), 32–37.Google Scholar
Wästlund, E.
(2007) Experimental studies of human-computer interaction: Working memory and mental workload in complex cognition (Doctoral dissertation). University of Gothenburg, Sweden.
Wixson, K. K., & Carlisle, J. F.
(2005) The influence of large-scale assessment of reading comprehension on classroom practice: A commentary. In S. G. Paris, & S. A. Stahl (Eds.), Children’s reading comprehension and assessment (pp. 395–405). Mahwah, New Jersey and London: Lawrence Erlbaum Ass.Google Scholar
Cited by

Cited by 2 other publications

Säuberli, Andreas
2021.  In The 23rd International ACM SIGACCESS Conference on Computers and Accessibility, ► pp. 1 ff. Crossref logo
Weyergang, Cecilie & Camilla G. Magnusson
2020.  In Like muligheter til god leseforståelse?, ► pp. 46 ff. Crossref logo

This list is based on CrossRef data as of 31 march 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.