Cover not available
Article published In:
Journal of Immersion and Content-Based Language Education
Vol. 9:1 (2021) ► pp.4–30

Are CLIL texts too complicated?

A computational analysis of their linguistic characteristics

| University of the Basque Country (UPV/EHU)
| University of the Basque Country (UPV/EHU)
ORCID logo | University of the Basque Country (UPV/EHU)

This article presents a comparative study of the linguistic characteristics of some materials used to teach English as a foreign language, and Geography and History through English in a Content and Language Integrated Learning (CLIL) experience in the Basque Country with students aged 11–13. This paper analyzes and compares the contents of current textbooks using Coh-Metrix and AzterTest, which calculate stylistic and linguistic metrics regarding lexical and grammatical complexity, readability and coherence. Finally, the study suggests that there are significant differences mainly in vocabulary level, narrativity and cohesion, it identifies the potential difficulties of CLIL texts and offers advice on how to overcome them. Raising awareness of the complexity of some texts used in CLIL can provide a starting point for pedagogical adaptations and contribute to optimizing learning.

Keywords: CLIL, multilingual education, second language teaching, primary education, secondary education
Article outline
  • 1.Introduction
  • 2.Method
    • 2.1Participants
    • 2.2Characteristics of genres in History
    • 2.3Tools
  • 3.Results
    • 3.1Analysis of principal components in CLIL texts and texts for native English speakers
    • 3.2Analysis of principal components in ESL and CLIL texts
    • 3.3Differences between ESL and CLIL texts
      • 3.3.1Descriptive metrics
      • 3.3.2Lexical diversity
      • 3.3.3Word information
      • 3.3.4Polysemy and hypernymy
      • 3.3.5Morphological information
      • 3.3.6Syntactic complexity
      • 3.3.7Referential cohesion
      • 3.3.8Connectives (logical cohesion)
      • 3.3.9Causal/Intentional cohesion
  • 4.Discussion
  • 5.Conclusion
  • Notes
  • References
Published online: 27 October 2020
DOI logo
https://doi.org/10.1075/jicb.19022.agu
References
Anderson, A., Garrod, S. C., & Sanford, A. J.
(1983) The accessibility of pronominal antecedents as a function of episode shifts in narrative text. Quarterly Journal of Experimental Psychology, 35A1, 427–440. DOI logoGoogle Scholar
Alva-Manchego, F., Scarton, C., & Specia, L.
(2020) Data-driven sentence simplification: Survey and benchmark. Computational Linguistics, 1–87. DOI logoGoogle Scholar
Beacco, J. C. & Byram, M.
(2007) From linguistic diversity to plurilingual education: Guide for the development of language education policies in Europe. Council of Europe.Google Scholar
Bengoetxea, K., Gonzalez-Dios, I., & Aguirregoitia, A.
(2020) AzterTest: Open source linguistic and stylistic analysis tool. Procesamiento del Lenguaje Natural, 641, 1–8. DOI logoGoogle Scholar
Biber, D., Gray, B., & Poonpon, K.
(2011) Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? Tesol Quarterly, 45(1), 5–35. DOI logoGoogle Scholar
Biber, D., Johansson, S., Leech, G., Conrad, S., & Finegan, E.
(1999) Longman grammar spoken and written English. Longman.Google Scholar
Boroş, T., Dumitrescu, S. D., & Burtica, R.
(2018) NLP-Cube: End-to-end raw text processing with neural networks. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies (pp. 171–179). Association for Computational Linguistics. DOI logoGoogle Scholar
Cenoz, J.
(2009) Towards multilingual education: Basque educational research from an international perspective. Multilingual Matters. DOI logoGoogle Scholar
(2019) Translanguaging pedagogies and English as a lingua franca. Language Teaching, 52(1), 71–85. DOI logoGoogle Scholar
Coyle, D.
(2010) Foreword. In D. Lasagabaster & Y. Ruiz de Zarobe (Eds.) CLIL in Spain: Implementation, results and teacher training, pp. vii–viii. Cambridge Scholars Publishing. DOI logoGoogle Scholar
Crossley, S. A., Allen, D. B., & McNamara, D. S.
(2011) Text readability and intuitive simplification: A comparison of readability formulas. Reading in a foreign language, 23(1), 84–101.
https://doi.org/
Crossley, S. A., Greenfield, J., & McNamara, D. S.
(2008) Assessing text readability using cognitively based indices. Tesol Quarterly, 42(3), 475–493. DOI logoGoogle Scholar
Crossley, S. A., Louwerse, M. M., McCarthy, P. M., & McNamara, D. S.
(2007) A linguistic analysis of simplified and authentic texts. The Modern Language Journal, 91(1), 15–30. DOI logoGoogle Scholar
Cummins, J.
(2008) Teaching for transfer: Challenging the two solitudes assumption in bilingual education. Encyclopedia of language and education, 51, 65–75. DOI logoGoogle Scholar
(2009) Multilingualism in the English-language classroom: Pedagogical considerations. TESOL quarterly, 43(2), 317–321. DOI logoGoogle Scholar
Der, G., & Everitt, B. S.
(2008) A handbook of statistical analyses using SAS. Chapman and Hall CRC.. DOI logoGoogle Scholar
Eusko Jaurlaritza
(2010) Proceso de experimentación del marco de educación trilingüe. Documento marco, 2010–2011. Departamento de educación. [URL]
(2018) Matrikula: Bilakaera irudiak. [URL]
Franceschini, R.
(2013) “History of multilingualism.” In the Encyclopedia of Applied Linguistics, edited by C. A. Chapelle. 1–9. Blackwell Publishing.Google Scholar
Gierlinger, E. M.
(2017) Teaching CLIL? Journal of Immersion and Content-Based Language Education, 5(2), 187–213. DOI logoGoogle Scholar
Graesser, A. C., McNamara, D. S., & Kulikowich, J. M.
(2011) Coh-Metrix: Providing multilevel analyses of text characteristics. Educational researcher, 40(5), 223–234. DOI logoGoogle Scholar
Graesser, A. C., McNamara, D. S., Cai, Z., Conley, M., Li, H., & Pennebaker, J.
(2014) Coh-Metrix measures text characteristics at multiple levels of language and discourse. The Elementary School Journal, 115(2), 210–229. DOI logoGoogle Scholar
Gonzalez-Dios, I., Bengoetxea, K., & Aguirregoitia, A.
(2020) LagunTest: A NLP based application to enhance reading comprehension. In Proceedings of the 1st Workshop on Tools and Resources to Empower People with REAding Difficulties (READI), pp. 63–69. European Language Resources Association.Google Scholar
Heras, A., & Lasagabaster, D.
(2015) The impact of CLIL on affective factors and vocabulary learning. Language Teaching Research, 19(1), 70–88. DOI logoGoogle Scholar
Huguet, À., Lasagabaster, D., & Vila, I.
(2008) Bilingual education in Spain: Present realities and future challenges. Encyclopedia of language and education, 1672–1682. DOI logoGoogle Scholar
Just, M. A., & Carpenter, P. A.
(1987) The psychology of reading and language comprehension. Allyn & Bacon, Inc.Google Scholar
Kintsch, W.
(1998) Comprehension: A paradigm for cognition. Cambridge University Press.Google Scholar
Kintsch, W., & Van Dijk, T. A.
(1978) Toward a model of text comprehension and production. Psychological review, 85(5), 363–394. DOI logoGoogle Scholar
Llinares, A., & Morton, T.
(Eds.) (2017) Applied linguistics perspectives on CLIL. John Benjamins Publishing Company. DOI logoGoogle Scholar
Llinares, A., Morton, T., & Whittaker, R.
(2012) The roles of language in CLIL. Cambridge University Press.Google Scholar
Llinares, A. & Whittaker, R.
(2010) Writing and speaking in the history class. In C. Dalton-Puffer & T. Nikula (Eds.) Language use and language learning in CLIL classrooms, (pp.125–124). John Benjamin Publishing. DOI logoGoogle Scholar
Lorenzo, F.
(2008) Instructional discourse in bilingual settings. An empirical study of linguistic adjustments in content and language integrated learning. Language Learning Journal, 36(1), 21–33. DOI logoGoogle Scholar
(2013) Genre-based curricula: multilingual academic literacy in content and language integrated learning. International Journal of Bilingual Education and Bilingualism, 16(3), 375–388. DOI logoGoogle Scholar
Lorenzo, F., Granados, A. & Ávila, I.
(2019) The development of cognitive academic language proficiency in multilingual education: Evidence of a longitudinal study on the language of history. Journal of English for Academic Purposes, 411, 100767. DOI logoGoogle Scholar
López, L. E., Sichra, I.
(2008) Intercultural Bilingual Education Among Indigenous Peoples in Latin America. In Hornberger, N. H. (eds). Encyclopedia of Language and Education. Springer. DOI logoGoogle Scholar
Lyster, R., Ruiz de Zarobe, Y.
(2018) Introduction: instructional practices and teacher development in CLIL and immersion school settings. International Journal of Bilingual Education and Bilingualism, 21(3), 273–274. DOI logoGoogle Scholar
McCarthy, P. M., & Jarvis, S.
(2007) vocd: A theoretical and empirical evaluation. Language Testing, 24(4), 459–488. DOI logoGoogle Scholar
McCarthy, P. M.
(2005) An assessment of the range and usefulness of lexical diversity measures and the potential of the measure of textual, lexical diversity (MTLD) [Doctoral dissertation]. The University of Memphis.Google Scholar
McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z.
(2014) Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press. DOI logoGoogle Scholar
Mercer, N., Wegerif, R., & Major, L.
(2019) The Routledge international handbook of research on dialogic education. Routledge. DOI logoGoogle Scholar
Meyer, O.
(2010) Towards quality CLIL: Successful planning and teaching strategies. PULSO. Revista de Educación, 331, 11–29.Google Scholar
Mikolov, T., Grave, É., Bojanowski, P., Puhrsch, C., & Joulin, A.
(2018) Advances in pre-training distributed word representations. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pp 52–55. European Language Resources Association.Google Scholar
Nikula, T., & Moore, P.
(2019) Exploring translanguaging in CLIL. International Journal of Bilingual Education and Bilingualism, 22(2), 237–249. DOI logoGoogle Scholar
O’Reilly, T., & McNamara, D. S.
(2007) The impact of science knowledge, reading skill, and reading strategy knowledge on more traditional “high-stakes” measures of high school students’ science achievement. American Educational Research Journal, 44(1), 161–196. DOI logoGoogle Scholar
Otwinowska, A., & Foryś, M.
(2017) They learn the CLIL way, but do they like it? Affectivity and cognition in upper-primary CLIL classes. International Journal of Bilingual Education and Bilingualism, 20(5), 457–480. DOI logoGoogle Scholar
Qi, P., Dozat, T., Zhang, Y., & Manning, C.
2018 Universal Dependency Parsing from Scratch. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pp. 160–170. DOI logoGoogle Scholar
Ruiz de Zarobe, Y.
(2013) CLIL implementation: From policy-makers to individual initiatives. International Journal of Bilingual Education and Bilingualism, 161, 231–243. DOI logoGoogle Scholar
Saggion, H.
(2017) Automatic text simplification. Synthesis Lectures on Human Language Technologies, 10(1), 1–137. DOI logoGoogle Scholar
Schleppegrell, M. J.
(2004) The language of schooling: A functional linguistics perspective. Lawrence Erlbaum Associates. DOI logoGoogle Scholar
Snow, C.
(2002) Reading for understanding: Toward an R&D program in reading comprehension. Rand.Google Scholar
Speer, R., J. Chin, A. Lin, S. Jewett, & Nathan, L.
(2018) Luminosoinsight/wordfreq:v2.2. DOI logoGoogle Scholar
Whittaker, R., Llinares, A., & McCabe, A.
(2011) Written discourse development in CLIL at secondary school. Language Teaching Research, 15(3), 343–362. DOI logoGoogle Scholar
Zeno, S., Ivens, S. H., Millard, R. T., & Duvvuri, R.
(1995) The educator’s word frequency guide. Touchstone Applied Science Associates.Google Scholar