Cover not available
Article published In:
Information Design Journal
Vol. 28:1 (2023) ► pp.33–52

Language independent optimization of text readability formulas with deep reinforcement learning

| University of Kansas
ORCID logo | Institute for Humanities and Cultural Studies

Readability formulas are used to assess the level of difficulty of a text. These language dependent formulas are introduced with pre-defined parameters. Deep reinforcement learning models can be used for parameter optimization. In this article we argue that an Actor-Critic based model can be used to optimize the parameters in the readability formulas. Furthermore, a selection model is proposed for selecting the most suitable formula to assess the readability of the input text. English and Persian data sets are used for both training and testing. The experimental results of the parameter optimization model show that, on average, the F-score of the model for English increases from 24.7% in the baseline to 38.8%, and for Persian from 23.5% to 47.7%. The proposed algorithm selection model further improves the parameter optimization model to 65.5% based on F-score for both English and Persian.

Keywords: text readability, deep reinforcement learning, parameter optimization
Article outline
  • 1.Introduction
  • 2.Related Works
  • 3.Proposed method
    • 3.1Reinforcement learning
      • 3.1.1State space
      • 3.1.2Action space
      • 3.1.3Rewarding
    • 3.2Parameter optimization
    • 3.3Algorithm selection
    • 3.4Model training
  • 4.Experimental results
    • 4.1Dataset
      • 4.1.1English dataset
      • 4.1.2Persian dataset
    • 4.2Setup of experiments
    • 4.3Results and discussion
  • 5.Conclusion and future work
  • Notes
  • References
Published online: 25 May 2023
DOI logo
https://doi.org/10.1075/idj.22015.had
References
Al Qundus, J., Paschke, A., Gupta, S., Alzouby, A. M., & Yousef, M.
(2020) Exploring the impact of short-text complexity and structure on its quality in social media. Journal of Enterprise Information Management. DOI logoGoogle Scholar
Amstad, T.
(1978) Wie verständlich sind unsere zeitungen? [How Readable Are Our Newspapers?] Zurich, Switzerland: University of Zurich.Google Scholar
Antunes, H., & Lopes, C. T.
(2019) Analyzing the adequacy of readability indicators to a non-English language. International Conference of the Cross-Language Evaluation Forum for European Languages, (pp. 149–155). DOI logo
Azpiazu, I. M., & Pera, M. S.
(2019) Multiattentive recurrent neural network architecture for multilingual readability assessment. Transactions of the Association for Computational Linguistics, 71, 421–436. DOI logoGoogle Scholar
Balyan, R., McCarthy, K. S., & McNamara, D. S.
(2018) Comparing machine learning classification approaches for predicting expository text difficulty. Grantee Submission.Google Scholar
Bijankhan, M.
(2004) The role of corpora in writing a grammar: Introducing a software. Journal of Linguistics, 19(2), 48–67.Google Scholar
Bohnet, B.
(2009) Efficient parsing of syntactic and semantic dependency structures. Proceedings of the 13th Conference on Computational Natural Language Learning: Shared Task, (pp. 67–72). DOI logoGoogle Scholar
Cha, M., Gwon, Y., & Kung, H.
(2017) Language modeling by clustering with word embeddings for text readability assessment. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, (pp. 2003–2006). DOI logoGoogle Scholar
Cuayáhuitl, H., Lee, D., Ryu, S., Cho, Y., Choi, S., Indurthi, S., Yu, S., Choi, H., Hwang, I., & Kim, J.
(2019) Ensemble-based deep reinforcement learning for chatbots. Neurocomputing, 3661, 118–130. DOI logoGoogle Scholar
Dale, E., & Chall, J. S.
(1948) A formula for predicting readability: Instructions. Educational research bulletin, 37–54.Google Scholar
Dayani, M.
(2000) A criteria for assessing the Persian texts’ readability. Journal of Social Science and Humanities, 101, 35–48.Google Scholar
DuBay, W. H.
(2004) The principles of readability. Impact Information.Google Scholar
Dueppen, A. J., Bellon-Harn, M. L., Radhakrishnan, N., & Manchaiah, V.
(2019) Quality and readability of English-language internet information for voice disorders. Journal of Voice, 33(3), 290–296. DOI logoGoogle Scholar
Eslami, M., SharifiAtashgah, M., Lamjiri, S. A., & Zandi, T.
(2004) Persian productive lexicon. Proceedings of the 1st Workshop on the Persian Language and Computer.Google Scholar
Flesch, R.
(1979) How to Write Plain English: A Book for Lawyers and Consumers. Harper & Row.Google Scholar
(1948) A new readability yardstick. Journal of Applied Psychology, 32(3), 221. DOI logoGoogle Scholar
Franccois, T., & Fairon, C.
(2012) An “AI readability” formula for French as a foreign language. Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, (pp. 466–477).Google Scholar
Ghaffari, M., MahmoodiBakhtiyari, B., & Zolfaghari, H.
(2004) Let’s learn Persian (Vol. 1–3). Madreseh Publication.Google Scholar
Ghayoomi, M.
(2012) Bootstrapping the development of an HPSG-based treebank for Persian. Linguistic Issues in Language Technology, 7(1). DOI logoGoogle Scholar
(2013) Introducing a treebank and a statistical parser for Persian. Proceedings of the 8th Conference of Iranian Linguistics, 21, 666–679.Google Scholar
(2019) Transition from rule-based to statistical lemmatization in Persian. Proceedings of the 5th National Conference on Computational Linguistics, (pp. 57–86).Google Scholar
(2022) Application of computational linguistics to predict language proficiency level of Persian learners’ textbooks. Journal of Language Horizons, 6(1), 29–52. DOI logoGoogle Scholar
Ghayoomi, M., & Kuhn, J.
(2014) Converting an HPSG-based treebank into its parallel dependency-based treebank. Proceedings of the 9th International Conference on Language Resources and Evaluation, (pp. 802–809).Google Scholar
Goudjil, M., Koudil, M., Bedda, M., & Ghoggali, N.
(2018) A novel active learning method using svm for text classification. International Journal of Automation and Computing, 15(3), 290–298. DOI logoGoogle Scholar
Gunning, R.
(1952) The Technique of Clear Writing. McGraw-Hill.Google Scholar
Hafner, R., & Riedmiller, M.
(2011) Reinforcement learning in feedback control. Machine Learning, 84(1–2), 137–169. DOI logoGoogle Scholar
Hausknecht, M., & Stone, P.
(2015) Deep reinforcement learning in parameterized action space. arXiv preprint arXiv:1511.04143. [URL]
Jiang, Z., Gu, Q., Yin, Y., & Chen, D.
(2018) Enriching word embeddings with domain knowledge for readability assessment. Proceedings of the 27th International Conference on Computational Linguistics, (pp. 366–378).Google Scholar
Karačić, J., Dondio, P., Buljan, I., Hren, D., & Marušić, A.
(2019) Languages for different health information readers: Multitrait-multimethod content analysis of cochrane systematic reviews textual summary formats. BMC Medical Research Methodology, 19(1), 75. DOI logoGoogle Scholar
Keneshloo, Y., Ramakrishnan, N., & Reddy, C. K.
(2019) Deep transfer reinforcement learning for text summarization. Proceedings of the 2019 SIAM International Conference on Data Mining, 675–683. DOI logoGoogle Scholar
Kincaid, J. P., Fishburne Jr, R. P., Rogers, R. L., & Chissom, B. S.
(1975) Derivation of new readability formulas (automated readability index, Fog count and Flesch reading ease formula) for navy enlisted personnel. Technical Report. Naval Technical Training Command Millington TN Research Branch. DOI logoGoogle Scholar
Kingma, D. P., & Ba, J.
(2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [URL]
Klein, D., & Manning, C. D.
(2003) Accurate unlexicalized parsing. Proceedings of the 41st Meeting of the Association for Computational Linguistics, (pp. 423–430). DOI logoGoogle Scholar
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D.
(2015) Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971. [URL]
Lively, B. A., & Pressey, S. L.
(1923) A method for measuring the “vocabulary Burden” of textbooks. Educational Administration and Supervision, 91, 389–398.Google Scholar
Manek, A. S., Shenoy, P. D., Mohan, M. C., & Venugopal, K.
(2017) Aspect term extraction for sentiment analysis in large movie reviews using Gini index feature selection method and SVM classifier. World wide web, 20(2), 135–154. DOI logoGoogle Scholar
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J.
(2013) Distributed representations of words and phrases and their compositionality. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, & K. Q. Weinberger (Eds.), Advances in neural information processing systems 261 (pp. 3111–3119). Curran Associates, Inc. DOI logoGoogle Scholar
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M.
(2013) Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602. [URL]
Mohammadi, H., & Khasteh, S. H.
(2020) A machine learning approach to Persian text readability assessment using a crowdsourced dataset. 2020 28th Iranian Conference on Electrical Engineering (ICEE), 1–7. DOI logoGoogle Scholar
(2019) Text as environment: A deep reinforcement learning text readability assessment model. arXiv preprint arXiv:1912.05957. [URL]
Müller, T., Cotterell, R., Fraser, A., & Schütze, H.
(2015) Joint lemmatization and morphological tagging with lemming. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, (pp. 2268–2274). DOI logoGoogle Scholar
Müller, T., Schmid, H., & Schütze, H.
(2013) Efficient higher-order CRFs for morphological tagging. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, (pp. 322–332).Google Scholar
Narayan, S., Cohen, S. B., & Lapata, M.
(2018) Ranking sentences for extractive summarization with reinforcement learning. arXiv preprint arXiv:1802.08636. [URL]. DOI logo
Ngo-Ye, T. L., Sinha, A. P., & Sen, A.
(2017) Predicting the helpfulness of online reviews using a scripts-enriched text regression model. Expert Systems with Applications, 711, 98–110. DOI logoGoogle Scholar
Nuruzzaman, M., & Hussain, O. K.
(2018) A survey on chatbot implementation in customer service industry through deep neural networks. 2018 IEEE 15th International Conference on e-Business Engineering (ICEBE), (pp. 54–61). DOI logoGoogle Scholar
Pancer, E., Chandler, V., Poole, M., & Noseworthy, T. J.
(2019) How readability shapes social media engagement. Journal of Consumer Psychology, 29(2), 262–270. DOI logoGoogle Scholar
Poornamdariyan, T.
(1994) The Persian Lesson for Foreign Persian Learners (For Beginners). Institute for Humanities; Cultural Studies Publications.Google Scholar
Rottensteiner, S.
(2010) Structure, function and readability of new textbooks in relation to comprehension. Procedia-Social and Behavioral Sciences, 21, 3892–3898. DOI logoGoogle Scholar
SaffarMoghaddam, A.
(2003) General Persian: Basic constructions. Council of Extending Persian Language; Linguistics at the Institute for Humanities; Cultural Studies.Google Scholar
(2008) The Persian language (Vol. 1–4). Council of Extending Persian Language; Linguistics at the Institute for Humanities; Cultural Studies.Google Scholar
Salton, G. M., Wong, A., & Yang, C.-S.
(1975) A vector space model for automatic indexing. Communications of the ACM, 18(11), 613–620. DOI logoGoogle Scholar
Samareh, Y.
(1989) Teaching the Persian Language (Vol. 1). Alhoda International Publications.Google Scholar
(2005) Teaching the Persian Language (Vol. 2–4). Alhoda International Publications.Google Scholar
Senter, R., & Smith, E. A.
(1967) Automated readability index (tech. rep.). CINCINNATI UNIV OH.Google Scholar
Serban, I. V., Sankar, C., Germain, M., Zhang, S., Lin, Z., Subramanian, S., Kim, T., Pieper, M., Chandar, S., Ke, N. R., et al.
(2017) A deep reinforcement learning chatbot. arXiv preprint arXiv:1709.02349. [URL]
Shen, C., Gonzalez, Y., Chen, L., Jiang, S. B., & Jia, X.
(2018) Intelligent parameter tuning in optimization-based iterative ct reconstruction via deep reinforcement learning. IEEE transactions on medical imaging, 37(6), 1430–1439. DOI logoGoogle Scholar
Sherman, L.
(1893) Analytics of Literature: A Manual for the Objective Study of English Prose and Poetry. Ginn. [URL]
Silveira, N., Dozat, T., de Marneffe, M. C., Bowman, S., Connor, M., Bauer, J., & Manning, C. D.
(2014) A gold standard dependency corpus for English. Proceedings of the 9th International Conference on Language Resources and Evaluation, (pp. 2897–2904).Google Scholar
Song, S., Huang, H., & Ruan, T.
(2019) Abstractive text summarization using lstm-cnn based deep learning. Multimedia Tools and Applications, 78(1), 857–875. DOI logoGoogle Scholar
Sutton, R. S., & Barto, A. G.
(1998) Reinforcement Learning: An Introduction (Vol. 135). MIT press Cambridge. DOI logoGoogle Scholar
Temnikova, I., Vieweg, S., & Castillo, C.
(2015) The case for readability of crisis communications in social media. Proceedings of the 24th International Conference on World Wide Web, (pp. 1245–1250). DOI logoGoogle Scholar
Vajjala, S., & Lučić, I.
(2018) OnestopEnglish corpus: A new corpus for automatic readability assessment and text simplification. Proceedings of the 13th Workshop on Innovative Use of NLP for Building Educational Applications, (pp. 297–304). DOI logoGoogle Scholar
Wang, Y., & Jin, H.
(2019) A deep reinforcement learning based multi-step coarse to fine question answering (MSCQA) system. Proceedings of the AAAI Conference on Artificial Intelligence, 331, 7224–7232. DOI logoGoogle Scholar
Wasike, B.
(2018) Preaching to the choir? An analysis of newspaper readability vis-a-vis public literacy. Journalism, 19(11), 1570–1587. DOI logoGoogle Scholar
Watkins, C. J., & Dayan, P.
(1992) Q-learning. Machine Learning, 8(3–4), 279–292. DOI logoGoogle Scholar
Watkins, C. J. C. H.
(1989) Learning from Delayed Rewards. Doctoral Dissertation. King’s College. Cambridge, UK.
Xia, M., Kochmar, E., & Briscoe, T.
(2019) Text readability assessment for second language learners. arXiv preprint arXiv:1906.07580. [URL]
Zalmout, N., Saddiki, H., & Habash, N.
(2016) Analysis of foreign language teaching methods: An automatic readability approach. Proceedings of the 3rd workshop on natural language processing techniques for educational applications (NLPTEA2016), (pp. 122–130).Google Scholar
Zarghamiyan, M.
(1998) Series of Teaching the Persian Language (From Beginner to Advanced) (Vol. 1). Council of Extending Persian Language; Linguistics.Google Scholar
(2001) Series of Teaching the Persian Language (From Beginner to Advanced) (Vol. 2–3). Council of Extending Persian Language; Linguistics.Google Scholar